Parallel algorithm based on singular value decomposition for high performance training of neural networks

Neural Networks (NNs) are frequently applied to Multi Input Multi Output (MIMO) problems, where the amount of data to manage is extremely high and, hence, the computational time required for the training process is too large. Therefore, MIMO problems are often split into Multi Input Single Output (MISO) problems; MISOs are further decomposed into several Single Input Single Output (SISO) problems. The aim of this paper is to present an optimized approach for NNs training based on properties of Singular Value Decomposition (SVD), allowing to decompose the MISO NN into a collection of SISO NNs. The decomposition provides a two-fold advantage: firstly, each SISO NN can be trained by using a one-dimensional function, namely a limited dataset, and then a parallel architecture can be implemented on a PC-cluster, decreasing the computational cost. The parallel algorithm performance are validated by using magnetic hysteresis dataset with the aim to prove the computational speed up by preserving the accuracy.