REDUCTION OF PARAMETERS IN FULLY CONNECTED LAYERS OF NEURAL NETWORKS BY LOW RANK FACTORIZATIONS

The present disclosure is drawn to the reduction of parameters in fully connected layers of neural networks. For a layer whose output is defined by y=Wx, where y∈Rm is the output vector, x∈Rn is the input vector, and W∈Rm×n is a matrix of connection parameters, matrices Uij and Vij are defined and submatrices Wij are computed as the product of Uij and Vij, so that Wij=VijUij, and W is obtained by appending submatrices Wi,j.