Deep compressed network
An embodiment of a semiconductor package apparatus may include technology to apply a low rank factorization to a weight matrix of a decision network to determine a first weight matrix approximation, reshape the first weight matrix approximation into a second weight matrix approximation, and compress the decision network based on the second weight matrix approximation. Other embodiments are disclosed and claimed.
1. An electronic processing system, comprising:
a processor;
memory communicatively coupled to the processor; and
a decision network communicatively coupled to the processor and the memory, the decision network including logic to:
apply a low rank factorization to a weight matrix of the decision network to determine a first weight matrix approximation,
reshape the first weight matrix approximation into a second weight matrix approximation,
compress the decision network based on the second weight matrix approximation,
read a row of data from the first weight matrix approximation in a row-major order, wherein a number of elements in the row is a first number,
write the row of data from the first weight matrix approximation into a column of the second weight matrix approximation in a column-major order, wherein a number of elements in the column is a second number, wherein the second number is different from the first number, and
generate a layer of the decision network based on the second weight matrix approximation and through a removal of one or more biases based on the row of data from the first weight matrix approximation being written into the column of the second weight matrix approximation in the column-major order.
2. The system of claim 1 , wherein the logic is further to:
reshape the first weight matrix approximation into the second weight matrix approximation to add nonlinearity to the second weight matrix approximation.
3. The system of claim 2 , wherein the logic is further to:
scatter data from the first weight matrix approximation into the second weight matrix approximation.
4. The system of claim 2 , wherein the logic is further to:
alter a layout of data from the first weight matrix approximation into the second weight matrix approximation.
5. The system of claim 1 , wherein the decision network comprises one or more of a convolutional neural network, a deep neural network, and a recurrent neural network.
6. A semiconductor package apparatus, comprising:
one or more substrates; and
logic coupled to the one or more substrates, wherein the logic is at least partly implemented in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates to:
apply a low rank factorization to a weight matrix of a decision network to determine a first weight matrix approximation,
reshape the first weight matrix approximation into a second weight matrix approximation,
compress the decision network based on the second weight matrix approximation,
read a row of data from the first weight matrix approximation in a row-major order, wherein a number of elements in the row is a first number,
write the row of data from the first weight matrix approximation into a column of the second weight matrix approximation in a column-major order, wherein a number of elements in the column is a second number, wherein the second number is different from the first number, and
generate a layer of the decision network based on the second weight matrix approximation and through a removal of one or more biases based on the row of data from the first weight matrix approximation being written into the column of the second weight matrix approximation in the column-major order.
7. The apparatus of claim 6 , wherein the logic is further to:
reshape the first weight matrix approximation into the second weight matrix approximation to add nonlinearity to the second weight matrix approximation.
8. The apparatus of claim 7 , wherein the logic is further to:
scatter data from the first weight matrix approximation into the second weight matrix approximation.
9. The apparatus of claim 7 , wherein the logic is further to:
alter a layout of data from the first weight matrix approximation into the second weight matrix approximation.
10. The apparatus of claim 6 , wherein the decision network comprises one or more of a convolutional neural network, a deep neural network, and a recurrent neural network.
11. The apparatus of claim 6 , wherein the logic coupled to the one or more substrates includes transistor channel regions that are positioned within the one or more substrates.
12. A method comprising:
applying a low rank factorization to a weight matrix of a decision network to determine a first weight matrix approximation;
reshaping the first weight matrix approximation into a second weight matrix approximation;
compressing the decision network based on the second weight matrix approximation;
reading a row of data from the first weight matrix approximation in a row-major order, wherein a number of elements in the row is a first number;
writing the row of data from the first weight matrix approximation into a column of the second weight matrix approximation in a column-major order, wherein a number of elements in the column is a second number, wherein the second number is different from the first number; and
generating a layer of the decision network based on the second weight matrix approximation and through a removal of one or more biases based on the row of data from the first weight matrix approximation being written into the column of the second weight matrix approximation in the column-major order.
13. The method of claim 12 , further comprising:
reshaping the first weight matrix approximation into the second weight matrix approximation to add nonlinearity to the second weight matrix approximation.
14. The method of claim 13 , further comprising:
scattering data from the first weight matrix approximation into the second weight matrix approximation.
15. The method of claim 13 , further comprising:
altering a layout of data from the first weight matrix approximation into the second weight matrix approximation.
16. The method of claim 12 , wherein the decision network comprises one or more of a convolutional neural network, a deep neural network, and a recurrent neural network.
17. At least one computer readable storage medium, comprising a set of instructions, which when executed by a computing device, cause the computing device to:
apply a low rank factorization to a weight matrix of a decision network to determine a first weight matrix approximation;
reshape the first weight matrix approximation into a second weight matrix approximation;
compress the decision network based on the second weight matrix approximation;
read a row of data from the first weight matrix approximation in a row-major order, wherein a number of elements in the row is a first number;
write the row of data from the first weight matrix approximation into a column of the second weight matrix approximation in a column-major order, wherein a number of elements in the column is a second number, wherein the second number is different from the first number; and
generate a layer of the decision network based on the second weight matrix approximation and through a removal of one or more biases based on the row of data from the first weight matrix approximation being written into the column of the second weight matrix approximation in the column-major order.
18. The at least one computer readable storage medium of claim 17 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to:
reshape the first weight matrix approximation into the second weight matrix approximation to add nonlinearity to the second weight matrix approximation.
19. The at least one computer readable storage medium of claim 18 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to:
scatter data from the first weight matrix approximation into the second weight matrix approximation.
20. The at least one computer readable storage medium of claim 18 , comprising a further set of instructions, which when executed by the computing device, cause the computing device to:
alter a layout of data from the first weight matrix approximation into the second weight matrix approximation.
21. The at least one computer readable storage medium of claim 17 , wherein the decision network comprises one or more of a convolutional neural network, a deep neural network, and a recurrent neural network.