Approximation of matrices for matrix multiply operations
A processing device is provided which comprises memory configured to store data and a processor configured to receive a portion of data of a first matrix comprising a first plurality of elements and receive a portion of data of a second matrix comprising a second plurality of elements. The processor is also configured to determine values for a third matrix by dropping a number of products from products of pairs of elements of the first and second matrices based on approximating the products of the pairs of elements as a sum of the exponents of the pairs of elements and performing matrix multiplication on remaining products of the pairs of elements of the first and second matrices.
1. A processing device that improves computational efficiency of a computing system, the processing device comprising:
memory configured to store data; and
a processor that is communicatively coupled to the memory, wherein the processor is configured to:
receive a portion of data of a first matrix and a portion of data of a second matrix, wherein the first matrix comprises a first plurality of elements and the second matrix comprises a second plurality of elements;
calculate values for a third matrix, wherein the values of the third matrix are determined by:
dropping a target number of products, determined at run time, from products of pairs of unsorted elements of the first matrix and unsorted elements of the second matrix based on approximating the products of the pairs of elements as a sum of values of bits of corresponding significance of exponents of the pairs of elements; and
performing matrix multiplication on remaining products of the pairs of elements of the first matrix and the second matrix.
2. The processing device according to claim 1 , wherein the products having the smallest sum of the exponents are dropped.
3. The processing device according to claim 2 , further comprising a plurality of multiplier accumulators (MACs), and
wherein the target number reduces a number of MACs used to perform the matrix multiplication.
4. The processing device according to claim 1 , wherein the processor is configured to determine which products are to be used to perform the matrix multiplication by:
summing values of bits of corresponding significance of the exponents of the product, starting with the most significant bits for the exponents of the product, and
comparing each result of the summing to the target number until the target number of product exponent values is determined.
5. The processing device according to claim 1 , wherein the first plurality of elements and the second plurality of elements are in an integer data type format.
6. The processing device according to claim 1 , wherein the processor is configured to extract the exponents of the product by:
determining absolute values of the first plurality of elements and the second plurality of elements;
determining, for each element, a number of leading zeros, and
approximating, for each element, an exponent value as a difference between a number of element bits −1 and the number of leading zeros of the element.
7. The processing device according to claim 6 , wherein the processor is configured to further extract the exponents of the product by representing each element as 1.M*2 e , where M is a mantissa and e is the exponent value.
8. The processing device according to claim 1 , wherein the first plurality of elements and the second plurality of elements are in a float data type format.
9. The processing device according to claim 1 , further comprising a display device,
wherein information generated from the matrix multiplication is displayed on the display device.
10. The processing device of claim 1 , wherein the computing system is a machine learning network and the third matrix is utilized to train the machine learning network.
11. The processing device of claim 10 , wherein the target number is determined based on an effect the approximating will have on an accuracy of the machine learning network.
12. A method that improves computational efficiency of a computing system, the method comprising:
receiving a portion of data of a first matrix and a portion of data of a second matrix, wherein the first matrix comprises a first plurality of elements and the second matrix comprises a second plurality of elements;
calculating values for a third matrix, wherein the values of the third matrix are determined by:
dropping a target number of products, determined at run time, from products of pairs of unsorted elements of the first matrix and unsorted elements of the second matrix based on approximating the products of the pairs of elements as a sum of values of bits of corresponding significance of exponents of the pairs of elements; and
performing matrix multiplication on remaining products of the pairs of elements of the first matrix and the second matrix.
13. The method according to claim 12 , wherein the products having the smallest sum of the exponents are dropped.
14. The method according to claim 12 , further comprising determining which products are to be used to perform the matrix multiplication by:
summing values of bits of corresponding significance of the exponents of the product, starting with the most significant bits for the exponents of the product, and
comparing each result of the summing to the target number until the target number of product exponent values is determined.
15. The method according to claim 12 , wherein the first plurality of elements and the second plurality of elements are in an integer data type format.
16. The method according to claim 15 , further comprising extracting the exponents of the product by:
determining absolute values of the first plurality of elements and the second plurality of elements;
determining, for each element, a number of leading zeros, and
approximating, for each element, an exponent value as a difference between a number of element bits −1 and the number of leading zeros of the element.
17. The method according to claim 16 , further comprising extracting the exponents by representing each element as 1.M*2 e , where M is a mantissa and e is the exponent value.
18. The method according to claim 12 , wherein:
the computing system is a machine learning network and the third matrix is utilized to train the machine learning network, and
the target number is determined based on an effect the approximating will have on an accuracy of the machine learning network.
19. A non-transitory computer readable medium comprising instructions for improving computational of a computing system, the instructions when executed by a processor cause the processor to execute a method comprising:
receiving a portion of data of a first matrix and a portion of data of a second matrix, wherein the first matrix comprises a first plurality of elements and the second matrix comprises a second plurality of elements;
calculating values for a third matrix, wherein the values of the third matrix are determined by:
dropping a target number of products, determined at runtime, from products of pairs of unsorted elements of the first matrix and unsorted elements of the second matrix based on approximating the products of the pairs of elements as a sum of values of bits of corresponding significance of exponents of the pairs of elements; and
performing matrix multiplication on remaining products of the pairs of elements of the first matrix and the second matrix.
20. The non-transitory computer readable medium of claim 19 , wherein the products having the smallest sum of the exponents are dropped.
21. The non-transitory computer readable medium of claim 19 , wherein the computing system is a machine learning network and the third matrix is utilized to train the machine learning network.