Optimization of neural networks using hardware calculation efficiency and adjustment factors
In one embodiment, a method includes receiving a request for an operation to be performed; determining that the operation is associated with a machine-learning algorithm, and in response, route the operation to a computing circuit; performing, at the computing circuit, the operation, including: determining a linear domain product of a first log-domain number and a second log-domain number associated with the operation based on a summation of the first log-domain number and the second log-domain number and output a third log-domain number approximating the linear domain product of the first log-domain number and the second log-domain number; converting the third log-domain number to a first linear-domain number; summing the first linear-domain number and a second linear-domain number associated with the operation, and output a third linear-domain number as the summed result.
1. A system, comprising:
a processor configured to receive a request for an operation to be performed, determine that the operation is associated with a machine-learning algorithm, and in response, route the operation to a computing circuit configured to perform the operation, wherein the computing circuit comprises:
a conversion computing module configured to receive a log-domain number, and convert the log-domain number to a first linear-domain number by identifying a correspondence between the log-domain number and the first linear-domain number as indicated by a look-up table (LUT);
an exponent adjustment computing module configured to determine a first adjustment factor to apply to a first linear-domain number to generate an adjusted first linear-domain number such that the adjusted first linear-domain number is able to be encoded in a number of bits of memory, the memory storing linear-domain numbers, and
a summation computing module configured to determine, as a third linear-domain number, a summation of the adjusted first linear-domain number and a second linear-domain number,
wherein the exponent adjustment computing module is further configured to determine a third adjustment factor to apply to the third linear-domain number to generate an adjusted third linear-domain number such that the adjusted third linear-domain number is able to be encoded in the number of bits of memory.
2. The system of claim 1 , wherein the first linear-domain number is a floating-point number.
3. The system of claim 1 , wherein the first adjustment factor and/or the third adjustment factor has a negative or a positive value.
4. The system of claim 1 , wherein the summation computing module is a Kulisch accumulator.
5. The system of claim 2 , wherein the conversion computing module is further configured to convert the adjusted first linear-domain number to a fixed-point number prior to the summation.
6. The system of claim 1 , wherein the summation computing module is a floating-point accumulator.
7. The system of claim 1 , wherein the second linear-domain number is associated with the operation.
8. A computer-implemented method, comprising:
receiving a request for an operation to be performed;
determining that the operation is associated with a machine-learning algorithm, and in response, routing the operation to a computing circuit; and
performing the operation at the computing circuit by:
receiving a log-domain number, and converting the log-domain number to a first linear-domain number by identifying a correspondence between the log-domain number and the first linear-domain number as indicated by a look-up table (LUT);
determining a first adjustment factor to apply to a first linear-domain number to generate an adjusted first linear-domain number such that the adjusted first linear-domain number is able to be encoded in a number of bits of memory, the memory storing linear-domain numbers;
determining, as a third linear-domain number, a summation of the adjusted first linear-domain number and a second linear-domain number;
determining a third adjustment factor to apply to the third linear-domain number to generate an adjusted third linear-domain number such that the adjusted third linear-domain number is able to be encoded in the number of bits of memory; and
outputting the adjusted third linear-domain number.
9. The computer-implemented method of claim 8 , wherein the first linear-domain number is a floating-point number.
10. The computer-implemented method of claim 8 , wherein the first adjustment factor and/or the third adjustment factor has a negative or a positive value.
11. The computer-implemented method of claim 8 , wherein the summation of the adjusted first linear-domain number and the second linear-domain number is determined using a Kulisch accumulator.
12. The computer-implemented method of claim 9 , further comprising converting the adjusted first linear-domain number to a fixed-point number prior to the summation.
13. The computer-implemented method of claim 8 , wherein the second linear-domain number is associated with the operation.
14. The computer-implemented method of claim 8 , wherein the summation of the adjusted first linear-domain number and the second linear-domain number is determined using a floating-point accumulator.
15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receive a log-domain number, and converting the log-domain number to a first linear-domain number by identifying a correspondence between the log-domain number and the first linear-domain number as indicated by a look-up table (LUT);
determine a first adjustment factor to apply to a first linear-domain number to generate an adjusted first linear-domain number such that the adjusted first linear-domain number is able to be encoded in a number of bits of memory, the memory storing linear-domain numbers;
determine, as a third linear-domain number, a summation of the adjusted first linear-domain number and a second linear-domain number;
determine a third adjustment factor to apply to the third linear-domain number to generate an adjusted third linear-domain number such that the adjusted third linear-domain number is able to be encoded in the number of bits of memory; and
output the adjusted third linear-domain number.
16. The computer-readable medium of claim 15 , wherein the first linear-domain number is a floating-point number.
17. The computer-readable medium of claim 15 , wherein the first adjustment factor and/or the third adjustment factor has a negative or a positive value.
18. The computer-readable medium of claim 15 , wherein the summation of the adjusted first linear-domain number and the second linear-domain number is determined using a Kulisch accumulator.
19. The computer-readable medium of claim 16 , the operations further comprising convert the adjusted first linear-domain number to a fixed-point number prior to the summation.
20. The computer-readable medium of claim 15 , wherein the summation of the adjusted first linear-domain number and the second linear-domain number is determined using a floating-point accumulator.