Mechanism to perform non-linear functions in a machine learning accelerator

1. An apparatus comprising accelerator circuitry, including:

a compute grid having a plurality of processing elements to execute neural network computations, store values resulting from the neural network computations, and perform piecewise linear (PWL) approximations of one or more non-linear functions using the stored values as input data, including:

a first stage including a multiplier to perform a multiplication operation of an input data value and a first version of a first parameter value associated with a first interval during a first processing cycle of a PWL approximation;

a second stage including an accumulator to perform a subtraction operation of a first version of a second parameter associated with the first interval from a result of the multiplication operation during the first processing cycle of the PWL approximation and perform an addition operation of a first version of a third parameter associated with the first interval and the result of the subtraction operation during a second processing cycle of the PWL approximation; and

a data selector coupled to the second stage to select the first version of the second parameter associated with the first interval to be received at the second stage during the first processing cycle and select the result of the subtraction operation to be received at the second stage during the second processing cycle.

2. The apparatus of claim 1 , further comprising one or more lookup tables (LUTs) to store a plurality of non-linear parameter values for the PWL approximations.

3. The apparatus of claim 2 , wherein the accumulator sets an enable bit indicating a sign of the result of the subtraction operation.

4. The apparatus of claim 3 , wherein a determination is made as to whether the enable bit indicates that the sign of the result of the subtraction operation has a first value.

5. The apparatus of claim 4 , wherein the multiplier performs a second multiplication operation of the input data value and a second version of the first parameter value associated with a second interval during a third processing cycle of the PWL approximation upon a determination that the enable bit indicates that the sign has the first value, and the accumulator performs a second subtraction operation of a second version of the second parameter associated with the second interval from a result of the second multiplication operation.

6. The apparatus of claim 4 , wherein a result of the addition operation is saved as the PWL approximation upon a determination that the sign has a second value.

7. A method comprising:

executing neural network computations at a compute grid having a plurality of processing elements;

storing values resulting from the neural network computations; and

performing piecewise linear (PWL) approximations of one or more non-linear functions using the stored values as input data, including:

performing a first processing cycle, comprising:

performing a multiplication operation at a first stage to multiply an input data value and a first version of a first parameter value associated with a first interval;

selecting a first version of a second parameter associated with the first interval to be received at the second stage; and

performing a subtraction operation at a second stage of the first version of the second parameter associated with the first interval from the result of the multiplication operation during the first processing cycle of a PWL approximation; and

performing a second processing cycle, including:

selecting the result of the subtraction operation; and

performing an addition operation at the second stage of a first version of a third parameter associated with the first interval and the result of the subtraction operation during the second processing cycle of the PWL approximation.

8. The method of claim 7 , wherein performing the first processing cycle further comprises setting an enable bit indicating a sign of the result of the subtraction operation.

9. The method of claim 8 , further comprising determining whether the enable bit indicates that the sign of the result of the subtraction operation has a first value.

10. The method of claim 9 , further comprising performing a third processing cycle, including:

performing a second multiplication operation of the input data value and a second version of the first parameter value associated with a second interval upon a determination that the enable bit indicates that the sign has the first value; and

performing a second subtraction operation of a second version of the second parameter associated with the second interval from the result of the second multiplication operation.

11. The method of claim 9 , further comprising saving a result of the addition operation upon a determination that the sign has a second value.

12. An accelerator comprising:

one or more memory devices to store one or more one or more lookup tables (LUTs); and

a compute grid having a plurality of processing elements to execute neural network computations, store values resulting from the neural network computations, and perform piecewise linear (PWL) approximations of one or more non-linear functions using the stored values as input data, including:

a first stage including a multiplier to perform a multiplication operation of an input data value and a first version of a first parameter value associated with a first interval during a first processing cycle of the PWL approximation;

a second stage including an accumulator to perform a subtraction operation of a first version of a second parameter associated with the first interval from a result of the multiplication operation during the first processing cycle of a PWL approximation and perform an addition operation of a first version of a third parameter associated with the first interval and the result of the subtraction operation during a second processing cycle of the PWL approximation; and

a data selector coupled to the second stage to select the first version of the second parameter associated with the first interval to be received at the second stage during the first processing cycle and select the result of the subtraction operation to be received at the second stage during the second processing cycle.

13. The accelerator of claim 12 , wherein the accumulator sets an enable bit indicating a sign of the result of the subtraction operation.

14. The accelerator of claim 13 , wherein a determination is made as to whether the enable bit indicates that the sign of the result of the subtraction operation has a first value.

15. The accelerator of claim 14 , wherein the multiplier performs a second multiplication operation of the input data value and a second version of the first parameter value associated with a second interval during a third processing cycle of the PWL approximation upon a determination that the enable bit indicates that the sign has the first value, and the accumulator performs a second subtraction operation of a second version of the second parameter associated with the second interval from a result of the second multiplication operation.

16. The accelerator of claim 14 , wherein a result of the addition operation is saved as the PWL approximation upon a determination that the sign has a second value.