Auto-configuration of hardware non-linear function acceleration

1. A computer-implemented method for configuring a hardware accelerator to perform a non-linear function, comprising:

determining a plurality of intervals that partition an input domain of the non-linear function;

determining, for each of the intervals, a plurality of subinterval configurations corresponding to different numbers of subintervals for partitioning that interval;

generating an error set comprising, for each of the subinterval configurations of each of the intervals, an error for using a polynomial function to approximate the non-linear function within one or more corresponding subintervals specified by the subinterval configuration;

using the error set,

selecting one of the subinterval configurations for each of the intervals to generate a configuration set that minimizes a worst-case error across the intervals without exceeding resource constraints of the hardware accelerator; and

selecting one of the subinterval configurations for each of the intervals to generate an improved configuration set that minimizes a cumulative error across the intervals without exceeding the worst-case error; and

configuring the hardware accelerator based on the improved configuration set.

2. The method of claim 1 , wherein:

the plurality of intervals that partition the input domain of the non-linear function are disjoint binary intervals; and

the one or more corresponding subintervals respectively specified by each of the subinterval configurations are of equal size.

3. The method of claim 1 , further comprising:

storing, in an index look-up-table, for each of the intervals, the selected one of the subinterval configurations used to generate the improved configuration set; and

storing, in a main look-up-table, for each subinterval specified by the subinterval configurations stored in the index look-up-table, a set of polynomial function coefficients defined by the improved configuration set;

wherein the resource constraints define a maximum index look-up-table size and a maximum main look-up-table size.

4. The method of claim 1 , wherein the non-linear function comprises one of a sigmoid function, a Swish function, a hyperbolic tangent (Tanh) function, a Gaussian Error Linear Unit (GELU) function, or a rectifier function.

5. The method of claim 1 , wherein the resource constraints of the hardware accelerator scale linearly with a number of parameters of the polynomial function.

6. The method of claim 1 , further comprising receiving information defining a division configuration space and configuration costs, the received information including the resource constraints.

7. The method of claim 3 , wherein the index look-up-table and the main look-up-table are stored in a special function unit of a processing element of an application-specific integrated circuit (ASIC).

8. The method of claim 6 , wherein:

the error set is generated using linear programming;

the configuration set is generated using binary search; and

the improved configuration set is generated using dynamic programming.

9. One or more computer-readable non-transitory storage media embodying software that is operable when executed to perform a method, the method comprising:

determining a plurality of intervals that partition an input domain of a non-linear function;

determining, for each of the intervals, a plurality of subinterval configurations corresponding to different numbers of subintervals for partitioning that interval;

generating an error set comprising, for each of the subinterval configurations of each of the intervals, an error for using a polynomial function to approximate the non-linear function within one or more corresponding subintervals specified by the subinterval configuration;

using the error set,

selecting one of the subinterval configurations for each of the intervals to generate a configuration set that minimizes a worst-case error across the intervals without exceeding resource constraints of a hardware accelerator; and

selecting one of the subinterval configurations for each of the intervals to generate an improved configuration set that minimizes a cumulative error across the intervals without exceeding the worst-case error; and

configuring the hardware accelerator based on the improved configuration set.

10. The media of claim 9 , wherein:

the plurality of intervals that partition the input domain of the non-linear function are disjoint binary intervals; and

the one or more corresponding subintervals respectively specified by each of the subinterval configurations are of equal size.

11. The media of claim 9 , the method further comprising:

storing, in an index look-up-table, for each of the intervals, the selected one of the subinterval configurations used to generate the improved configuration set; and

storing, in a main look-up-table, for each subinterval specified by the subinterval configurations stored in the index look-up-table, a set of polynomial function coefficients defined by the improved configuration set;

wherein the resource constraints define a maximum index look-up-table size and a maximum main look-up-table size.

12. The media of claim 9 , wherein the non-linear function comprises one of a sigmoid function, a Swish function, a hyperbolic tangent (Tanh) function, a Gaussian Error Linear Unit (GELU) function, or a rectifier function.

13. The media of claim 9 , wherein the resource constraints of the hardware accelerator scale linearly with a number of parameters of the polynomial function.

14. The media of claim 9 , the method further comprising receiving information defining a division configuration space and configuration costs, the received information including the resource constraints.

15. The media of claim 11 , wherein the index look-up-table and the main look-up-table are stored in a special function unit of a processing element of an application-specific integrated circuit (ASIC).

16. The media of claim 14 , wherein:

the error set is generated using linear programming;

the configuration set is generated using binary search; and

the improved configuration set is generated using dynamic programming.

17. A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to perform a method, the method comprising:

determining a plurality of intervals that partition an input domain of a non-linear function;

determining, for each of the intervals, a plurality of subinterval configurations corresponding to different numbers of subintervals for partitioning that interval;

generating an error set comprising, for each of the subinterval configurations of each of the intervals, an error for using a polynomial function to approximate the non-linear function within one or more corresponding subintervals specified by the subinterval configuration;

using the error set,

selecting one of the subinterval configurations for each of the intervals to generate a configuration set that minimizes a worst-case error across the intervals without exceeding resource constraints of a hardware accelerator; and

selecting one of the subinterval configurations for each of the intervals to generate an improved configuration set that minimizes a cumulative error across the intervals without exceeding the worst-case error; and

configuring the hardware accelerator based on the improved configuration set.

18. The system of claim 17 , wherein:

the plurality of intervals that partition the input domain of the non-linear function are disjoint binary intervals; and

the one or more corresponding subintervals respectively specified by each of the subinterval configurations are of equal size.

19. The system of claim 17 , the method further comprising:

storing, in an index look-up-table, for each of the intervals, the selected one of the subinterval configurations used to generate the improved configuration set; and

storing, in a main look-up-table, for each subinterval specified by the subinterval configurations stored in the index look-up-table, a set of polynomial function coefficients defined by the improved configuration set;

wherein the resource constraints define a maximum index look-up-table size and a maximum main look-up-table size.

20. The system of claim 19 , wherein the index look-up-table and the main look-up-table are stored in a special function unit of a processing element of an application-specific integrated circuit (ASIC).