Optimized processing engine of an Internet of Things (IoT) device and a method of generating the same

1 . An optimized processing engine of an Internet of Things (IoT) device, the optimized processing engine configured to perform processing operations on input data received at the IoT device, wherein the optimized processing engine comprises:

an optimized number of processing units, each processing unit configured to perform one or more of the processing operations;

a memory unit configured to store (i) the input data, (ii) one or more processing parameters for performing the processing operations, and (iii) one or more intermediate outputs of the processing units while performing the processing operations; and

an engine controller configured to control a loading sequence of the input data and the one or more processing parameters to the processing units while performing the processing operations,

wherein the optimized number of the processing units is determined by an optimization engine of an engine generation device by:

determining an initial number of the processing units for performing the processing operations based on a processing engine description input and one or more constraint parameters associated with the processing units; and

iteratively changing the number of the processing units from the initial number to the optimized number, wherein the optimized number of the processing units is operable to perform the processing operations while meeting the one or more constraint parameters, and

wherein the processing engine includes a neural network model having one or more layers, the processing units include multiply-accumulate (MAC) units arranged in layers corresponding to the one or more layers in the neural network model, and iteratively changing the number of the processing units from the initial number to the optimized number includes:

starting iterations from the initial number of the processing units, the initial number being determined to meet the one or more constraint parameters;

iteratively changing a number of layers of the MAC units and a number of MAC units per layer of the MAC units until a first iteration for which the one or more constraint parameters are not met; and

selecting the optimized number of the processing units based on the number of layers of the MAC units and the number of MAC units per layer for an iteration immediately preceding the first iteration.

2 . The optimized processing engine of claim 1 , wherein the one or more constraint parameters include one or more of (i) a total number of the processing units, (ii) a total area of the processing units, (iii) a total power consumption of the processing units while performing the processing operations, and (iv) a latency time constraint of the processing units to perform a set of the processing operations.

3 . The optimized processing engine of claim 2 , wherein:

the one or more constraint parameters include the latency time constraint;

the initial number of the processing units corresponds to (i) a maximum number of the processing units and (ii) a minimum latency time, for performing the set of the processing operations; and

iteratively changing the number of the processing units from the initial number to the optimized number includes iterative reduction from the initial number of the processing units to a minimum number of the processing units operable to perform the set of the processing operations while meeting the latency time constraint.

4 . The optimized processing engine of claim 2 , wherein:

the one or more constraint parameters include one or more of the total number of the processing units, the total area of the processing units, and the total power consumption of the processing units while performing the processing operations;

the initial number of processing units corresponds to (i) a minimum number of the processing units and (ii) a maximum latency time, for performing the set of the processing operations; and

iteratively changing the number of the processing units from the initial number to the optimized number includes iteratively increasing the initial number of the processing units to a maximum number of the processing units operable to perform the set of the processing operations while meeting the one or more constraint parameters.

5 . The optimized processing engine of claim 1 , wherein the one or more processing parameters for performing the processing operations include a matrix of weights corresponding to the one or more layers.

6 . The optimized processing engine of claim 5 , wherein the processing engine description input includes one or more of a number of layers in the neural network model, a maximum number of neurons per layer in the neural network model, a feature size of the neural network model, an input width of the processing engine, and an output width of the processing engine.

7 . The optimized processing engine of claim 6 , wherein the optimized number of the processing units includes reusing a layer of the MAC units to perform the processing operations for a first layer and a second layer of the neural network model; and

the engine controller is configured to sequentially (i) load the MAC units of the reused layer with inputs and weights corresponding to the first layer, (ii) store intermediate outputs of the MAC units of the reused layer in the memory unit, and (iii) load the MAC units of the reused layer with inputs and weights corresponding to the second layer.

8 . The optimized processing engine of claim 6 , wherein iteratively changing the number of the processing units from the initial number to the optimized number includes iteratively changing the number of layers of the MAC units followed by iteratively changing the number of MAC units per layer of the MAC units.

9 . The optimized processing engine of claim 6 , wherein iteratively changing the number of the processing units from the initial number to the optimized number includes iteratively changing the number of MAC units per layer of the MAC units followed by iteratively changing the number of layers of the MAC units.

10 . The optimized processing engine of claim 1 , wherein the one or more processing parameters for performing the processing operations includes a quantization input specifying a sigmoid activation function.

11 . The optimized processing engine of claim 10 , wherein the sigmoid activation function is included in the optimized processing engine as a read-only memory (ROM) look-up table stored in the memory unit.

12 . The optimized processing engine of claim 10 , wherein the sigmoid activation function is included in the optimized processing engine as a Taylor series decomposition hardware implementation.

13 . The optimized processing engine of claim 1 , further comprising a pre-scaler engine configured to perform a pre-scaling operation to adjust a data width of the input data to an input width of the processing units of the optimized processing engine.

14 . The optimized processing engine of claim 13 , further comprising a denormalization engine configured to perform a denormalization operation complementary to the pre-scaling operation to adjust an output width of an output of the optimized processing engine.

15 . The optimized processing engine of claim 1 , wherein the engine controller is further configured to monitor operation of the processing units; and switch off at least one of a clock signal and an input power to any processing units not being used by the optimized processing engine to perform the processing operations.

16 . The optimized processing engine of claim 1 , wherein the engine generation device is configured to generate an automated testing environment for testing and validation of the optimized processing engine.

17 . The optimized processing engine of claim 1 , wherein the engine generation device is configured to generate an output defining the optimized processing engine in one or more of a Register Transfer Level Language (RTL) format, a Netlist format or a Graphic Data System (GDS) format.

18 . The optimized processing engine of claim 1 , wherein the processing engine includes a Kalman filter estimator.

19 . The optimized processing engine of claim 18 , wherein the processing units include systolic arrays.

20 . An IoT device comprising the optimized processing engine of any one of claims 1-6 or 7-19 .

21 . A method of generating an optimized processing engine of an Internet of Things (IoT) device, the optimized processing engine configured to perform processing operations on input data received at the IoT device, wherein the optimized processing engine includes:

an optimized number of processing units, each processing unit configured to perform one or more of the processing operations;

a memory unit configured to store (i) the input data, (ii) one or more processing parameters for performing the processing operations, and (iii) one or more intermediate outputs of the processing units while performing the processing operations; and

an engine controller configured to control a loading sequence of the input data and the one or more processing parameters to the processing units while performing the processing operations,

wherein the processing engine includes a neural network model having one or more layers, the processing units include multiply-accumulate (MAC) units arranged in layers corresponding to the one or more layers in the neural network model, and

wherein the method comprises:

determining, by an optimization engine of an engine generation device, the optimized number of the processing units by:

receiving a processing engine description input and one or more constraint parameters associated with the processing units;

determining an initial number of the processing units for performing the processing operations based on the processing engine description input and the one or more constraint parameters; and

iteratively changing the number of processing units from the initial number to the optimized number by:

iteratively changing a number of layers of the MAC units and a number of MAC units per layer of the MAC units until a first iteration for which the one or more constraint parameters are not met, and

selecting the optimized number of the processing units based on the number of layers of the MAC units and the number of MAC units per layer for an iteration immediately preceding the first iteration,

wherein the optimized number of processing units is operable to perform the processing operations while meeting the one or more constraint parameters; and

generating, by the engine generation device, the optimized processing engine based on the determined optimized number of the processing units.