SYSTEMS, APPARATUSES, METHODS, AND COMPUTER PROGRAM PRODUCTS FOR MACHINE LEARNING WITH A LONG SHORT-TERM MEMORY ACCELERATOR

1 . A system comprising:

a long short-term memory (LSTM) accelerator comprising:

a finite state machine (FSM) configured with a plurality of states comprising a machine learning algorithm;

a weight memory configured to at least store a plurality of weights and a plurality of biases;

one or more activation registers;

a hidden state memory; and

a plurality of processing elements;

at least one processor and at least one memory coupled to the processor, wherein the processor is configured to:

apply the machine learning algorithm of the FSM, wherein the machine learning algorithm is configured to:

perform a plurality of operations with the plurality of processing elements including one or more matrix-vector multiplication operations, vector-vector multiplication operations, vector-vector addition operations, and non-linear activation operations; and

wherein at least one non-linear activation operation comprises receiving at least one input and negating at least one negative input.

2 . The system of claim 1 , wherein the weight memory comprises a look up table.

3 . The system of claim 2 , wherein the look up table of the weight memory is portioned into a plurality of portions, including at least a first portion associated with a forget gate of the FSM, a second portion associated with an input gate of the FSM, a third portion associated with a cell gate of the FSM, and a fourth portion associated with an output gate of the FSM.

4 . The system of claim 3 , wherein the first portion associated with a forget gate of the FSM stores a plurality of weights and a plurality of biases associated with the forget gate;

wherein the second portion associated with an input gate of the FSM stores a plurality of weights and a plurality of biases associated with the input gate;

wherein the third portion associated with a cell gate of the FSM stores a plurality of weights and a plurality of biases associated with the cell gate; and

wherein the fourth portion associated with the output gate of the FSM stores a plurality of weights and a plurality of biases associated with the output gate.

5 . The system of claim 3 , wherein the first portion associated with a forget gate of the FSM is pre-allocated, the second portion associated with an input gate of the FSM is pre-allocated, the third portion associated with a cell gate of the FSM is pre-allocated, and the fourth portion associated with the output gate of the FSM is pre-allocated.

6 . The system of claim 1 , wherein at least one non-linear activation operation includes a tanh operation.

7 . The method of claim 1 , wherein at least one non-linear activation operation includes a sigmoid operation.

8 . The system of claim 1 , wherein the one or more matrix-vector multiplication operations, vector-vector multiplication operations, vector-vector addition operations, and non-linear activation operations include:

at least four matrix-vector multiplication operations;

at least three vector-vector multiplication operations;

at least one vector-vector addition operations; and

at least one non-linear activation.

9 . The system of claim 1 further comprising:

a laser; and

at least one photodetector; and

wherein the processor is further configured to:

transmit a sensor pulses with the laser;

generate sensor signals and timestamps based on one or more reflections received by the at least one photodetector, wherein the reflections are associated with the one or more sensor pulses;

generate, with the machine learning algorithm of the FSM of the LSTM accelerator, at least one phase associated with each of the at sensor signals and timestamps; and

determine a distance to an object based on the at least one phase.

10 . The system of claim 9 , wherein the at least one photodetector includes at least one single-photon avalanche diode.

11 . A method comprising:

providing a long short-term memory (LSTM) accelerator comprising:

a finite state machine (FSM) configured with a plurality of states comprising a machine learning algorithm;

a weight memory configured to at least store a plurality of weights and a plurality of biases;

one or more activation registers;

a hidden state memory; and

a plurality of processing elements;

apply the machine learning algorithm of the FSM comprising performing one or more matrix-vector multiplication operations, vector-vector multiplication operations, vector-vector addition operations, and non-linear activation operations; and

wherein at least one non-linear activation operation comprises receiving at least one input and negating at least one negative input.

12 . The method of claim 11 , wherein the weight memory comprises a look up table.

13 . The method of claim 12 , wherein the look up table of the weight memory is portioned into a plurality of portions, including at least a first portion associated with a forget gate of the FSM, a second portion associated with an input gate of the FSM, a third portion associated with a cell gate of the FSM, and a fourth portion associated with an output gate of the FSM.

14 . The method of claim 13 , wherein the first portion associated with a forget gate of the FSM stores a plurality of weights and a plurality of biases associated with the forget gate;

wherein the second portion associated with an input gate of the FSM stores a plurality of weights and a plurality of biases associated with the input gate;

wherein the third portion associated with a cell gate of the FSM stores a plurality of weights and a plurality of biases associated with the cell gate; and

wherein the fourth portion associated with the output gate of the FSM stores a plurality of weights and a plurality of biases associated with the output gate.

15 . The method of claim 13 , wherein the first portion associated with a forget gate of the FSM is pre-allocated, the second portion associated with an input gate of the FSM is pre-allocated, the third portion associated with a cell gate of the FSM is pre-allocated, and the fourth portion associated with the output gate of the FSM is pre-allocated.

16 . The method of claim 11 , wherein at least one non-linear activation operation includes a tanh operation.

17 . The method of claim 11 , wherein at least one non-linear activation operation includes a sigmoid operation.

18 . The method of claim 11 , wherein the one or more matrix-vector multiplication operations, vector-vector multiplication operations, vector-vector addition operations, and non-linear activation operations include:

at least four matrix-vector multiplication operations;

at least three vector-vector multiplication operations;

at least one vector-vector addition operations; and

at least one non-linear activation.

19 . The method of claim 11 further comprising:

providing a laser and at least one photodetector, and

transmitting a sensor pulses with the laser;

generating sensor signals and timestamps based on one or more reflections received by the at least one photodetector, wherein the reflections are associated with the one or more sensor pulses;

generating, with the machine learning algorithm of the FSM of the LSTM accelerator, at least one phase associated with each of the at sensor signals and timestamps; and

determining a distance to an object based on the at least one phase.

20 . The method of claim 19 , wherein the at least one photodetector includes at least one single-photon avalanche diode.