Neural network mapping dictionary generation

1. A method of generating mapping dictionaries for a neural network, comprising:

receiving, at a current layer, encoded activation addresses from a previous layer and encoded weight addresses;

decoding the encoded activation addresses to generate decoded activation addresses;

decoding the encoded weight addresses to generate decoded weight addresses;

generating original activation addresses from the decoded activation addresses and the decoded weight addresses; and

matching the original activation addresses to a mapping dictionary to generate encoded activation addresses for the current layer.

2. The method of claim 1 , further comprising:

assigning each original activation address to a closest dictionary entry in the mapping dictionary to generate one or more clusters, wherein a cluster comprises one or more original activation address assigned to a common dictionary entry;

calculating an average distance between each original activation address and the closest dictionary entry in each cluster; and

updating the mapping dictionary according to the average distance calculations.

3. The method of claim 2 , wherein updating the mapping dictionary comprises updating each dictionary entry according to the average distance of an associated cluster and a learning rate.

4. The method of claim 3 , further comprising reducing the learning rate.

5. The method of claim 1 , wherein generating original activation addresses comprises generating 32-bit floating values.

6. The method of claim 1 , wherein decoding the encoded activation addresses comprises decoding 32-bit floating values.

7. The method of claim 1 , wherein matching the original activation addresses to a mapping dictionary comprises matching the original activation addresses to the mapping dictionary shared between the current layer and a next layer.

8. The method of claim 1 , wherein matching the original activation addresses to a mapping dictionary comprises matching the original activation addresses to the mapping dictionary comprising a plurality of 32-bit floating values.

9. The method of claim 1 , further comprising transmitting the encoded activation addresses to a next layer.

10. One or more non-transitory computer-readable media that include instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operations, the operations comprising:

receiving, at a current layer, encoded activation addresses from a previous layer and encoded weight addresses;

decoding the encoded activation addresses to generate decoded activation addresses;

decoding the encoded weight addresses to generate decoded weight addresses;

generating original activation addresses from the decoded activation addresses and the decoded weight addresses; and

matching the original activation addresses to a mapping dictionary to generate encoded activation addresses for the current layer.

11. The computer-readable media of claim 10 , the operations further comprising:

assigning each original activation address to a closest dictionary entry in the mapping dictionary to generate one or more clusters, wherein a cluster comprises one or more original activation addresses assigned to a common dictionary entry;

calculating an average distance between each original activation address and the closest dictionary entry in each cluster; and

updating the mapping dictionary according to the average distance calculations.

12. The computer-readable media of claim 11 , wherein updating the mapping dictionary comprises updating each dictionary entry according to the average distance of an associated cluster and a learning rate.

13. The computer-readable media of claim 12 , the operations further comprising reducing the learning rate.

14. The computer-readable media of claim 10 , wherein matching the original activation addresses to a mapping dictionary comprises matching the original activation addresses to the mapping dictionary shared between the current layer and a next layer.

15. The computer-readable media of claim 10 , the operations further comprising transmitting the encoded activation addresses to a next layer.

16. A system for generating mapping dictionaries for a neural network, comprising:

a plurality of layers, each layer of the plurality of layers including at least one storage device and a processing element, each processing element configured to:

decode encoded activation addresses to generate decoded activation addresses;

decode encoded weight addresses to generate decoded weight addresses;

generate original activation addresses from the decoded activation addresses and the decoded weight addresses; and

match the original activation addresses to a mapping dictionary to generate encoded activation addresses for the layer.

17. The system of claim 16 , each processing element further configured to:

assign each original activation address to a closest dictionary entry in the mapping dictionary to generate one or more clusters, wherein a cluster comprises one or more original activation addresses assigned to a common dictionary entry;

calculate an average distance between each original activation address and the closest dictionary entry in each cluster; and

update the mapping dictionary according to the average distance calculations.

18. The system of claim 16 , wherein each processing element is configured to update each dictionary entry according to the average distance of an associated cluster.

19. The system of claim 16 , each processing element further configured to transmit the encoded activation addresses to a next layer.

20. The system of claim 16 , wherein the neural network includes at least one of a convolutional neural network, a recurrent neural network, and a long short term memory (LSTM) neural network.