Lookup-based convolutional neural network

1. A method, comprising:

applying a convolutional neural network to input data to obtain output data, in which a layer of the convolutional neural network includes filters with weights that are stored as a dictionary (D) of channel weight vectors, a respective lookup index tensor (I) that indexes the dictionary, and a respective lookup coefficient tensor (C), wherein applying the convolutional neural network includes:

obtaining an input dictionary (S) based at least in part on the channel weight vectors of the dictionary (D); and

combining entries of the input dictionary (S) based at least in part on the respective lookup index tensor (I) and the respective lookup coefficient tensor (C); and

storing, displaying, or transmitting an output based on the output data of the convolutional neural network.

2. The method of claim 1 , further comprising training the convolutional neural network by back-propagation using a sparse tensor (P), wherein non-zero entries of the sparse tensor (P) correspond to coefficients of the respective lookup coefficient tensor (C) at locations corresponding to indices of the respective lookup index tensor (I), and wherein performing a training iteration comprises:

convolving the channel weight vectors of the dictionary (D) with a training input tensor to obtain a training input dictionary (S);

convolving the training input dictionary (S) with the sparse tensor (P) to obtain training activations;

determining a loss function based on the training activations; and

updating the sparse tensor (P) based on the loss function.

3. The method of claim 2 , wherein the loss function includes a regularization term based on a norm of the sparse tensor (P).

4. The method of claim 2 , wherein performing the training iteration further comprises:

sorting entries of a vector of the sparse tensor (P) corresponding to a spatial location; and

zeroing out all but the s largest entries of the vector of the sparse tensor (P), wherein s is a sparsity constraint.

5. The method of claim 2 , wherein performing the training iteration further comprises:

applying a threshold function to entries of the sparse tensor (P).

6. The method of claim 2 , wherein performing the training iteration further comprises:

back-propagating through a threshold function to determine a gradient with respect to the sparse tensor (P).

7. The method of claim 1 , further comprising training the convolutional neural network using back-propagation to update the respective lookup index tensor (I) and the respective lookup coefficient tensor (C) in multiple training iterations, while holding the dictionary (D) of channel weight vectors fixed.

8. The method of claim 1 , wherein the dictionary (D) of channel weight vectors is used in multiple layers of the convolutional neural network.

9. The method of claim 1 , wherein the dictionary (D) of channel weight vectors is determined by training a different neural network that includes the dictionary (D), wherein the different neural network is a different depth than the convolutional neural network.

10. The method of claim 1 , further comprising determining the dictionary (D) of channel weight vectors by training a different neural network that includes the dictionary (D), wherein the different neural network has less layers than the convolutional neural network.

11. The method of claim 1 , wherein the input data is based on an image and the output data is a classification of the image.

12. A non-transitory computer readable memory storing instructions which, when executed by one or more processors, cause the one or more processors to:

apply a convolutional neural network to input data to obtain output data, in which a layer of the convolutional neural network includes filters with weights that are stored as a dictionary (D) of channel weight vectors, a respective lookup index tensor (I) that indexes the dictionary, and a respective lookup coefficient tensor (C), wherein applying the convolutional neural network includes:

obtaining an input dictionary (S) based at least in part on the channel weight vectors of the dictionary (D); and

combining entries of the input dictionary (S) based at least in part on the respective lookup index tensor (I) and the respective lookup coefficient tensor (C); and

storing, displaying, or transmitting an output based on the output data of the convolutional neural network.

13. The non-transitory computer readable memory of claim 12 , wherein the instructions, when executed by one or more processors, further cause the one or more processors to train the convolutional neural network by back-propagation using a sparse tensor (P), wherein non-zero entries of the sparse tensor (P) correspond to coefficients of the respective lookup coefficient tensor (C) at locations corresponding to indices of the respective lookup index tensor (I), and wherein a training iteration comprises:

convolving the channel weight vectors of the dictionary (D) with a training input tensor to obtain a training input dictionary (S);

convolving the training input dictionary (S) with the sparse tensor (P) to obtain training activations;

determining a loss function based on the training activations; and

updating the sparse tensor (P) based on the loss function.

14. The non-transitory computer readable memory of claim 13 , wherein the loss function includes a regularization term based on a norm of the sparse tensor (P).

15. The non-transitory computer readable memory of claim 13 , wherein the training iteration further comprises:

sorting entries of a vector of the sparse tensor (P) corresponding to a spatial location; and

zeroing out all but the s largest entries of the vector of the sparse tensor (P), wherein s is a sparsity constraint.

16. The non-transitory computer readable memory of claim 12 , wherein the input data is based on an image and the output data is a classification of the image.

17. A device, comprising:

memory; and

one or more processors configured to:

apply a convolutional neural network to input data to obtain output data, in which a layer of the convolutional neural network includes filters with weights that are stored as a dictionary (D) of channel weight vectors, a respective lookup index tensor (I) that indexes the dictionary, and a respective lookup coefficient tensor (C), wherein the one or more processors are configured to apply the convolutional neural network by:

obtaining an input dictionary (S) based at least in part on the channel weight vectors of the dictionary (D); and

combining entries of the input dictionary (S) based at least in part on the respective lookup index tensor (I) and the respective lookup coefficient tensor (C); and

storing, displaying, or transmitting an output based on the output data of the convolutional neural network.

18. The device of claim 17 , wherein the device comprises a wearable device.

19. The device of claim 17 , wherein the device comprises a sensor configured to obtain the input data.

20. The device of claim 17 , wherein the dictionary (D) of channel weight vectors is determined by training a different neural network that includes the dictionary (D), wherein the different neural network is a different depth than the convolutional neural network.