Low precision neural networks using subband decomposition

1. An artificial neural network system implemented by one or more computers, the artificial neural network system being configured to:

receive input data that defines a pattern to be recognized;

decompose the input data into a first subband and a second subband, wherein the first subband and the second subband include different characterizing features of the pattern in the input data; and

feed the first subband and not the second subband into a first neural network and feed the second subband and not the first subband into a second neural network, the first neural network and the second neural network being trained to recognize the pattern, wherein the first neural network is different from the second neural network.

2. The system of claim 1 , wherein the system is configured to decompose the input data based on frequency.

3. The system of claim 1 , wherein the input data is decomposed using at least one of a low-pass filter, a high-pass filter, a Laplacian filter, a Gaussian filter, and a Gabor filter.

4. The system of claim 1 , wherein the system is further configured to adjust bit precision of the first and second neural networks during training based on the first and second subbands.

5. The system of claim 1 , wherein the system is further configured to fuse the first neural network being trained on the first subband and the second neural network being trained on the second subband.

6. The system of claim 5 , wherein the system is further configured to feed the fused neural network into a subsequent hidden layer of at least one of the first neural network and the second neural network.

7. The system of claim 5 , wherein the fused neural network comprises a weighted average of the output of the first neural network and the second neural network.

8. The system of claim 1 , wherein the system is further configured to feed the first neural network trained on the first subband and the second neural network trained on the second subband into a subsequent hidden layer of at least one of the first neural network and the second neural network.

9. The system of claim 1 , wherein the pattern is a phoneme, and the first subband and the second subband characterize at least one of frequency and pitch.

10. The system of claim 1 , wherein the system is further configured to vary a width of at least one of the first neural network during training based on the first subband and the second neural network during training based on the second subband.

11. An artificial neural network method including the steps of:

receiving input data that defines a pattern to be recognized;

decomposing the input data into a first subband and a second subband, wherein the first subband and the second subband include different characterizing features of the pattern in the input data;

applying a first weight to the first subband and a second weight to the second subband, wherein the first weight is different from the second weight; and

feeding the first subband and not the second subband into a first neural network and feeding the second subband and not the first subband into a second neural network, the first neural network and the second neural network being trained to recognize the pattern, wherein the first neural network is different from the second neural network.

12. The method of claim 11 , wherein the input data is decomposed based on frequency.

13. The method of claim 11 , wherein the input data is decomposed using at least one of a low-pass filter, a high-pass filter, a Laplacian filter, a Gaussian filter, and a Gabor filter.

14. The method of claim 11 , further including the step of varying bit precision of the first and second neural networks during training based on the first and second subbands.

15. The method of claim 11 , further including the step of fusing the first neural network trained on the first subband and the second neural network being trained on the second subband trained on the second subband.

16. The method of claim 15 , wherein the fused neural network comprises a weighted average of the output of the first neural network and the second neural network.

17. The method of claim 11 , further including the step of varying a width of at least one of the first neural network during training based on the first subband and the second neural network during training based on the second subband.

18. The method of claim 11 ,

wherein the input data is decomposed into the first subband, the second subband, and a third subband, and

wherein the method further comprises removing an element of at least one of the first neural network and the second neural network based on a substantially zero weight contribution of at least one of the first subband, the second subband, and the third subband.

19. A non-transitory, computer-readable medium comprising instructions that, when executed, are configured to cause processing circuitry of an artificial neural network system to

receive input data that defines a pattern to be recognized;

decompose the input data into a first subband and a second subband, wherein the first subband and the second subband include different characterizing features of the pattern in the input data;

apply a first weight to the first subband and a second weight to the second subband, wherein the first weight is different from the second weight; and

feed the first subband and not the second subband into a first neural network and feed the second subband and not the first subband into a second neural network, the first neural network and the second neural network being trained to recognize the pattern, wherein the first neural network is different from the second neural network.

20. The system of claim 1 , wherein the system is further configured to apply a first weight to the first subband and a second weight to the second subband, wherein the first weight is different from the second weight.