Order statistic techniques for neural networks
View Patent ↗According to some aspects, a method of classifying speech recognition results is provided, using a neural network comprising a plurality of interconnected network units, each network unit having one or more weight values, the method comprising using at least one computer, performing acts of providing a first vector as input to a first network layer comprising one or more network units of the neural network, transforming, by a first network unit of the one or more network units, the input vector to produce a plurality of values, the transformation being based at least in part on a plurality of weight values of the first network unit, sorting the plurality of values to produce a sorted plurality of values, and providing the sorted plurality of values as input to a second network layer of the neural network.
1. A method of classifying speech recognition results using a neural network comprising a plurality of interconnected network units, each network unit having one or more weight values, the method comprising:
using at least one computer, performing acts of:
providing a first vector as input to a first network layer comprising one or more network units of the neural network;
transforming, by a first network unit of the one or more network units, the input vector to produce a plurality of values, the transformation being based at least in part on a plurality of weight values of the first network unit;
sorting the plurality of values to produce a sorted plurality of values; and
providing the sorted plurality of values as input to a second network layer of the neural network.
2. The method of claim 1 , wherein the method further comprises, using the at least one computer:
transforming, by a second network unit of the one or more network units, the input vector to produce a second plurality of values;
sorting the second plurality of values to produce a second sorted plurality of values; and
providing the second sorted plurality of values as input to the second network layer.
3. The method of claim 2 , wherein providing the sorted plurality of values and the second sorted plurality of values as input to the second network layer comprises:
providing a second vector as input to the second network layer, wherein the second vector comprises the plurality of values and the second plurality of values.
4. The method of claim 1 , wherein the neural network is a feed forward network.
5. The method of claim 1 , wherein the first network layer and the second network layer are hidden layers.
6. The method of claim 1 , wherein the first network unit comprises a plurality of weight vectors.
7. The method of claim 1 , wherein said transformation of the input vector to produce the plurality of values includes a plurality of linear transformations.
8. The method of claim 1 , wherein the method further comprises, using the at least one computer, applying a non-linear function to each of the plurality of values.
9. The method of claim 1 , wherein the input vector comprises a plurality of speech recognition features.
10. The method of claim 9 , wherein the speech recognition features are log-mel coefficients and/or mel-frequency cepstral coefficients.
11. The method of claim 1 , wherein the input vector comprises a plurality of image features.
12. At least one non-transitory computer readable storage medium storing instructions that, when executed by at least one processor, perform a method of classifying speech recognition results using a neural network comprising a plurality of interconnected network units, each network unit having one or more weight values, the method comprising: providing a first vector as input to a first network layer comprising one or more network units of the neural network; transforming, by a first network unit of the one or more network units, the input vector to produce a plurality of values, the transformation being based at least in part on a plurality of weight values of the first network unit; sorting the plurality of values to produce a sorted plurality of values; and providing the sorted plurality of values as input to a second network layer of the neural network.
13. The at least one computer readable storage medium of claim 12 , wherein the method further comprises, using the at least one computer:
transforming, by a second network unit of the one or more network units, the input vector to produce a second plurality of values;
sorting the second plurality of values to produce a second sorted plurality of values; and
providing the second sorted plurality of values as input to the second network layer.
14. The at least one computer readable storage medium of claim 12 , wherein the neural network is a feed forward network.
15. The at least one computer readable storage medium of claim 12 , wherein the first network layer and the second network layer are hidden layers.
16. The at least one computer readable storage medium of claim 12 , wherein said transformation of the input vector to produce the plurality of values includes a plurality of linear transformations.
17. The at least one computer readable storage medium of claim 12 , wherein the method further comprises, using the at least one computer, applying a non-linear function to each of the plurality of values.