Neural network for clustering input data based on a Gaussian Mixture Model

1. A computer-implemented method for clustering a set of input data in a first cluster and a second cluster, wherein the input data includes a first input pattern, comprising:

providing the first input pattern to individual ones of a plurality of cluster microcircuits of a neural network, wherein the plurality of cluster microcircuits includes a first circuit and a second circuit, the first circuit including a first set of response-determining neurons, a first mean vector that encodes a center of the first cluster, and a first co-variance matrix, and the second circuit including a second set of response-determining neurons, a second mean vector that encodes a center of the second cluster, and a second co-variance matrix;

determining a first response of the first circuit with respect to the first input pattern, wherein the first response is based on the first set of response-determining neurons;

calculating a first difference between the first response and the first mean vector;

determining a second response of the second circuit with respect to the first input pattern, wherein the second response is based on the second set of response-determining neurons;

calculating a second difference between the second response and the second mean vector;

determining a smallest difference between the first difference and the second difference; and

modifying, responsive to the first difference being the smallest difference, the first mean vector and the first co-variance matrix based on the first response, or, responsive to the second difference being the smallest difference, modifying the second mean vector and the second co-variance matrix based on the second response.

2. The computer-implemented method of claim 1 , wherein the input data is K-dimensional, K being an integer greater than one, wherein the first difference is based on a K-dimensional distance between the first response and the first mean vector.

3. The computer-implemented method of claim 1 , wherein modification of the first co-variance matrix is based on a first user-specified learning rate, and wherein modification of the second co-variance matrix is based on a second user-specified learning rate.

4. The computer-implemented method of claim 3 , wherein the first user-specified learning rate is different from the second user-specified learning rate.

5. The computer-implemented method of claim 1 , wherein modification of the first mean vector and the first co-variance matrix is based on a first learning rate, wherein modification of the second mean vector and the second co-variance matrix is based on a second learning rate,

wherein, responsive to the first difference being the smallest difference, the first learning rate is greater than the second learning rate, and

wherein, responsive to the second difference being the smallest difference, the second learning rate is greater than the first learning rate.

6. The computer-implemented method of claim 5 , wherein the first learning rate is based on the first response, and wherein the second learning rate is based on the second response.

7. The computer-implemented method of claim 1 , wherein the set of input data includes multiple input patterns, wherein operations are performed for individual ones of the set of multiple input patterns, wherein, responsive to performance of the operations for individual ones of the set of multiple input patterns, the set of input data is clustered such that individual input patterns of the set of input data have a first probability indicating association with the first cluster and a second probability indicating association with the second cluster.

8. The computer-implemented method of claim 7 , wherein the set of input data includes an audio signal, wherein the first circuit is configured to distinguish when a particular speaker is talking, and wherein the second circuit is configured to distinguish when a particular language is being spoken.

9. A computer-program product comprising a non-transitory computer-readable storage medium having instructions stored thereon that, if executed by a computing device, cause the computing device to perform a method for clustering a set of input data in a first cluster and a second cluster, wherein the input data includes a first input pattern, the method comprising:

providing the first input pattern to individual ones of a plurality of cluster microcircuits of a neural network, wherein the plurality of cluster microcircuits includes a first circuit and a second circuit, the first circuit including a first set of response-determining neurons, a first mean vector that encodes a center of the first cluster, and a first co-variance matrix, and the second circuit including a second set of response-determining neurons, a second mean vector that encodes a center of the second cluster, and a second co-variance matrix;

determining a first response of the first circuit with respect to the first input pattern, wherein the first response is based on the first set of response-determining neurons;

calculating a first difference between the first response and the first mean vector;

determining a second response of the second circuit with respect to the first input pattern, wherein the second response is based on the second set of response-determining neurons;

calculating a second difference between the second response and the second mean vector;

determining a smallest difference between the first difference and the second difference; and

modifying, responsive to the first difference being the smallest difference, the first mean vector and the first co-variance matrix based on the first response, or, responsive to the second difference being the smallest difference, modifying the second mean vector and the second co-variance matrix based on the second response.

10. The computer-program product of claim 9 , wherein the input data is K-dimensional, K being an integer greater than one, wherein the first difference is based on a K-dimensional distance between the first response and the first mean vector.

11. The computer-program product of claim 9 , wherein modification of the first co-variance matrix is based on a first user-specified learning rate, and wherein modification of the second co-variance matrix is based on a second user-specified learning rate.

12. The computer-program product of claim 11 , wherein the first user-specified learning rate is different from the second user-specified learning rate.

13. The computer-program product of claim 9 , wherein modification of the first mean vector and the first co-variance matrix is based on a first learning rate, wherein modification of the second mean vector and the second co-variance matrix is based on a second learning rate,

wherein, responsive to the first difference being the smallest difference, the first learning rate is greater than the second learning rate, and

wherein, responsive to the second difference being the smallest difference, the second learning rate is greater than the first learning rate.

14. The computer-program product of claim 13 , wherein the first learning rate is based on the first response, and wherein the second learning rate is based on the second response.

15. A computing system for implementing a neural network for clustering a set of input data in a first cluster and a second cluster, comprising:

an input neural group configured to receive a set of input data, wherein the set of input data includes a first input pattern;

a plurality of cluster microcircuits of the neural network, wherein the plurality of cluster microcircuits includes a first circuit and a second circuit,

the first circuit including:

a first set of response-determining neurons configured to determine a first response with respect to the first input pattern,

a first mean vector that encodes a center of the first cluster,

a first co-variance matrix, and

a first neural modification group,

the second circuit including:

a second set of response-determining neurons configured to determine a second response with respect to the first input pattern,

a second mean vector that encodes a center of the second cluster,

a second co-variance matrix, and

a second neural modification group,

wherein the first circuit is configured to calculate a first difference between the first response and the first mean vector,

wherein the second circuit is configured to calculate a second difference between the second response and the second mean vector; and

a neural group configured to determine the smallest difference between the first difference and the second difference,

wherein, responsive to the first difference being the smallest difference, the first neural modification group is configured to modify the first mean vector and the first co-variance matrix based on the first response, and wherein, responsive to the second difference being the smallest difference, the second neural modification group is configured to modify the second mean vector and the second co-variance matrix based on the second response.

16. The computing system of claim 15 , wherein the input data is K-dimensional, K being an integer greater than one, wherein the first difference is based on a K-dimensional distance between the first response and the first mean vector.

17. The computing system of claim 15 , wherein the first neural modification group is configured to modify the first co-variance matrix based on a first user-specified learning rate, and wherein the second neural modification group is configured to modify the second co-variance matrix based on a second user-specified learning rate.

18. The computing system of claim 17 , wherein the first user-specified learning rate is different from the second user-specified learning rate.

19. The computing system of claim 15 , wherein the first neural modification group is configured to modify the first mean vector and the first co-variance matrix based on a first learning rate, wherein the second neural modification group is configured to modify the second mean vector and the second co-variance matrix based on a second learning rate,

wherein, responsive to the first difference being the smallest difference, the first learning rate is greater than the second learning rate, and

wherein, responsive to the second difference being the smallest difference, the second learning rate is greater than the first learning rate.

20. The computing system of claim 19 , wherein the first learning rate is based on the first response, and wherein the second learning rate is based on the second response.