Competitive BCM learning rule for identifying features
Disclosed are systems, apparatuses, and methods for implementing a competitive BCM learning rule used in a neural network. Such a method includes identifying a maximally responding neuron with respect to a feature of an input signal. The maximally responding neuron is the neuron in a group that has a response with respect to the feature of the input signal that is greater than a response of each other neuron in the group. Such a method also includes applying a learning rule to weaken the response of each other neuron with respect to the feature of the input signal. The learning rule may also strengthen the response of the maximally responding neuron with respect to the feature of the input signal.
1. A computer-implemented method for implementing an artificial neural system including an input layer configured to respond to stimulus, a set of neurons and a set of connections between the input layer and the set of neurons, the method comprising:
determining a response for individual ones of the set of neurons with respect to a feature in the stimulus, wherein the response correlates positively to a frequency of the feature in the stimulus;
comparing responses for neurons in the set of neurons with respect to the feature in the stimulus;
identifying, from the set of neurons, a maximally responding neuron, the maximally responding neuron being a neuron having a larger response to the feature in the stimulus than all other neurons in the set of neurons;
updating weights of individual connections of the maximally responding neuron according to a first learning rule, wherein the first learning rule uses a first function based on a first modification threshold;
updating weights of individual connections of the other neurons in the set of neurons according to a second learning rule, wherein the other neurons are different from the maximally responding neuron, wherein the second learning rule uses a second function based on a second modification threshold, wherein a value of the second modification threshold for a particular individual other neuron is greater than the response of the particular individual other neuron to the feature in the stimulus and the value of the second modification threshold is obtained by multiplying the response of the particular individual other neuron by a factor larger than one; and
generating responses to the feature in the stimulus by the maximally responding neuron and other neurons respectively according to their respective updated weights, wherein application of the first learning rule strengthens the response of the maximally responding neuron to the feature in the stimulus and wherein application of the second learning rule weakens the response of the other neurons to the feature in the stimulus.
2. A computing system for implementing an artificial neural system, the system comprising:
a neural network, comprising:
an input layer configured to respond to stimulus;
a set of neurons including output neurons;
a set of connections between the input layer and the set of neurons, wherein values of individual connections represent weights;
a memory configured to store the set of neurons; and
a processing unit configured to
determine a response for individual ones of the set of neurons with respect to a feature in the stimulus, wherein the response correlates positively to a frequency of the feature in the stimulus;
compare responses for neurons in the set of neurons with respect to the feature in the stimulus;
identify, from the set of neurons, a maximally responding neuron, the maximally responding neuron being a neuron having a larger response to the feature in the stimulus than all other neurons in the set of neurons;
update weights of individual connections of the maximally responding neuron according to a first learning rule, wherein the first learning rule uses a first function based on a first modification threshold;
update weights of individual connections of the other neurons in the set of neurons according to a second learning rule, wherein the other neurons are different from the maximally responding neuron, wherein the second learning rule uses a second function based on a second modification threshold, and wherein a value of the second modification threshold for a particular individual other neuron is greater than the response of the particular individual other neuron to the feature in the stimulus and the value of the second modification threshold is obtained by multiplying the response of the particular individual other neuron by a factor larger than one;
generate responses to the feature in the stimulus by the maximally responding neuron and other neurons respectively according to their respective updated weights, wherein application of the first learning rule strengthens the response of the maximally responding neuron to the feature in the stimulus; and
wherein application of the second learning rule to weakens the response of the other neurons to the feature in the stimulus.
3. A non-transitory machine-readable medium that stores instructions, which, when executed by a computer processor, cause the computer processor to perform a computer-implemented method for implementing an artificial neural system including an input layer configured to respond to stimulus, a set of neurons and a set of connections between the input layer and the set of neurons, the method comprising:
determining a response for individual ones of the set of neurons with respect to a feature in the stimulus, wherein the response correlates positively to a frequency of the feature in the stimulus;
comparing responses for neurons in the set of neurons with respect to the feature in the stimulus;
identifying, from the set of neurons, a maximally responding neuron, the maximally responding neuron being a neuron having a larger response to the feature in the stimulus than all other neurons in the set of neurons;
updating weights of individual connections of the maximally responding neuron according to a first learning rule, wherein the first learning rule uses a first function based on a first modification threshold;
updating weights of individual connections of the other neurons in the set of neurons according to a second learning rule, wherein the other neurons are different from the maximally responding neuron, wherein the second learning rule uses a second function based on a second modification threshold, wherein a value of the second modification threshold for a particular individual other neuron is greater than the response of the particular individual other neuron to the feature in the stimulus and the value of the second modification threshold is obtained by multiplying the response of the particular individual other neuron by a factor larger than one; and
generating responses to the feature in the stimulus by the maximally responding neuron and other neurons respectively according to their respective updated weights, wherein application of the first learning rule strengthens the response of the maximally responding neuron to the feature in the stimulus and wherein application of the second learning rule weakens the response of the other neurons to the feature in the stimulus.
4. The non-transitory machine-readable medium of claim 3 , wherein the factor is an integer value two (2).
5. The non-transitory machine-readable medium of claim 3 , wherein the factor is determined depending on a level of depression desired for each particular application.
6. The non-transitory machine-readable medium of claim 3 , wherein the method further comprises generating different neural responses by the set of neurons to different features of the stimulus, wherein application of the first and second learning rules facilitates each neuron of the set of neurons to respond to a unique feature of the stimulus.
7. The computer-implemented method of claim 1 , wherein the factor is an integer value two (2).
8. The computer-implemented method of claim 1 , wherein the factor is determined depending on a level of depression desired for each particular application.
9. The computer-implemented method of claim 1 , further comprising generating different neural responses by the set of neurons to different features of the stimulus, wherein application of the first and second learning rules facilitates each neuron of the set of neurons to respond to a unique feature of the stimulus.
10. The computing system of claim 2 , wherein the factor is an integer value two (2).
11. The computing system of claim 2 , wherein the factor is determined depending on a level of depression desired for each particular application.
12. The computing system of claim 2 , wherein the processing unit is further configured to generate different neural responses by the set of neurons to different features, wherein application of the first and second learning rules facilitates each neuron of the set of neurons to respond to a unique feature of the stimulus.