Spatio-temporal learning algorithms in hierarchical temporal networks
A spatio-temporal learning node is a type of HTM node which learns both spatial and temporal groups of sensed input patterns over time. Spatio-temporal learning nodes comprise spatial poolers which are used to determine spatial groups in a set of sensed input patterns. The spatio-temporal learning nodes further comprise temporal poolers which are used to determine groups of sensed input patterns that temporally co-occur. A spatio-temporal learning network is a hierarchical network including a plurality of spatio-temporal learning nodes.
1. A computer-implemented hierarchical network comprising a plurality of spatio-temporal learning nodes, wherein each spatio-temporal learning node comprises:
a spatial pooler adapted to:
receive a sensed input pattern;
generate a first set of spatial probabilities associated with a set of spatial co-occurrence patterns, wherein each spatial co-occurrence pattern represents a first set of one or more sensed input patterns and each spatial probability in the first set of spatial probabilities indicates the likelihood that the sensed input pattern has the same cause as a spatial co-occurrence pattern;
a temporal pooler adapted to:
receive the first set of spatial probabilities from the spatial pooler;
generate a set of temporal probabilities associated with a set of temporal groups based at least in part the first set of spatial probabilities, wherein each temporal group comprises one or more temporally co-occurring input patterns and each temporal probability indicates the likelihood that the sensed input pattern has the same cause as the one or more temporally co-occurring input patterns in a temporal group; and
transmit the set of temporal probabilities to a parent node in the hierarchical network of nodes.
2. The network of claim 1 , further comprising a spatio-temporal learning top node adapted to:
receive a plurality of sets of temporal probabilities from a plurality of spatio-temporal learning nodes in the hierarchical network of nodes;
generate a second set of spatial probabilities based on the plurality of sets of temporal probabilities; and
generate a set of category probabilities based on the second set of spatial probabilities, wherein each category probability indicates the likelihood that the sensed input pattern belongs to a category of a set of categories.
3. The network of claim 1 , wherein:
the spatial pooler is further adapted to:
determine the set of spatial co-occurrence patterns based on a second set of one or more sensed input patterns; and
the temporal pooler is further adapted to:
determine the set of temporal groups based on the second set of one or more sensed input patterns.
4. The network of claim 3 , wherein the spatial pooler is further adapted to:
determine, for each of the second set of one or more sensed input patterns, at least a first similarity value based on at least a first spatial co-occurrence pattern and a defined similarity metric; and
determine the set of spatial co-occurrence patterns based on the at least a first similarity value.
5. The network of claim 4 , wherein the defined similarity metric is a Euclidean distance metric.
6. The network of claim 3 , wherein the spatial pooler is further adapted to determine the set of spatial co-occurrence patterns based on a sparse distributed coding algorithm.
7. The network of claim 1 , wherein the spatial pooler is further adapted to:
determine a set of similarity values which indicate the similarity between the sensed input pattern and the set of spatial co-occurrence patterns based on a defined similarity metric; and
generate the first set of spatial probabilities based on applying a Gaussian transform to the set of similarity values.
8. A computer-readable storage medium encoded with program code for a hierarchical network comprising a plurality of spatio-temporal learning nodes, wherein program code for each spatio-temporal learning node comprises:
program code for a spatial pooler comprising:
program code for receiving a sensed input pattern;
program code for generating a first set of spatial probabilities associated with a set of spatial co-occurrence patterns, wherein each spatial co-occurrence pattern represents a first set of one or more sensed input patterns and each spatial probability in the first set of spatial probabilities indicates the likelihood that the sensed input pattern has the same cause as a spatial co-occurrence pattern;
program code for a temporal pooler comprising:
program code for receiving the first set of spatial probabilities from the spatial pooler;
program code for generating a set of temporal probabilities associated with a set of temporal groups based at least in part the first set of spatial probabilities, wherein each temporal group comprises one or more temporally co-occurring input patterns and each temporal probability indicates the likelihood that the sensed input pattern has the same cause as the one or more temporally co-occurring input patterns in a temporal group; and
program code for transmitting the set of temporal probabilities to a parent node in the hierarchical network of nodes.
9. The storage medium of claim 8 , further comprising program code for a spatio-temporal learning top node comprising:
program code for receiving a plurality of sets of temporal probabilities from a plurality of spatio-temporal learning nodes in the hierarchical network of nodes;
program code for generating a second set of spatial probabilities based on the plurality of sets of temporal probabilities; and
program code for generating a set of category probabilities based on the second set of spatial probabilities, wherein each category probability indicates the likelihood that the sensed input pattern belongs to a category of a set of categories.
10. The storage medium of claim 8 , wherein:
program code for the spatial pooler further comprises:
program code for determining the set of spatial co-occurrence patterns based on a second set of one or more sensed input patterns; and
program code for the temporal pooler further comprises:
program code for determining the set of temporal groups based on the second set of one or more sensed input patterns.
11. The storage medium of claim 10 , wherein program code for the spatial pooler further comprises:
program code for determining, for each of the second set of one or more sensed input patterns, at least a first similarity value based on at least a first spatial co-occurrence pattern and a defined similarity metric; and
program code for determining the set of spatial co-occurrence patterns based on the at least a first similarity value.
12. The storage medium of claim 11 , wherein the defined similarity metric is a Euclidean distance metric.
13. The storage medium of claim 10 , wherein program code for the spatial pooler further comprises program code for determining the set of spatial co-occurrence patterns based on a sparse distributed coding algorithm.
14. The storage medium of claim 8 , wherein program code for the spatial pooler further comprises:
program code for determining a set of similarity values which indicate the similarity between the sensed input pattern and the set of spatial co-occurrence patterns based on a defined similarity metric; and
program code for generating the first set of spatial probabilities based on applying a Gaussian transform to the set of similarity values.