Assessing performance in a spatial and temporal memory system
A spatial and temporal memory system (STMS) processes input data to detect whether spatial patterns and/or temporal sequences of spatial patterns exist within the data, and to make predictions about future data. The data processed by the STMS may be retrieved from, for example, one or more database fields and is encoded into a distributed representation format using a coding scheme. The performance of the STMS in predicting future data is evaluated for the coding scheme used to process the data as performance data. The selection and prioritization of STMS experiments to perform may be based on the performance data for an experiment. The best fields, encodings, and time aggregations for generating predictions can be determined by an automated search and evaluation of multiple STMS systems.
1. A method of evaluating predictive performance of a spatial and temporal memory system, comprising:
generating a spatial and temporal memory system output by a sequence processor based on stored relationships between candidate spatial patterns determined by a spatial pooler from received input data representing a spatial pattern at a first time;
determining a predicted spatial pattern at a second time by predicting a first input to the sequence processor based on the spatial and temporal memory system output, and by predicting a second input to the spatial pooler based on the predicted first input and a mapping between input data and stored spatial patterns; and
evaluating the performance of the spatial and temporal memory system by comparing the predicted spatial pattern at the second time and a spatial pattern received at the second time.
2. The method of claim 1 , wherein generating a spatial and temporal memory system output comprises:
encoding the received input data into a distributed representation format at an encoder; and
determining the candidate spatial patterns in the encoded input data by the spatial pooler.
3. The method of claim 2 , wherein predicting the first input to the sequence processor comprises determining a highest confidence score of cells in a column of the sequence processor as a column confidence score, the prediction of the first input to the sequence processor based on the column confidences of columns indicated as being active in the output.
4. The method of claim 2 , wherein predicting the second input to the spatial pooler comprises, for each element of the second input to the spatial pooler, determining an average value of column confidence scores of columns mapped to the element of the second input to the spatial pooler.
5. The method of claim 4 , wherein determining a predicted spatial pattern at a second time further comprises predicting a third input to the encoder segmenting the predicted second input to the spatial pooler into one or more segments, each segment corresponding to a field of data in the third input.
6. The method of claim 1 , wherein comparing comprises determining whether the predicted spatial pattern at the second time and the spatial pattern received at the second time are identical.
7. The method of claim 1 , wherein comparing comprises quantifying a difference between the predicted spatial pattern at the second time and the spatial pattern received at the second time.
8. The method of claim 1 , wherein evaluating the performance of the spatial and temporal memory system comprises determining an average of scores representing the accuracy of a plurality of predictions.
9. A system for evaluating predictive performance of a spatial and temporal memory system, the system comprising:
a spatial and temporal memory system configured to generate a spatial and temporal memory system output by a sequence processor based on stored relationships between candidate spatial patterns determined by a spatial pooler from received input data representing a spatial pattern at a first time;
a decoder configured to determine a predicted spatial pattern at a second time by predicting a first input to the sequence processor based on the spatial and temporal memory system output, and by predicting a second input to the spatial cooler based on the predicted first input and a mapping between input data and stored spatial patterns; and
an evaluation module configured to evaluate the performance of the spatial and temporal memory system by comparing the predicted spatial pattern at the second time and a spatial pattern received at the second time.
10. The system of claim 9 , wherein generating a spatial and temporal memory system output comprises:
encoding the received input data into a distributed representation format at an encoder; and
determining the candidate spatial patterns in the encoded input data by the spatial pooler.
11. The system of claim 10 , wherein predicting the first input to the sequence processor comprises determining a highest confidence score of cells in a column of the sequence processor as a column confidence score, the prediction of the first input to the sequence processor based on the column confidences of columns indicated as being active in the output.
12. The system of claim 10 , wherein predicting the second input to the spatial pooler comprises, for each element of the second input to the spatial pooler, determining an average value of column confidence scores of columns mapped to the element of the second input to the spatial pooler.
13. The system of claim 12 , wherein determining a predicted spatial pattern at a second time further comprises predicting a third input to the encoder segmenting the predicted second input to the spatial pooler into one or more segments, each segment corresponding to a field of data in the third input.
14. The system of claim 9 , wherein comparing comprises determining whether the predicted spatial pattern at the second time and the spatial pattern received at the second time are identical.
15. The system of claim 9 , wherein comparing comprises quantifying a difference between the predicted spatial pattern at the second time and the spatial pattern received at the second time.
16. The system of claim 9 , wherein evaluating the performance of the spatial and temporal memory system comprises determining an average of scores representing the accuracy of a plurality of predictions.
17. A non-transitory computer-readable storage medium storing executable computer program instructions for evaluating predictive performance of a spatial and temporal memory system, the instructions comprising instructions for:
generating a spatial and temporal memory system output by a sequence processor based on stored relationships between candidate spatial patterns determined by a spatial pooler from received input data representing a spatial pattern at a first time;
determining a predicted spatial pattern at a second time by predicting a first input to the sequence processor based on the spatial and temporal memory system output, and by predicting a second input to the spatial pooler based on the predicted first input and a mapping between input data and stored spatial patterns; and
evaluating the performance of the spatial and temporal memory system by comparing the predicted spatial pattern at the second time and a spatial pattern received at the second time.
18. The non-transitory computer-readable storage medium of claim 17 , wherein generating a spatial and temporal memory system output comprises:
encoding the received input data into a distributed representation format at an encoder; and
determining the candidate spatial patterns in the encoded input data by the spatial pooler.