Set of resonator networks for factorizing hyper vectors
View Patent ↗The present disclosure relates to a resonator network system comprising a set of resonator networks, each resonator network being configured to execute a resonator network, the resonator network being configured to receive an input hypervector representing a data structure and to perform an iterative process in order to factorize the input hypervector into individual hypervectors representing a set of concepts respectively, the set of N resonator networks being associated with N permutations respectively. The resonator network system being configured for applying the N permutations to N first hypervectors respectively, the N first hypervectors representing a set of N data structures respectively; and combining the N permuted hypervectors into a bundled hypervector. The resonator networks being configured for processing the bundled hypervector respectively, thereby factorizing the first hypervectors.
1. A method comprising:
providing an encoder for representing data structures in a vector space, the vector space being defined by a set of codebooks, which encode a set of cognitive concepts respectively, the codebooks comprising candidate code hypervectors representing items of the respective cognitive concepts;
providing a set of N resonator networks, where N>1, each resonator network being configured to receive an input hypervector representing a data structure and to perform an iterative process in order to factorize the input hypervector into individual hypervectors representing the set of cognitive concepts respectively utilizing superposition and clean-up memory, the set of N resonator networks being associated with N permutations respectively;
representing using the encoder a set of N data structures by N first hypervectors respectively;
applying the N permutations to the N first hypervectors respectively;
combining the N permuted hypervectors into a bundled hypervector; and
processing the bundled hypervector by the resonator networks, thereby simultaneously factorizing the N first hypervectors.
2. The method of claim 1 , the processing of the bundled hypervector at the respective resonator network comprising:
applying to the bundled hypervector an inverse permutation of the permutation associated with the resonator network resulting in a second hypervector; and
processing the second hypervector.
3. The method of claim 2 , the processing of the bundled hypervector being performed in parallel by the set of resonator networks.
4. The method of claim 1 , the processing comprising:
ranking the N resonator networks;
scaling the bundled hypervector by the number N of resonator networks;
sequentially processing the scaled bundled hypervector in accordance with the ranking; wherein the processing by a current resonator network comprises:
producing a second hypervector by subtracting from the scaled bundled hypervector zero or more outputs of the zero or more preceding resonator networks;
processing by the resonator network the second hypervector.
5. The method of claim 4 , the ranking of the resonator network being performed such that an ordering of subtractions is applied to prevent noise amplification in the second hypervector.
6. The method of claim 1 , wherein a search space is defined by the number of candidate code hypervectors each having a size D, the method further comprising: increasing the size D of the candidate hypervectors while maintaining the same search space.
7. The method of claim 1 , comprising preventing access to a permutation of a given resonator network by the other resonator networks.
8. The method of claim 1 , the iterative process comprising for each concept of the set of concepts: computing an unbound version of a hypervector representing the concept, computing a similarity vector indicating a similarity of the unbound version with each candidate code hypervector of the concept, and generating an estimate of a hypervector representing the concept by a linear combination of the candidate code hypervectors, with weights given by the similarity vector, followed by an application of a non-linear function.
9. The method of claim 8 , wherein the non-linear function is a sign function.
10. The method of claim 1 , the encoder being a feed forward neural network.
11. The method of claim 1 , the data structure being an image, the codebooks representing the concept of colours, the concept of shapes, the concept vertical positioning and the concept of horizontal positioning.
12. A resonator network system comprising:
a set of resonator networks, each resonator network being configured to receive an input hypervector representing a data structure and to perform an iterative process in order to factorize the input hypervector into individual hypervectors representing a set of cognitive concepts respectively utilizing superposition and clean-up memory, the set of N resonator networks being associated with N permutations respectively;
the resonator network system being configured for applying the N permutations to N first hypervectors respectively, the N first hypervectors representing a set of N data structures respectively; and combining the N permuted hypervectors into a bundled hypervector; and
the resonator networks being configured for processing the bundled hypervector, thereby simultaneously factorizing the N first hypervectors.
13. The system of claim 12 , further comprising an encoder, the encoder being configured for representing data structures in a vector space, the vector space being defined by a set of matrices, referred to herein as codebooks, which encode the set of concepts respectively, the codebooks comprising candidate code hypervectors representing items of the respective concept; the encoder being configured for representing the encoder the set of N data structures by the N first hypervectors respectively.
14. The system of claim 12 , each resonator network being configured for processing the bundled hypervector by:
applying to the bundled hypervector an inverse permutation of the permutation associated with the resonator network resulting in a second hypervector; and processing the second hypervector.
15. The system of claim 12 , being configured for scaling the bundled hypervector by the number N of resonator networks; wherein each resonator network is configured for:
producing a second hypervector by subtracting from the scaled bundled hypervector zero or more outputs of the zero or more resonator networks preceding the resonator network; and
processing by the resonator network the second hypervector.
16. The system of claim 12 , the iterative process comprising for each concept of the set of concepts:
computing an unbound version of a hypervector representing the concept,
computing a similarity vector indicating a similarity of the unbound version with each candidate code hypervector of the concept, and
generating an estimate of a hypervector representing the concept by a linear combination of the candidate code hypervectors, with weights given by the similarity vector, followed by an application of a non-linear function.