Bundling hypervectors
View Patent ↗Embodiments are disclosed for a method. The method includes bundling a set of M code hypervectors, each of dimension D, where M>1. The bundling includes receiving an M-dimensional vector comprising weights for weighting the set of code hypervectors. The bundling further includes mapping the M-dimensional vector to an S-dimensional vector, s k , such that each element of the S-dimensional vector, s k , indicates one of the set of code hypervectors, where S=D/L and L≥1. Additionally, the bundling includes building a hypervector such that an ith element of the built hypervector is an ith element of the code hypervector indicated in an ith element of the S-dimensional vector, s k .
1 . A system comprising:
a computer processing circuit; and
a computer-readable storage medium storing instructions, which, when executed by the computer processing circuit, are configured to cause the computer processing circuit to perform a method comprising:
encoding, by an encoder, a data structure so that the data structure is represented by a hypervector, wherein the encoder is a feed-forward neural network that is trained to produce the hypervector as a compound hypervector describing the data structure, and wherein encoding comprises:
bundling a set of M code hypervectors, each of dimension D, where M>1, by:
receiving an M-dimensional vector comprising a plurality of weights for weighting the set of code hypervectors;
mapping the M-dimensional vector to an S-dimensional vector, s k , such that each element of the S-dimensional vector, s k , indicates one of the set of code hypervectors, where S=D/L and L≥1; and
building the hypervector such that an ith element of the built hypervector is an ith element of the code hypervector indicated in an ith element of the S-dimensional vector, s k ; and
decoding, by a resonator network, the hypervector that is encoded in a vector space.
2 . The system of claim 1 , wherein the hypervector comprises binary values {0, 1} D , and wherein the hypervector comprises a sparsity smaller than a sparsity threshold, and wherein the sparsity is equal to S/D, and wherein the hypervector comprises a fraction of non-zero values.
3 . The system of claim 2 , wherein the sparsity threshold is in a range of: 0.3% to 50%.
4 . The system of claim 1 , wherein the S-dimensional vector, s k , is defined as: for every i th element, generating a value v, and determining the i th element with a mapping function:
s
k
(
i
)
=
{
0
if
v
≤
∂
0
…
m
if
∂
m
-
1
<
v
≤
∂
m
where v comprises a randomly generated value or a deterministic value defined as v=i/S, where ∂=cumsum(a)/sum(a) is a step function, where a is the M-dimensional vector, where cumsum(a) returns a vector containing a cumulative sum of a plurality of elements of vector a, and sum(a) returns a sum of elements of vector a, and ∂ m refers to an m th element of ∂.
5 . The system of claim 1 , the method further comprising repeating the mapping and the building, a p number of times, to generate p built hypervectors, and using the p built hypervectors to determine a hypervector that represents a bundling version of the set of code hypervectors.
6 . The system of claim 1 , wherein the bundling is performed for online hyperdimensional learning, wherein the set of code hypervectors comprises an encoded hypervector and a model hypervector, and wherein the M-dimensional vector comprises weights w and 1−w, and wherein the built hypervector is provided as an update of the model hypervector, and w is a scalar.
7 . The system of claim 1 , wherein the built hypervector is a vectorized representation of a probability mass function (PMF), and wherein the M-dimensional vector comprises a plurality of values in the PMF, and the set of code hypervectors are basis vectors of a codebook for representing the PMF in a vector space.
8 . A method comprising:
encoding, by an encoder, a data structure so that the data structure is represented by a hypervector, wherein the encoder is a feed-forward neural network that is trained to produce the hypervector as a compound hypervector describing the data structure, and wherein encoding comprises:
bundling a set of M code hypervectors, each of dimension D, where M>1, by:
receiving an M-dimensional vector comprising a plurality of weights for weighting the set of code hypervectors;
mapping the M-dimensional vector to an S-dimensional vector, s k , such that each element of the S-dimensional vector, s k , indicates one of the set of code hypervectors, where S=D/L and L≥1; and
building the hypervector such that an ith element of the built hypervector is an ith element of the code hypervector indicated in an ith element of the S-dimensional vector, s k ; and
decoding, by a resonator network, the hypervector that is encoded in a vector space.
9 . The method of claim 8 , wherein the hypervector comprises binary values {0, 1} D , and wherein the hypervector comprises a sparsity smaller than a sparsity threshold, and wherein the sparsity is equal to S/D, and wherein the hypervector comprises a fraction of non-zero values.
10 . The method of claim 9 , wherein the sparsity threshold is in a range of: 0.3% to 50%.
11 . The method of claim 8 , wherein the S-dimensional vector, s k , is defined as: for every i th element, generating a value v, and determining the i th element with a mapping function:
s
k
(
i
)
=
{
0
if
v
≤
∂
0
…
m
if
∂
m
-
1
<
v
≤
∂
m
where v comprises a randomly generated value or a deterministic value defined as v=i/S, where ∂=cumsum(a)/sum(a) is a step function, where a is the M-dimensional vector, where cumsum(a) returns a vector containing a cumulative sum of a plurality of elements of vector a, and sum(a) returns a sum of elements of vector a, and ∂ m refers to an m th element of ∂.
12 . The method of claim 8 , further comprising repeating the mapping and the building a p number of times to generate p built hypervectors, and using the p built hypervectors to determine a hypervector that represents a bundling version of the set of code hypervectors.
13 . The method of claim 8 , wherein the bundling is performed for online hyperdimensional learning, wherein the set of code hypervectors comprises an encoded hypervector and a model hypervector, and wherein the M-dimensional vector comprises weights w and 1−w, and wherein the built hypervector is provided as an update of the model hypervector, and w is a scalar.
14 . The method of claim 8 , wherein the built hypervector is a vectorized representation of a probability mass function (PMF), and wherein the M-dimensional vector comprises a plurality of values in the PMF, and the set of code hypervectors are basis vectors of a codebook for representing the PMF in a vector space.
15 . A computer program product including program instructions stored on a computer readable storage medium, the program instructions executable by a processor to cause the processor to perform a method comprising:
encoding, by an encoder, a data structure so that the data structure is represented by a hypervector, wherein the encoder is a feed-forward neural network that is trained to produce the hypervector as a compound hypervector describing the data structure, and wherein encoding comprises:
bundling a set of M code hypervectors, each of dimension D, where M>1, by:
receiving an M-dimensional vector comprising a plurality of weights for weighting the set of code hypervectors;
mapping the M-dimensional vector to an S-dimensional vector, s k , such that each element of the S-dimensional vector, s k , indicates one of the set of code hypervectors, where S=D/L and L≥1; and
building the hypervector such that an ith element of the built hypervector is an ith element of the code hypervector indicated in an ith element of the S-dimensional vector, s k ; and
decoding, by a resonator network, the hypervector that is encoded in a vector space.
16 . The computer program product of claim 15 , wherein the hypervector comprises binary values {0, 1} D , and wherein the hypervector comprises a sparsity smaller than a sparsity threshold, and wherein the sparsity is equal to S/D, and wherein the hypervector comprises a fraction of non-zero values.
17 . The computer program product of claim 16 , wherein the sparsity threshold is in a range of: 0.3% to 50%.
18 . The computer program product of claim 15 , wherein the S-dimensional vector, s k , is defined as: for every i th element, generating a value v, and determining the i th element with a mapping function:
s
k
(
i
)
=
{
0
if
v
≤
∂
0
…
m
if
∂
m
-
1
<
v
≤
∂
m
where v comprises a randomly generated value or a deterministic value defined as v=i/S, where ∂=cumsum(a)/sum(a) is a step function, where a is the M-dimensional vector, where cumsum(a) returns a vector containing a cumulative sum of a plurality of elements of vector a, and sum(a) returns a sum of elements of vector a, and ∂ m refers to an m th element of ∂.
19 . The computer program product of claim 15 , the method further comprising repeating the mapping and the building a p number of times to generate p built hypervectors, and using the p built hypervectors to determine a hypervector that represents a bundling version of the set of code hypervectors.
20 . The computer program product of claim 15 , wherein the bundling is performed for online hyperdimensional learning, wherein the set of code hypervectors comprises an encoded hypervector and a model hypervector, and wherein the M-dimensional vector comprises weights w and 1−w, and wherein the built hypervector is provided as an update of the model hypervector, and w is a scalar.