Methods and systems for identifying a level of similarity between a plurality of data representations
A method for identifying a level of similarity between binary vectors includes storing, by a processor on a computing device, in each of a plurality of memory cells on the computing device, one of a plurality of binary vectors, each of the plurality of memory cells including a bitwise comparison circuit. The processor provides, to each of the plurality of memory cells, a received binary vector. Each of the bitwise comparison circuits determines a level of overlap between the received binary vector and the binary vector stored in the memory cell associated with the bitwise comparison circuit. Each of the comparison circuits that determines that the level of overlap satisfies a threshold provides, to the processor, an identification of the stored binary vector with the satisfactory level of overlap. The processor provides an identification of each stored binary vector satisfying the threshold.
1. A computer-implemented method for identifying a level of similarity between a first log data item and a log data item within a set of data documents, the method comprising:
clustering, by a reference map generator executing on a first computing device, in a two-dimensional metric space, a set of data documents selected according to at least one criterion, generating a semantic map;
associating, by the semantic map, a coordinate pair with each of the set of data documents;
generating, by a parser executing on the first computing device, an enumeration of log data items occurring in the set of data documents;
determining, by a representation generator executing on the first computing device, for each log data item in the enumeration, occurrence information including: (i) a number of data documents in which the log data item occurs, (ii) a number of occurrences of the log data item in each data document, and (iii) the coordinate pair associated with each data document in which the log data item occurs;
generating, by the representation generator, for each log data item in the enumeration, a sparse distributed representation (SDR) using the occurrence information, resulting in a plurality of generated SDRs;
storing, by a processor on a second computing device, in each of a plurality of memory cells on the second computing device, one of the plurality of generated SDRs, each of the plurality of memory cells including a bitwise comparison circuit;
receiving, by the second computing device, from a third computing device, a first log data item;
providing, by the processor, via a data bus, to each of the plurality of memory cells, an SDR of the first log data item;
determining, by each of the plurality of bitwise comparison circuits, a level of overlap between the SDR of the first log data item and the generated SDR stored in the memory cell associated with the bitwise comparison circuit;
determining, by each of the plurality of bitwise comparison circuits, whether the level of overlap satisfies a threshold provided by the processor;
providing, to the processor, by each of the comparison circuits that determined the level of overlap did satisfy the threshold, a document reference number stored in the associated memory cell, the document reference number identifying a document including the log data item from which the SDR stored in the memory cell was generated; and
providing, by the second computing device, to the third computing device, an identification of each log data item from which the SDRs stored in the memory cells satisfying the threshold were generated and a level of similarity between the log data item from which the stored SDR was generated and the received log data item.
2. A computer-implemented method for identifying a level of similarity between a first sensor data item and a sensor data item within a set of data documents, the method comprising:
clustering, by a reference map generator executing on a first computing device, in a two-dimensional metric space, a set of data documents selected according to at least one criterion, generating a semantic map;
associating, by the semantic map, a coordinate pair with each of the set of data documents;
generating, by a parser executing on the first computing device, an enumeration of sensor data items occurring in the set of data documents;
determining, by a representation generator executing on the first computing device, for each sensor data item in the enumeration, occurrence information including: (i) a number of data documents in which the sensor data item occurs, (ii) a number of occurrences of the sensor data item in each data document, and (iii) the coordinate pair associated with each data document in which the sensor data item occurs;
generating, by the representation generator, for each sensor data item in the enumeration, a sparse distributed representation (SDR) using the occurrence information, resulting in a plurality of generated SDRs;
storing, by a processor on a second computing device, in each of a plurality of memory cells on the second computing device, one of the plurality of generated SDRs, each of the plurality of memory cells including a bitwise comparison circuit;
receiving, by the second computing device, from a third computing device, a first sensor data item;
providing, by the processor, via a data bus, to each of the plurality of memory cells, an SDR of the first sensor data item;
determining, by each of the plurality of bitwise comparison circuits, a level of overlap between the SDR of the first sensor data item and the generated SDR stored in the memory cell associated with the bitwise comparison circuit;
determining, by each of the plurality of bitwise comparison circuits, whether the level of overlap satisfies a threshold provided by the processor;
providing, to the processor, by each of the comparison circuits that determined the level of overlap did satisfy the threshold, a document reference number stored in the associated memory cell, the document reference number identifying a document including the sensor data item from which the SDR stored in the memory cell was generated; and
providing, by the second computing device, to the third computing device, an identification of each sensor data item from which the SDRs stored in the memory cells satisfying the threshold were generated and a level of similarity between the sensor data item from which the stored SDR was generated and the received sensor data item.