Methods and systems for identifying a level of similarity between a plurality of data representations
A reference map generator clusters, into a semantic map, a set of data documents selected according to at least one criterion and associated with a medical diagnosis. A parser generates an enumeration of measurements occurring in the set of data documents. A representation generator generates for each measurement in the enumeration, a sparse distributed representation (SDR). The method includes storing, by a processor on a second computing device, in each of a plurality of memory cells on the second computing device, one of the generated SDRs. A diagnosis support module receives a document comprising a plurality of measurements. The representation generator generates a compound SDR for the document. Each of the plurality of bitwise comparison circuits determine a level of overlap between the compound SDR and the stored generated SDR. The diagnosis support module provides an identification of the medical diagnosis associated with a stored SDR.
1. A computer-implemented method for identifying a level of similarity between a first data item and a data item within a set of data documents, the method comprising:
clustering, by a reference map generator executing on a first computing device, in a two-dimensional metric space, a set of data documents selected according to at least one criterion and associated with a medical diagnosis, generating a semantic map;
associating, by the semantic map, a coordinate pair with each of the set of data documents;
generating, by a parser executing on the first computing device, an enumeration of measurements occurring in the set of data documents;
determining, by a representation generator executing on the first computing device, for each measurement in the enumeration, occurrence information including: (i) a number of data documents in which the measurement occurs, (ii) a number of occurrences of the measurement in each data document, and (iii) the coordinate pair associated with each data document in which the measurement occurs;
generating, by the representation generator, for each measurement in the enumeration, a sparse distributed representation (SDR) using the occurrence information, resulting in a plurality of generated SDRs;
storing, by a processor on a second computing device, in each of a plurality of memory cells on the second computing device, one of the plurality of generated SDRs, each of the plurality of memory cells including a bitwise comparison circuit;
receiving, by a diagnosis support module executing on the first computing device, from a third computing device, a document comprising a plurality of measurements, the document associated with a medical patient;
generating, by the representation generator, at least one SDR for the plurality of measurements;
generating, by the representation generator, a compound SDR for the document, based on the at least one SDR generated for the plurality of measurements;
providing, by the processor on the second computing device, via a data bus, to each of the plurality of memory cells, the compound SDR;
determining, by each of the plurality of bitwise comparison circuits, a level of overlap between the compound SDR and the generated SDR stored in the memory cell associated with the bitwise comparison circuit;
determining, by each of the plurality of bitwise comparison circuits, whether the level of overlap satisfies a threshold provided by the processor;
providing, to the processor on the second computing device, by each of the comparison circuits that determined the level of overlap did satisfy the threshold, a document reference number stored in the associated memory cell, the document reference number identifying a document including the measurement from which the SDR stored in the memory cell was generated; and
providing, by the first computing device, to the third computing device, (i) an identification of each measurement from which the SDRs stored in the memory cells satisfying the threshold were generated and (ii) a level of similarity between the measurement from which the stored SDR was generated and the received document comprising the plurality of measurements and (iii) an identification of the medical diagnosis associated with the stored SDR, based on the determined level of semantic similarity.
2. The method of claim 1 further comprising receiving, prior to clustering, access to the set of data documents, at least one of the set of documents including a set of lab values taken from a sample.
3. The method of claim 1 , wherein generating, by the parser, the enumeration of measurements occurring in the set of data documents further comprises receiving, by the parser, from a binning module, the enumeration of measurements.
4. The method of claim 3 further comprising:
identifying, by the binning module, a range of values for each of a plurality of types of measurement included in the set of data documents;
receiving, by the binning module, user input specifying a method for distributing the range of values; and
distributing, by the binning module, the range of values into one or more sub-groups, responsive to the received user input.