Federated system and method for analyzing language coherency, conformance, and anomaly detection
Aspects of the present disclosure involve systems and methods for evaluating a piece of text or document against many corpuses of text or documents located on sources which may be the same and/or different from the text of interest in a tensorized manner and aggregating the coherence/anomaly score against some or all of the entire corpus. This joining of multiple data sources for evaluating the given piece or text may be a “federated” system as disparate data sources, each of which may contain confidential or otherwise private information, may be considered as a single repository of texts or documents. The systems and methods provide for a coherency and/or anomaly check of a piece of text of a document against similar pieces of text to determine a similarity of the piece of text to a large corpus of documents stored in disparate locations.
1. A system for processing an electronic document, the system comprising:
a processor; and
a memory comprising instructions that, when executed, cause the processor to:
transmit an initial tensor generated from a text portion of an electronic document and a relative location of the text portion to a plurality of computing environments, each of the plurality of computing environments hosting a language coherency system remotely located separate from the processor and configured to calculate a comparison score based on a comparison of the initial tensor to a corpus of local electronic documents of a respective language coherency system;
generate a distribution of comparison scores received from each of the language coherency systems;
execute a risk scoring model to generate an assigned risk score of the initial tensor based on the comparison of the initial tensor to the corpus of local electronic documents, the assigned risk score associated with the comparison scores received from each of the language coherency systems;
display, on a display device, the distribution of comparison scores for the initial tensor for comparison of the text portion to a plurality of corresponding tensors from the corpus of local electronic documents and the assigned risk score; and
identify, based on the distribution of comparison scores, a similarity of the text portion of the electronic document to the corpus of local electronic documents of the respective language coherency systems while maintaining inaccessibility of the corpus of local electronic documents by the processor.
2. The system of claim 1 wherein the instructions further cause the processor to:
execute a conversion algorithm to convert the text portion of the electronic document into the initial tensor.
3. The system of claim 2 wherein the conversion algorithm is one of a hashing algorithm, a term frequency-inverse document frequency (tf-idf) algorithm, or a trained machine learning-based embedding model.
4. The system of claim 1 wherein the instructions further cause the processor to:
receive the initial tensor from a computing device in communication with the processor.
5. The system of claim 1 wherein the language coherency system is further configured to:
identify a portion of the corpus of local electronic documents of a same type as the text portion of the electronic document; and
generate the corresponding tensors based on the identified portion of the corpus of local electronic documents.
6. The system of claim 5 wherein calculating the comparison score comprises:
executing a distance-based scoring algorithm to determine a plurality of distance calculations in a dimensional ontological space, each of the plurality of distance calculations corresponding to a similarity of the text portion of the electronic document associated with the initial tensor to the identified portion of the corpus of local electronic documents.
7. The system of claim 1 wherein the plurality of computing environments is one of a public cloud computing environment, a private cloud computing environment, or a private tenant network.
8. The system of claim 1 wherein the distribution of comparison scores indicates a similarity of the text portion of the electronic document to the corpus of local electronic documents.
9. A method for processing a portion of an electronic document, the method comprising:
executing, via a processing device, a conversion algorithm to convert a text portion of the electronic document into an initial tensor;
transmitting the initial tensor and a relative location of the text portion to a plurality of computing environments different than the processing device, each of the plurality of computing environments hosting a language coherency system to calculate a similarity score through a comparison of the initial tensor to a corpus of local electronic documents of a respective language coherency system;
generating, via the processing device, a distribution of similarity scores generated by each of the language coherency systems based on the comparison of the initial tensor to a plurality of comparison tensors of local electronic documents; and
executing a risk scoring model to generate an assigned risk score of the initial tensor based on the comparison of the initial tensor to the corpus of local electronic documents, the assigned risk score associated with the comparison scores received from each of the language coherency systems.
10. The method of claim 9 further comprising:
executing a conversion algorithm to convert the text portion of the electronic document into the initial tensor.
11. The method of claim 10 wherein the conversion algorithm is one of a hashing algorithm, a term frequency-inverse document frequency (tf-idf) algorithm, or a trained machine learning-based embedding model.
12. The method of claim 9 further comprising:
identifying a portion of the corpus of local electronic documents of a same type as the text portion of the electronic document; and
generating the comparison tensors based on the identified portion of the corpus of local electronic documents.
13. The method of claim 12 wherein calculating the similarity score comprises:
executing a distance-based scoring algorithm to determine a plurality of distance calculations in a dimensional ontological space, each of the plurality of distance calculations corresponding to a similarity of the text portion of the electronic document associated with the initial tensor to the identified portion of the corpus of local electronic documents.
14. The method of claim 9 wherein the plurality of computing environments is one of a public cloud computing environment, a private cloud computing environment, or a private tenant network.
15. The method of claim 9 wherein the distribution of comparison scores indicates a similarity of the text portion of the electronic document to the corpus of local electronic documents.
16. One or more non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:
executing, via a processing device, a conversion algorithm to convert a text portion of a received electronic document into an initial tensor;
transmitting the initial tensor and a relative location of the text portion to a plurality of computing environments, each of the plurality of computing environments hosting a language coherency system to calculate a similarity score through a comparison of the initial tensor to a corpus of local electronic documents of a respective language coherency system;
generating, via the processing device, a distribution of similarity scores generated by each of the language coherency systems based on the comparison of the initial tensor to a plurality of comparison tensors of local electronic documents; and
executing a risk scoring model to generate an assigned risk score of the initial tensor based on the comparison of the initial tensor to the corpus of local electronic documents, the assigned risk score associated with the comparison scores received from each of the language coherency systems.
17. The one or more non-transitory computer-readable storage media of claim 16 storing computer-executable instructions for performing the computer process on the computing system, the computer process further comprising:
executing a conversion algorithm to convert the text portion of the electronic document into the initial tensor, the conversion algorithm is one of a hashing algorithm, a term frequency-inverse document frequency (tf-idf) algorithm, or a trained machine learning-based embedding model.
18. The one or more non-transitory computer-readable storage media of claim 16 storing computer-executable instructions for performing the computer process on the computing system, the computer process further comprising:
identifying a portion of the corpus of local electronic documents of a same type as the text portion of the electronic document; and
generating the comparison tensors based on the identified portion of the corpus of local electronic documents, wherein calculating the similarity score comprises executing a distance-based scoring algorithm to determine a plurality of distance calculations in a dimensional ontological space, each of the plurality of distance calculations corresponding to a similarity of the text portion of the electronic document associated with the initial tensor to the identified portion of the corpus of local electronic documents.