Universal data language translator
The present disclosure is directed to a universal data language (UDL) translator. Specifically, the systems and methods disclosed enable input data from a variety of sources to be translated into a UDL that can be consistently analyzed and compared against other sources of data. For example, an entity may upload input data that has a plurality of data terms and definitions (e.g., header column in a spreadsheet). These terms may be duplicative and/or inaccurate with respect to the underlying data. If the entity wishes to compare and transact data within a data marketplace, the entity may not fully comprehend what data it is missing and/or what data another entity may have to offer for trade. To remedy this problem of business semantic management, the present invention discloses steps for creating a UDL and a UDL translator so that any input data can be translated to UDL.
1 . A system for translating input data into a universal data language, comprising:
a memory configured to store non-transitory computer readable instructions; and
a processor communicatively coupled to the memory, wherein the processor, when
executing the non-transitory computer readable instructions, is configured to:
receive input data from at least one trusted source, wherein the input data comprises at least one data term;
compare the at least one data term to at least one universal data language (UDL) library, wherein the at least one UDL library comprises a UDL entry for the at least one data term, wherein the at least one UDL library is comprised of a plurality of definitions from the at least one trusted source;
train at least one machine learning algorithm on the at least one UDL library;
process the at least one data term using the at least one machine learning algorithm to calculate a similarity score based on i) a similarity of context characteristics between the at least one data term and at least one UDL term in the at least one UDL library, and ii) a source of the at least one data term; and
if the similarity score is above a similarity threshold, translate the at least one data term to the at least one UDL term in the at least one UDL library.
2 . The system of claim 1 , wherein the processor is further configured to:
determine if the at least one data term is a duplicate of an already-existing UDL term in the at least one UDL library; and
based on the at least one data term being determined to be the duplicate, map the at least one data term to the already-exiting UDL term.
3 . The system of claim 2 , wherein determining if the at least one data term is the duplicate of the already-existing UDL term comprises comparing each word or each character in the at least one data term with each word or each character with the already-existing UDL term.
4 . The system of claim 2 , wherein determining if the at least one data term is the duplicate of the already-existing UDL term comprises comparing a definition of the at least one data term to a definition to the already-existing UDL term.
5 . The system of claim 4 , wherein the definition of the at least one data term is at least one equation.
6 . The system of claim 1 , wherein the at least one trusted source is at least one of: a publisher of a business glossary, a publisher of a dictionary, and a publisher of industry-specific ontology.
7 . The system of claim 1 , wherein the processor is further configured to: format the at least one data term to conform to a formatting for the at least one UDL term.
8 . The system of claim 1 , wherein processing the at least one data term further comprises evaluating a level of accuracy of the at least one data term.
9 . The system of claim 8 , wherein the level of accuracy is determined based on at least one of: an inactivity score and an outdatedness score.
10 . A method of creating a universal data language (UDL) translator, comprising:
receiving input data from at least one trusted source, wherein the input data comprises a plurality of data terms from at least one business glossary;
analyzing lexical features of each data term in the plurality of data terms;
analyzing contextual features of each data term in the plurality of data terms;
extracting at least one semantic ontology for each data term in the plurality of data terms;
creating a UDL library, wherein the UDL library comprises a UDL entry for each data term in the plurality of data terms;
training at least one machine learning algorithm on the UDL library;
receiving client-specific input data, wherein the client-specific input data comprises a newly-received data term;
processing the newly-received data term using the at least one machine learning algorithm by calculating a similarity score based on i) a similarity of context characteristics between the newly-received data term and at least one UDL term in the UDL library, and ii) a source of the newly-received data term; and
mapping the newly-received data term to at least one UDL term in the UDL library.
11 . The method of claim 10 , wherein the at least one trusted source is at least one of: a publisher of a business glossary, a publisher of an industry-specific glossary, and a publisher of a business ontology.
12 . The method of claim 10 , wherein the similarity score is calculated by comparing each character or each word in the newly-received data term with each character or each word in the at least one UDL term.
13 . The method of claim 10 , wherein the similarity score is calculated by comparing a definition of the newly-received data term to a definition of the at least one UDL term, wherein the definition of the newly-received data term is an equation.
14 . The method of claim 10 , wherein the similarity score is calculated by comparing at least one domain classification of the newly-received data term with at least one domain classification of the at least one UDL term.
15 . The method of claim 10 , further comprising determining if the newly-received data term is a duplicate of a preexisting UDL term.
16 . The method of claim 15 , wherein determining if the newly-received data term is a duplicate of a preexisting UDL term comprises evaluating the similarity score.
17 . The method of claim 10 , wherein the UDL library is comprised of at least one general business ontology, at least one financial ontology, and at least one life sciences ontology.
18 . A non-transitory computer-readable media storing computer executable instructions that when executed cause a computer system to perform a method for translating input data into a universal data language (UDL), comprising:
receiving input data from at least one trusted source, wherein the input data comprises at least one data term;
comparing the at least one data term to at least one universal data language (UDL) library, wherein the at least one UDL library comprises a UDL entry for the at least one data term;
training at least one machine learning algorithm on the at least one UDL library;
processing the at least one data term using the at least one machine learning algorithm to calculate a similarity score based on i) a similarity of context characteristics between the at least one data term and at least one UDL term in the at least one UDL library, and ii) a source of the at least one data term; and
if the similarity score is above a similarity threshold, translating the at least one data term to the at least one UDL term in the at least one UDL library;
determine if the at least one data term is a duplicate of an already-existing UDL term in the at least one UDL library; and
based on the at least one data term being determined to be the duplicate, map the at least one data term to the already-exiting UDL term.