Systems and methods to determine and utilize semantic relatedness between multiple natural language sources to determine strengths and weaknesses
A microprocessor executable method transforms unstructured natural language texts by way of a preprocessing pipeline into a structured data representation of the entities described in the original text. The structured data representation is conducive to further processing by machine methods. The transformation process is learned by a machine learned model trained to identify relevant text segments and disregard irrelevant text segments The resulting structured data representation is refined to more accurately represent the respective entities.
1. A microprocessor executable method to transform unstructured natural language texts by way of a preprocessing pipeline, into a structured data representation of the entities described in the original text wherein, the structured data representation is conducive to further processing by machine methods, and the process of the transformation is learned by a machine neural network or other machine learned model trained to identify relevant text segments and disregard irrelevant text segments such that the resulting structured data representation is refined to more accurately represent the respective entities.
2. A microprocessor executable method decomposing a natural language document into a sequence of text excerpts or segments, the microprocessor executable method comprising:
dividing the text into a sequence of small fragments;
using a machine learned model to classify each possible recombination of those fragments; and,
optimizing over the possible result sequences to obtain an ideal segmentation, wherein the optimizing step further comprises relation detection using our proposed bipartite graph-based optimization.