Automated translation of subject matter specific documents
Documents in source natural languages are translated into target natural languages using a computer-implemented translation that is configured to operate within the domain of the subject matter of the documents that imposes specialized requirements for translation and readability. Subject matter specific documents typically include domain-specific terminology, are subject to various regulatory guidelines, and have different readability requirements depending on the intended reader. The computer-implemented translation applies machine-learning techniques that deconstruct elements of the subject matter specific document into a standard data structure and perform pre-processing steps to tokenize digitized document text to identify the correct sentence structure and syntax for the target natural language to optimize translation by, e.g., a neural machine translation engine. The text segments that are input into the neural machine translation engine are generated to be semantically meaningful in the target natural language to thereby enhance the understanding of the neural machine translation engine.
1. A computer-implemented method comprising:
splitting sentences in a digitized text of a stored document into segments;
ordering words in the segmented sentences having reduced complexity relative to the sentences prior to splitting;
at least partially translating the segments to a target natural language by matching the ordered segments to segments in a database of documents previously translated from a source natural language, wherein content of the documents have similar subject matter as the new document;
producing a single representation of the sentences that share a common meaning by applying transformational grammar to the digitized text; and
outputting a representation of the stored document that includes a semantic meaning in the target natural language.
2. The method of claim 1 , wherein the step of splitting sentences determines tokens that identify key sentence structures.
3. The method of claim 1 , wherein the step of splitting sentences maintains semantic content of the sentences before they are split.
4. The method of claim 1 , further comprising steps of:
recognizing words in the digitized text that match entries in a named entity table; and
classifying the recognized words into pre-defined classes.
5. The method of claim 4 , wherein the named entity table comprises one or more of proper nouns, abbreviations, acronyms; and
the pre-defined classes comprise expressions of one or more of subject matter, sponsor, patron, location, organization, date, address, and time.
6. The method of claim 1 , further comprising a step of marking portions of the digitized text for exclusion from translation to the target natural language.
7. The method of claim 1 , further comprising a step of masking confidential information in the digitized text.
8. The method of claim 1 , wherein, in the step of partially translating, the matching uses fuzzy logic that is less than 100 percent accurate.
9. The method of claim 1 , wherein the applying transformational grammar includes detecting and transforming the detected passive voice sentences into active voice sentences.
10. The method of claim 1 , wherein the applying transformational grammar includes detecting and transforming the detected indirect sentence form into a direct sentence form.
11. The method of claim 1 , wherein the applying transformational grammar includes re-ordering words in the sentences based on sentence structure requirements of the target natural language.
12. The method of claim 1 , further comprising a step of providing a user interface (UI) to enable a human to manually adjust one or more of splitting sentences, named entity recognition, matching database segments, and applying transformational grammar.
13. The method of claim 12 , further comprising a step of executing a supervised machine learning process that accepts the manual adjustments as input.
14. The method of claim 1 , wherein the representation of the new document is provided to a neural machine translation engine.
15. The method of claim 1 , wherein a user-defined glossary is used to translate the identified words that match the entries in the named entity table.
16. The method of claim 1 , further comprising a step of identifying and tagging parts of speech in the segments.
17. A computer-implemented method comprising:
splitting the digitized text in a new document into segments by identifying sentence boundaries using a gazetteer list of abbreviations to identify sentence marking stops;
identifying, using named entity recognition, the digitized text that is excluded from translation to a target natural language;
searching, using fuzzy matching, a translation history from the source natural language to the target natural language, for the segments between and existing translations;
identifying and tagging parts of speech in the digitized text;
grammatically transforming the digitized text to provide a single representation of sentences that have a common meaning;
over an application programming interface (API):
transmitting the segments to an external translation engine for translation;
receiving a translation of the segments in the target natural language from the external translation engine;
correcting the translation for subject matter specific acronyms and/or subject matter specific terminology; and
reconstructing the new document using the corrected translation in the target natural language.
18. The method of claim 17 , further comprising a step of translating the new document using adjustable machine learning processes.
19. A non-transitory computer-readable storage media storing executable instructions which, when executed by a processor steps of:
generating digitized text by deconstructing elements of one or more new documents into a data structure;
optimizing translation of the one or more new documents from a source language to a target language by pre-processing the generated digitized text into tokens, wherein the pre-processing identifies (1) key sentence structures that express relationships within a semantic domain of one of a subject matter specific terminology, and (2) a token order for translation; and
providing the pre-processed digitized text token by token in an identified order to a neural machine translation engine.
20. The non-transitory computer-readable storage media of claim 19 , further comprising steps of:
receiving, from the neural machine translation engine, translated tokens in the target language;
processing the received translated tokens to correct ontology in the semantic domain of the one of subject matter specific terminology; and
reconstructing the new documents using the translated tokens with the corrected semantic ontology in which the reconstructed new documents in the target language maintain characteristics of the original new documents in the source language, the characteristics including one of formatting or embedded images.