Cross-lingual document analysis and translation system

1 . A system, comprising:

a network interface configured to retrieve a first plurality of multilingual documents and a second plurality of multilingual documents;

a document generation component configured to generate a cross-lingual mixed document based on the first plurality of multilingual documents, wherein the document generation component is configured to generate the cross-lingual mixed document using a paragraph-based document extraction and mixing process, comprising:

selecting, from the first plurality of multilingual documents, a source document in a source language and a target document in a target language, wherein the source document corresponds to a source HTML document tree and the target document corresponds to a target HTML document tree;

extracting textual contents at a paragraph level by iterating simultaneously through tree nodes of the source HTML document tree and the target HTML document tree;

concatenating the extracted textual contents associated with the tree node nested under one paragraph tag to rebuild a complete paragraph;

applying an alignment algorithm between the extracted textual contents in the source language and the extracted textual contents in the target language to obtain parallel sentence-aligned pairs;

mixing the source document and the target document at a sentence level based on the parallel sentence-aligned pairs;

extracting paragraphs under paragraph tags from the source document and the target document to generate a source corpus and a target corpus;

leveraging a sentence aligner to identify bilingual equivalent sentence pairs each comprising a source sentence and a target sentence;

iterating the bilingual equivalent sentence pairs to replace each sourced sentence by a corresponding target sentence with an assigned exchange probability less than a predefined threshold; and

outputting a cross-lingual HTML document;

an artificial intelligence (AI) driven document processing component configured to:

retrain a pre-trained machine learning model to learn cross-lingual relations based on the cross-lingual mixed document; and

translate the second plurality of multilingual documents based on the cross-lingual relations; and

one or more task-specific components configured to complement the AI-driven document processing component to implement one or more specific tasks.

2 . The system of claim 1 , wherein the first plurality of multilingual documents and the second plurality of multilingual documents comprise domain specific content in a plurality of languages and in a plurality of types comprising emails, forms, receipts, PDF files, Excel files, or other types of scanned or digital versions of documents.

3 . The system of claim 2 , wherein the domain specific content comprises energy industry multilingual data, wherein the energy industry multilingual data comprises energy industry related terminology, diagrams, special format, tables, or special units and notations.

4 . The system of claim 1 , wherein the pre-trained machine learning model comprises a LayoutXLM model.

5 . The system of claim 4 , wherein the LayoutXLM model is pre-trained using a plurality of multilingual documents comprising a plurality of languages, wherein the plurality of languages comprises at least Chinese, Japanese, Spanish, French, Italian, German, and Portuguese.

6 . The system of claim 4 , wherein the pre-trained machine learning model is retrained to learn cross-lingual relations based on textual information from the cross-lingual mixed document and layout information extracted from the plurality of multilingual documents.

7 . The system of claim 1 , wherein the one or more task-specific components comprises a document analysis element configured to identify and extract energy domain-specific contents from the first plurality of multilingual documents and the second plurality of multilingual documents, wherein the energy domain-specific contents comprise energy field terminology and vocabularies, diagrams and tables, specialized formatting, notations, formulations, and expressions.

8 . The system of claim 7 , wherein the extracted energy domain-specific contents are stored in a structured format.

9 . The system of claim 1 , wherein the one or more task-specific components comprises a document classification element configured to assign one or more predefined categories or classes for each document of the first plurality of multilingual documents and the second plurality of multilingual documents.

10 . The system of claim 9 , wherein the AI-driven document processing component is configured to use the one or more predefined categories or classes to:

organize a collection of energy industry domain-specific documents comprising the first plurality of multilingual documents and the second plurality of multilingual documents;

filter out irrelevant or spam documents from the collection of energy industry domain-specific documents; and

route each of the collection of energy industry domain-specific documents to a related section or user within an organization.

11 . The system of claim 1 , wherein the one or more task-specific components comprises a document translation element configured to:

select one or more target languages; and

cause the AI-driven document processing component to translate each of the second plurality of multilingual documents into a respective language of the one or more target languages.

12 . The system of claim 11 , wherein selecting the one or more target languages is based on one or more criteria comprising user preference or source languages of the second plurality of multilingual documents.

13 . The system of claim 12 , wherein each of the second plurality of multilingual documents is translated into the respective language of the one or more target languages with the same or similar structured layout comprising position, size, and orientation information of texts, images, tables, and other elements in the second plurality of multilingual documents.

14 . The system of claim 1 , wherein the paragraph-based document extraction and mixing process comprises:

organizing the extracted textual contents in a table comprising a plurality of data cells; and

extracting each data cell of the plurality of data cells as one linguistic paragraph.