System and method for comparing documents using an artificial intelligence (AI) model
A method for comparing documents using an AI model is disclosed herein. The method includes preprocessing a plurality of input documents into a structured, standardized format for comparison. Further, the method includes generating a layout of components in the plurality of input documents. Furthermore, the method includes generating a canonical representation indicating relationships between the components. Furthermore, the method includes determining sections within each of the plurality of input documents by segmenting the layout Furthermore, the method includes determining a textual difference matrix using the AI model, based on the comparison of the plurality of sections within each of the plurality of input documents, wherein the textual difference matrix indicates alterations in one or more expressions in each of the plurality of sections.
1 . A method for comparing documents using an artificial intelligence (AI) model, the method comprising:
preprocessing a plurality of input documents into a structured, standardized format for comparison;
generating a layout indicative of an arrangement of one or more components in each of the plurality of input documents using the AI model, based on parsing the one or more components,
wherein generating the layout comprises:
predicting a positional coordinate corresponding to each of the one or more components for each of the plurality of input documents, wherein predicting the positional coordinate comprises:
training the AI model with an annotated plurality of training documents based on at least one of a precision value, a recall value, and a Mean Average Precision (mAP) value associated with the AI model;
determining hyperparameters associated with the annotated plurality of training documents based on the training;
learning the positional coordinate corresponding to the one or more components in the plurality of training documents based on the determined hyperparameters; and
predicting the positional coordinate corresponding to the one or more components for each of the plurality of input documents based on the learning;
parsing each of the one or more components in each of the plurality of input documents based on the positional coordinate to each of the one or more components; and
generating the layout for each of the plurality of input documents based on the parsing;
generating a canonical representation based on the layout, wherein the canonical representation indicates relationships between the one or more components of each of the plurality of input documents;
determining a plurality of sections within each of the plurality of input documents by segmenting the layout based on the canonical representation, wherein the plurality of sections indicates logical units in the corresponding input document of the plurality of input documents; and
determining a textual difference matrix using the AI model, based on a comparison of the plurality of sections within each of the plurality of input documents, wherein the textual difference matrix indicates an alteration in one or more expressions in each of the plurality of sections.
2 . The method of claim 1 , further comprising:
highlighting the alteration in the one or more expressions; and
displaying the highlighted alteration in each of the plurality of input documents such that the highlighted alteration indicates differences between the plurality of input documents.
3 . The method of claim 1 , wherein determining the plurality of sections within each of the plurality of input documents comprises:
identifying a heading component among the one or more components based on an iteration for each of the one or more components;
determining a section-start value upon identifying the heading component; and
determining the plurality of sections within each of the plurality of input documents based on the section-start value to be one of true or false.
4 . The method of claim 3 , wherein when the section-start value is true indicates that a section is in progress, and when the section-start value is false indicates start of another section.
5 . The method of claim 1 , wherein determining the textual difference matrix comprises:
obtaining a sequence of text from each of the plurality of sections within each of the plurality of input documents;
comparing the sequence of text to identify the alteration between the sequence of text between each of the corresponding plurality of sections, wherein the alteration indicates at least one of an insertion and a deletion in one or more expressions in the sequence of text;
creating a change log indicative of a list of alterations between the sequence of text on each page and on each of the plurality of sections within each of the plurality of input documents;
determining a set of coordinates associated with the plurality of sections within each of the plurality of input documents based on the change log and the comparison of the sequence of text; and
determining the textual difference matrix based on the set of coordinates, such that the textual difference matrix provides location of the alteration in each of the plurality of sections.
6 . The method of claim 5 , further comprising:
generating an identification boundary based on the set of coordinates surrounding the identified altered sequence of text in each of the plurality of sections within each of the plurality of input documents.
7 . A system for comparing documents using an artificial intelligence (AI) model, the system comprising:
a memory; and
at least one processor in communication with the memory, wherein the at least one processor is configured to:
preprocess a plurality of input documents into a structured, standardized format for comparison;
generate a layout indicative of an arrangement of one or more components in each of the plurality of input documents using the AI model, based on parsing the one or more components,
wherein to generate the layout, the at least one processor is configured to:
predict a positional coordinate corresponding to each of the one or more components for each of the plurality of input documents, wherein to predict the positional coordinate, the at least one processor is configured to:
train the AI model with an annotated plurality of training documents based on at least one of a precision value, a recall value, and a Mean Average Precision (mAP) value associated with the AI model;
determine hyperparameters associated with the annotated plurality of training documents based on the training;
learn the positional coordinate corresponding to the one or more components in the plurality of training documents based on the determined hyperparameters; and
predict the positional coordinate corresponding to the one or more components for each of the plurality of input documents based on the learning;
parse each of the one or more components in each of the plurality of input documents based on the positional coordinate to each of the one or more components; and
generate the layout for each of the plurality of input documents based on the parsing;
generate a canonical representation based on the layout, wherein the canonical representation indicates relationships between the one or more components of each of the plurality of input documents;
determine a plurality of sections within each of the plurality of input documents by segmenting the layout based on the canonical representation, wherein the plurality of sections indicate logical units in the corresponding input document of the plurality of input documents; and
determine a textual difference matrix using the AI model, based on a comparison of the plurality of sections within each of the plurality of input documents, wherein the textual difference matrix indicates an alteration in one or more expressions in each of the plurality of sections.
8 . The system of claim 7 , wherein the at least one processor is further configured to:
highlight the alteration in the one or more expressions; and
display the highlighted alteration in each of the plurality of input documents such that the highlighted alteration indicates differences between the plurality of input documents.
9 . The system of claim 7 , wherein to determine the plurality of sections within each of the plurality of input documents, the at least one processor is configured to:
identify a heading component among the one or more components based on an iteration for each of the one or more components;
determine a section-start value upon identifying the heading component; and
determine the plurality of sections within each of the plurality of input documents based on the section-start value to be one of true or false.
10 . The system of claim 9 , wherein when the section-start value is true indicates that a section is in progress, and when the section-start value is false indicates start of another section.
11 . The system of claim 7 , wherein to determine the textual difference matrix, the at least one processor is configured to:
obtain a sequence of text from each of the plurality of sections within each of the plurality of input documents;
compare the sequence of text to identify the alteration between the sequence of text between each of the corresponding plurality of sections, wherein the alteration indicates at least one of an insertion and a deletion in one or more expressions in the sequence of text;
create a change log indicative of a list of alterations between the sequence of text on each page and on each of the plurality of sections within each of the plurality of input documents;
determine a set of coordinates associated with the plurality of sections within each of the plurality of input documents based on the change log and the comparison of the sequence of text; and
determine the textual difference matrix based on the set of coordinates, such that the textual difference matrix provides location of the alteration in each of the plurality of sections.
12 . The system of claim 11 , wherein the at least one processor is further configured to:
generate an identification boundary based on the set of coordinates surrounding the identified altered sequence of text in each of the plurality of sections within each of the plurality of input documents.