Granted Patent B2

US 12,361,742 · App. 18/314,618 · Granted Jul 15, 2025

Inventors: Jeroen Mattijs van Rotterdam (Fort Lauderdale, FL); Michael T Mohen (Columbia, MD); Chao Chen (Shanghai, CN); Kun Zhao (Shanghai, CN)

Assignee: OPEN TEXT CORPORATION

G06V30/418G06F16/335G06F16/93G06V30/40G06V30/416G06V2201/10

Abstract

Claims (88)

1. A system for assessing similarity of documents, comprising:

a processor;

a document repository storing archived documents, wherein each of the archived documents comprises:

a plurality of archived document sentences; and

archived document metadata;

a non-transitory computer readable medium, storing instructions for:

tokenizing the sentences of each archived document;

vectorizing the tokens of the sentences of each archived document, thereby obtaining an archived document sentence vector for each of the archived documents;

tokenizing the metadata of each archived document;

vectorizing the tokens of the metadata of each archived document, thereby obtaining an archived document metadata vector for each of the archived documents;

obtaining reference document data associated with a reference document, the reference document data comprising:

a plurality of reference document sentences; and

reference document metadata;

tokenizing the sentences of the reference document; and

vectorizing the tokens of the sentences of the reference document;

tokenizing the reference document metadata for the reference document;

vectorizing the tokens of the reference document metadata;

in response to quantifying the reference document by tokenizing and vectorizing tokens of the reference document metadata for the reference document, performing a similarity analysis between the reference document and each of the one or more archived document by:

determining a degree of similarity between the reference document and the respective archived document based on:

comparing the reference document sentence vector to the respective archived document sentence vector; and

comparing the reference document metadata vector to the respective archived document metadata vector;

and

in response to performing the similarity analysis, identifying a number of the one or more archived documents based on the degree of similarity of the each of the one or more archived documents to the reference document.

2. The system of claim 1 , wherein the reference document data comprises text content or non-text content and each archived document comprises text content or non-text content.

3. The system of claim 2 , wherein the instructions further comprise instructions for obtaining an archived text vector for a portion of text of each of the archived documents, the archived document text vector for the portion of the archived document created by tokenizing the portion of text of the archived document and vectorizing the tokenized portion to obtain the archived document text vector for the portion for that archived document,

tokenizing a portion of the reference document data comprising text content and vectorizing the tokenized portion of the text content of the reference document data to obtain a reference document text vector for the portion of the text content of the reference document data, wherein the similarity analysis between the reference document and the archived document is based on the reference document text vector and the archived document text vector.

4. The system of claim 3 , wherein the portion of the reference document data comprising text content comprises the entire text content of the reference document data, a word, a linguistic unit, or a paragraph.

5. The system of claim 4 , wherein the instructions further comprise instructions for:

generating a reference document path for the portion of the reference document metadata or the portion of the text content of the reference document data;

for each archived document, generating an archived document path for the portion of the archived document metadata or the portion of the text of the archived document, wherein the similarity analysis between the reference document and the archived document is based on the reference document path and the archived document path.

6. The system of claim 1 , wherein identifying the number of the one or more archived documents comprises ranking the one or more archived documents based on the degree of similarity determined for each of the one or more archived documents and identifying the number of top ranked archived documents or identifying the number of the one or more archived documents having a degree of similarity with the reference document over a threshold.

7. A method for assessing similarity of documents, comprising:

accessing a document repository storing archived documents, wherein each of the archived documents comprises:

a plurality of archived document sentences; and

archived document metadata;

tokenizing the sentences of each archived document;

vectorizing the tokens of the sentences of each archived document, thereby obtaining an archived document sentence vector for each of the archived documents;

tokenizing the metadata of each archived document;

vectorizing the tokens of the metadata of each archived document, thereby obtaining an archived document metadata vector for each of the archived documents;

obtaining reference document data associated with a reference document, the reference document data comprising:

a plurality of reference document sentences; and

reference document metadata;

tokenizing the sentences of the reference document; and

vectorizing the tokens of the sentences of the reference document;

tokenizing the reference document metadata for the reference document;

vectorizing the tokens of the reference document metadata;

in response to quantifying the reference document by tokenizing and vectorizing tokens of the reference document metadata for the reference document, performing a similarity analysis between the reference document and each archived document by:

determining a degree of similarity between the reference document and the respective archived document based on;

comparing the reference document sentence vector to the respective archived document sentence vector; and

comparing the reference document metadata vector to the respective archived document metadata vector;

and

in response to performing the similarity analysis, identifying a number of the one or more archived documents based on the degree of similarity of the each of the one or more archived documents to the reference document.

8. The method of claim 7 , wherein the reference document data comprises text content or non-text content and each archived document comprises text content or non-text content.

9. The method of claim 8 , further comprising:

obtaining an archived text vector for a portion of text of each of the archived documents, the archived document text vector for the portion of the archived document created by tokenizing the portion of text of the archived document and vectorizing the tokenized portion to obtain the archived document text vector for the portion for that archived document,

tokenizing a portion of the reference document data comprising text content and vectorizing the tokenized portion of the text content of the reference document data to obtain a reference document text vector for the portion of the text content of the reference document data, wherein the similarity analysis between the reference document and the archived document is based on the reference document text vector and the archived document text vector.

10. The method of claim 9 , wherein the portion of the reference document data comprising text content comprises the entire text content of the reference document data, a word, a linguistic unit, a sentence or a paragraph.

11. The method of claim 10 , further comprising:

generating a reference document path for the portion of the reference document metadata or the portion of the text content of the reference document data;

for each archived document, generating an archived document path for the portion of the archived document metadata or the portion of the text of the archived document, wherein the similarity analysis between the reference document and the archived document is based on the reference document path and the archived document path.

12. The method of claim 7 , wherein identifying the number of the one or more archived documents comprises ranking the one or more archived documents based on the degree of similarity determined for each of the one or more archived documents and identifying the number of top ranked archived documents or identifying the number of the one or more archived documents having a degree of similarity with the reference document over a threshold.

13. A non-transitory computer readable medium, storing instructions for:

accessing a document tokenizing the sentences of each archived document;

vectorizing the tokens of the sentences of each archived document, thereby obtaining an archived document sentence vector for each of the archived documents;

tokenizing the metadata of each archived document;

vectorizing the tokens of the metadata of each archived document, thereby obtaining an archived document metadata vector for each of the archived documents;

obtaining reference document data associated with a reference document, the reference document data comprising:

a plurality of reference document sentences; and

reference document metadata;

tokenizing the sentences of the reference document; and

vectorizing the tokens of the sentences of the reference document;

tokenizing the reference document metadata for the reference document;

vectorizing the tokens of the reference document metadata;

in response to quantifying the reference document by tokenizing and vectorizing tokens of the reference document metadata for the reference document, performing a similarity analysis between the reference document and each archived document by:

determining a degree of similarity between the reference document and the respective archived document based on;

comparing the reference document sentence vector to the respective archived document sentence vector; and

comparing the reference document metadata vector to the respective archived document metadata vector;

and

in response to performing the similarity analysis, identifying a number of the one or more archived documents based on the degree of similarity of the each of the one or more archived documents to the reference document.

14. The non-transitory computer readable medium of claim 13 , wherein the reference document data comprises text content or non-text content and each archived document comprises text content or non-text content.

15. The non-transitory computer readable medium of claim 14 , further storing instructions for:

obtaining an archived text vector for a portion of text of each of the archived documents, the archived document text vector for the portion of the archived document created by tokenizing the portion of text of the archived document and vectorizing the tokenized portion to obtain the archived document text vector for the portion for that archived document,

tokenizing a portion of the reference document data comprising text content and vectorizing the tokenized portion of the text content of the reference document data to obtain a reference document text vector for the portion of the text content of the reference document data, wherein the similarity analysis between the reference document and the archived document is based on the reference document text vector and the archived document text vector.

16. The non-transitory computer readable medium of claim 15 , wherein the portion of the reference document data comprising text content comprises the entire text content of the reference document data, a word, a linguistic unit, or a paragraph.

17. The non-transitory computer readable medium of claim 16 , further storing instructions for:

generating a reference document path for the portion of the reference document metadata or the portion of the text content of the reference document data;

for each archived document, generating an archived document path for the portion of the archived document metadata or the portion of the text of the archived document, wherein the similarity analysis between the reference document and the archived document is based on the reference document path and the archived document path.

18. The non-transitory computer readable medium of claim 13 , wherein identifying the number of the one or more archived documents comprises ranking the one or more archived documents based on the degree of similarity determined for each of the one or more archived documents and identifying the number of top ranked archived documents or identifying the number of the one or more archived documents having a degree of similarity with the reference document over a threshold.

Assignments (2)

Continuity (5)

References Cited (24)