Granted Patent B2

US 12,265,567 · App. 17/733,351 · Granted Apr 1, 2025

Inventors: Yinghao Ma (Potomac, MD); Chengmin Jiang (Potomac, MD); Sonja Krane (Blue Bell, PA); Jofia Jose Prakash (Aldie, VA); Utpal Tejookaya (Springfield, VA); Jeroen Van Prooijen (Fairfax, VA); Jonathan Hansford (Burke, VA); Jinglei Li (Falls Church, VA); Wallace Scott (Fairfax Station, VA)

Assignee: American Chemical Society

G06F16/3347G06F16/328

Abstract

Claims (83)

1. A method comprising:

converting each structured text document stored in a database into one or more vectors, each structured text document in the database having a title, an abstract, and an author;

using the vectors of the structured text documents stored in a database to create one or more similarity search index;

for each structured text document from the database, searching the search index using the one or more vectors of the structured text document;

for each structured text document from the database, generating a list of at least N other structured text documents from the database similar to the structured text document based on said search, wherein N equals 50; and

storing each list of at least N other structured text documents from the database similar to the structured text document in a table.

2. The method of claim 1 , further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, and an author;

converting the new structured text document into one or more vectors; searching the search index using the one or more vectors of the new structured text document;

generating a second list of at least N structured text documents from the database similar to the new structured text document using the said search index; and

storing the second list of at least N structured text documents from the database similar to the new structured text document in a second table.

3. The method of claim 1 , wherein converting each structured text document stored in the database into one or more vectors comprises converting the structured text document into one or more vectors by means of SPECTER embedding.

4. The method of claim 3 , further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, and an author;

converting the new structured text document into one or more new vectors by means of SPECTER embedding;

searching the search index using the one or more new vectors of the new structured text document;

generating a second list of at least N structured text documents from the database similar to the new structured text document using the said search index; and

storing the second list of at least N structured text documents from the database similar to the new structured text document in a second table.

5. The method of claim 1 , wherein:

each structured text document stored in the database is associated with a full text.

6. The method of claim 5 , further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, an author, and a full text;

converting the new structured text document into one or more new vectors;

searching the search index using the one or more new vectors of the new structured text document;

generating a second list of N structured text documents from the database similar to the new structured text document using the said search index; and

storing the second list of at least N structured text documents from the database similar to the new structured text document in a second table.

7. The method of claim 1 , wherein:

each structured text document stored in the database is associated with a metadata.

8. The method of claim 7 , further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, an author, and a metadata;

converting the new structured text document into one or more new vectors; searching the search index using the one or more new vectors;

generating a second list of at least N structured text documents from the database similar to the new structured text document based on said search; and

storing the second list of at least N structured text documents from the database similar to the new structured text document in a second table.

9. The method of claim 1 , wherein:

converting each structured text document stored in the database into one or more vectors comprises converting each structured text document using a natural language processing algorithm; and

generating the list of at least N other structured text documents from the database similar to the structured text document based on said search comprises determining a similarity based on one or more of an Euclidian distance, a Gaussian distance, or a cosine similarity.

10. The method of claim 1 , wherein the database is a collection of training data.

11. The method of claim 1 , wherein the search index is one of a flat index type, a locally sensitive hash (LSH) type, an inverted file index type (IVF), or a Hierarchical Navigable Small World (HNSW) graph.

12. The method of claim 1 , further comprising querying the database to compile a list of authors of each structured text document on the list of at least N structured text documents.

13. A system for identifying similar structured text documents to another structured text document, comprising:

at least one processor, and

at least one non-transitory computer readable media storing instructions configured to cause the processor to perform operations comprising:

converting each structured text document stored in a database into one or more vectors, each structured text document in the database having a title, an abstract, and an author;

using the vectors of the structured text documents stored in a database to create a similarity search index;

for each structured text document from the database, searching the search index using the one or more vectors of the structured text document

for each structured text document from the database, generating a list of at least N other structured text documents from the database similar to the structured text document based on said search, wherein N is 50; and

storing each list of at least N other structured text documents from the database similar to the structured text document in a table.

14. The system of claim 13 , the operations further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, and an author;

converting the new structured text document into one or more new vectors;

searching the search index using the one or more new vectors of the new structured text document;

generating a second list of at least N structured text documents from the database similar to the new structured text document using the said search index; and

storing the second list of at least N other structured text documents from the database similar to the new structured text document in a second table.

15. The system of claim 13 , the operations further comprising:

converting the structured text document into one or more vectors by means of SPECTER embedding.

16. The system of claim 15 , the operations further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, and an author;

converting the new structured text document into one or more new vectors by means of SPECTER embedding;

searching the search index using the one or more new vectors of the new structured text document;

generating a second list of at least N structured text documents from the database similar to the new structured text document using the said search index; and

storing the second list of at least N other structured text documents from the database similar to the new structured text document in a second table.

17. The system of claim 13 , wherein:

each structured text document stored in the database is associated with a full text.

18. The system of claim 17 , the operations further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, an author, and a full text;

converting the new structured text document into one or more new vectors;

searching the search index using the one or more new vectors of the new structured text document;

generating a second list of N structured text document from the database similar to the new structured text document using the said search index; and

storing the second list of N structured text documents from the database similar to the new structured text document in a second table.

19. The system of claim 13 , wherein:

each structured text document stored in the database is associated with a metadata.

20. The system of claim 19 , the operations further comprising:

receiving a new structured text document, the new structured text document having a title, an abstract, an author, and a metadata;

converting the new structured text document into one or more new vectors;

comparing the one or more new vectors of each structured text document to the vectors of the structured text documents from the database; and

generating a second list of at least N structured text document from the database similar to the new structured text document based on said comparison;

storing the second list of at least N structured text documents from the database similar to the new structured text document in a second table.

21. The system of claim 13 , wherein:

converting each structured text document stored in the database into one or more vectors comprises converting each structured text document using a natural language processing algorithm; and

generating the list of at least N other structured text documents from the database similar to the structured text document based on said search comprises determining a similarity based on one or more of an Euclidian distance, a Gaussian distance, or a cosine similarity.

22. The system of claim 13 , wherein the database is a collection of training data.

23. The system of claim 13 , wherein the search index is one of a flat index type, a locally sensitive hash (LSH) type, an inverted file index type (IVF), or a Hierarchical Navigable Small World (HNSW) graph.

24. The system of claim 13 , further comprising querying the database to compile a list of authors of each structured text document on the list of at least N structured text documents.

Assignments (1)

Continuity (2)

References Cited (17)