IP Library Granted Patent US 12,254,033
Granted Patent B2
US 12,254,033 · App. 17/199,509 · Granted Mar 18, 2025

Search in knowledge graphs

Inventors: Birgit Monika Pfitzmann (Wettswil, CH); Kasper Dinkla (Adliswil, CH); Michele Dolfi (Zurich, CH); Christoph Auer (Zürich, CH); Peter Willem Jan Staar (Zurich, CH); André Carvalho (Tomar, PT)
Assignee: International Business Machines Corporation
G06F16/3344G06F16/9027G06F40/103G06F40/295G06F40/30G06N5/02G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,254,033
App. No.
17/199,509
Granted
Mar 18, 2025
Kind
B2
Abstract

The present disclosure relates to a method for searching a graph representing content of digital objects. A set of operations for traversing the graph may be determined according to a search request. The set of operations may be executed, resulting in intermediate result vectors of nodes and a result vector of nodes, wherein the result vector of nodes is associated with a result set of one or more object units of the digital objects. Intermediate result vectors may be selected from of the intermediate result vectors. A set of result entities may be identified. The set of result entities are entities which are part of the object units and part of entities represented by nodes of said selected intermediate result vectors. The set of result entities and the result set of object units may be provided as a result of the search request.

Claims (63)

1. A computer-implemented method comprising:

receiving a search request for searching a graph representing content of digital objects, the graph comprising nodes representing entities and edges representing relationships between the entities, the entities being descriptive of the content of the digital objects;

determining a set of operations for traversing the graph according to the search request, wherein each operation of the set of operations receives an input and provides, as output, a vector of nodes;

executing a set of intermediate operations resulting in intermediate vectors of nodes that represent intermediate object units that are portions of the digital objects, and identifying intermediate entities in the intermediate object units;

after executing the intermediate operations, executing a result operation resulting in a result vector of the nodes that represent result object units, and identifying result entities in the result object units;

for at least one of the intermediate operations, comparing the intermediate entities of the intermediate object units with the result entities of the result object units to find an intersection of common entities that are present in both the intermediate object units and the result object units;

highlighting the common entities in the result object units; and

displaying the result object units with the highlighted common entities,

wherein the set of operations comprises a node retrieval operation and at least one traversal operation, wherein the node retrieval operation receives the search request as the input and provides an initial vector of nodes that fulfils the search request,

wherein the traversal operation provides a result vector of nodes that fulfil a traversal condition and are reachable after one step traversal starting from an input vector, and

wherein the input vector is the initial vector of nodes in case the node retrieval operation is an operation immediately preceding the traversal operation, otherwise the input vector is an intermediate result vector of another immediately preceding operation.

2. The method of claim 1 , the digital objects comprising at least one of: electronic documents, video files and images.

3. The method of claim 1 , wherein the graph is stored in a first storage device and the digital objects are stored in a second storage device, wherein the first storage device has higher data access performance than the second storage device.

4. The method of claim 1 , further comprising selecting the intermediate result vectors based on a selection criterion requiring any one of:

a selection based on a type of operations that precede a result operation that provides the result vector;

a random selection of a subset of intermediate result vectors;

a selection of intermediate result vectors of operations succeeding an initial operation of the set of operations;

a selection of an intermediate result vector of an operation immediately preceding the operation that resulted in the result vector;

a user selection of intermediate result vectors.

5. The method of claim 1 , wherein the selected at least one of the intermediate result vectors are all intermediate result vectors.

6. The method of claim 1 , wherein the set of operations comprises at least two distinct subsets of operations comprising a first subset of operations and a second subset of operations, wherein the execution of the first and second subsets of operations result respectively in a first intermediate result vector and a second intermediate result vector, the first and second intermediate result vectors being associated respectively with a first set of object units and second set of object units of the digital objects; wherein the result vector is a result of a combination of the first and second intermediate result vectors; the method further comprising:

selecting intermediate result vectors of the first subset;

identifying a first set of intermediate result entities as entities which are part of the retrieved object units and part of entities represented by nodes of said selected intermediate result vectors of the first subset;

selecting intermediate result vectors of the second subset;

identifying a second set of intermediate result entities as entities which are part of the retrieved object units and part of entities represented by nodes of said selected intermediate result vectors of the second subset;

wherein the set of result entities is a combination of two sets of intermediate result entities.

7. The method of claim 6 , being performed in response to determining that the first set of objects and the second set of objects units comprise the result set of object units.

8. The method of claim 1 , wherein the result vector is a result of an operation on one intermediate result vector associated with respective set of object units of the digital objects, the method further comprising:

selecting intermediate result vectors that precedes the one intermediate result vector;

identifying a set of intermediate result entities as entities which are part of the retrieved object units and part of entities represented by the nodes of said selected intermediate result vectors;

wherein the set of result entities is the set of intermediate result entities.

9. The method of claim 1 , wherein the result vector comprises: nodes representing the result set of object units.

10. The method of claim 1 , further comprising:

associating each node of the graph with an identifier for identifying the entity represented by said each node;

wherein the identifying of the set of result entities comprises: comparing the identifiers of entities of the object units with the identifiers of nodes of the selected intermediate result vectors; wherein the set of result entities are the matching entities.

11. The method of claim 1 , further comprising upon receiving the search request, selecting one of predefined search workflows, wherein the determined set of operations are operations of the selected search workflow.

12. The method of claim 1 , the object unit being any one of: a paragraph, section or chapter, a sequence of video frame, image portion.

13. The method of claim 1 , wherein the entity is a word or combination of words of a digital object or an object that can be represented in an image or video.

14. A computer program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by one or more processors to cause the one or more processors to perform actions comprising:

receiving a search request for searching a graph representing content of digital objects, the graph comprising nodes representing entities and edges representing relationships between the entities, the entities being descriptive of the content of the digital object;

determining a set of operations for traversing the graph according to the search request, wherein each operation of the set of operations receives an input and provides, as output, a vector of nodes;

executing a set of intermediate operations resulting in intermediate vectors of nodes that represent intermediate object units that are portions of the digital objects, and identifying intermediate entities in the intermediate object units;

after executing the intermediate operations, executing a result operation resulting in a result vector of the nodes that represent result object units, and identifying result entities in the result object units;

for at least one of the intermediate operations, comparing the intermediate entities of the intermediate object units with the result entities of the result object units to find an intersection of common entities that are present in both the intermediate object units and the result object units;

highlighting the common entities in the result object units; and

displaying the result object units with the highlighted common entities,

wherein the set of operations comprises a node retrieval operation and at least one traversal operation, wherein the node retrieval operation receives the search request as the input and provides an initial vector of nodes that fulfils the search request,

wherein the traversal operation provides a result vector of nodes that fulfil a traversal condition and are reachable after one step traversal starting from an input vector, and

wherein the input vector is the initial vector of nodes in case the node retrieval operation is an operation immediately preceding the traversal operation, otherwise the input vector is an intermediate result vector of another immediately preceding operation.

15. The computer program product of claim 14 , wherein the computer-readable program code is provided as a service in a cloud environment.

16. A system comprising:

one or more computer readable storage media with program instructions collectively stored on the one or more computer readable storage media; and

one or more processors configured to execute the program instructions to perform a method comprising:

receiving a search request for searching a graph representing content of digital objects, the graph comprising nodes representing entities and edges representing relationships between the entities, the entities being descriptive of the content of the digital objects;

determining a set of operations for traversing the graph according to the search request, wherein each operation of the set of operations receives an input and provides, as output, a vector of nodes;

executing a set of intermediate operations resulting in intermediate vectors of nodes that represent intermediate object units that are portions of the digital objects, and identifying intermediate entities in the intermediate object units;

after executing the intermediate operations, executing a result operation resulting in a result vector of the nodes that represent result object units, and identifying result entities in the result object units;

for at least one of the intermediate operations, comparing the intermediate entities of the intermediate object units with the result entities of the result object units to find an intersection of common entities that are present in both the intermediate object units and the result object units;

highlighting the common entities in the result object units; and

displaying the result object units with the highlighted common entities,

wherein the set of operations comprises a node retrieval operation and at least one traversal operation, wherein the node retrieval operation receives the search request as the input and provides an initial vector of nodes that fulfils the search request,

wherein the traversal operation provides a result vector of nodes that fulfil a traversal condition and are reachable after one step traversal starting from an input vector, and

wherein the input vector is the initial vector of nodes in case the node retrieval operation is an operation immediately preceding the traversal operation, otherwise the input vector is an intermediate result vector of another immediately preceding operation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2021
From: PFITZMANN, BIRGIT MONIKA; DINKLA, KASPER; DOLFI, MICHELE; AUER, CHRISTOPH; STAAR, PETER WILLEM JAN; CARVALHO, ANDRÉ
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 055594/0772 →
Continuity (1)
Related Publication 20220292121A1 · Sep 15, 2022
References Cited (19)
US 9576007B1 · Sivathanu · 2017 [cited by examiner]
US 10296524B1 · Tung et al. · 2019 [cited by applicant]
US 20210271711A1 · Schlegel · 2021 [cited by examiner]
US 20210326531A1 · Kumar · 2021 [cited by examiner]
US 20220067273A1 · Shin · 2022 [cited by examiner]
US 20230028983A1 · Oechsle · 2023 [cited by examiner]
CN 111190966A · 2020 [cited by applicant]
WO WO2021195149A1 · 2021 [cited by examiner]
Matteo Lissandrini, Davide Mottin, Themis Palpanas, Yannis Velegrakis; Multi-Example Search in Rich Information Graphs; Apr. 19, 2018 URL: https://ieeexplore.ieee.org/document/8509299 (Year: 2018). [cited by examiner]
Iztok Savnik, Mikita Akulich, Matjaz ̌ Krncl, Riste S ̌ krekovski; Data structure set-trie for storing and querying sets: Theoretical and empirical analysis; Feb. 10, 2021 URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PM… [cited by examiner]
De Donato et al., “QueDI: From Knowledge Graph Querying to Data Visualization”, SEMANTiCS 2020, LNCS 12378, pp. 70-86, 2020. [cited by applicant]
Yahya et al., “Exploratory Querying of Extended Knowledge Graphs”, Proceedings of the VLDB Endowment, vol. 9, No. 13, Copyright 2016 VLDB Endowment 21508097/16/09, Published Date: Sep. 2016, 4 pages. [cited by applicant]
Song et al., “Building and Querying an Enterprise Knowledge Graph”, IEEE Transactions on Services Computing, Published Date: Jun. 13, 2017, 14 pages. [cited by applicant]
Chai et al., “Querying Enterprise Knowledge Graph With Natural Language”, Copyright 2019 for this paper by its authors, Use permitted under Creative Commons License Attribution 4.0 International, printed Feb. 26, 2021, … [cited by applicant]
Gómez-Romero et al, “Visualizing large knowledge graphs: A performance analysis”, Future Generation Computer Systems, Available online Jun. 30, 2018, Science Direct, 15 pages. https://www.sciencedirect.com/science/artic… [cited by applicant]
“Graph Visualization Tools”, neo4j developer, printed Feb. 26, 2021, 13 pages. https://neo4j.com/developer/tools-graph-visualization/. [cited by applicant]
Xu et al., “LogCanvas: Visualizing Search History Using Knowledge Graphs”, SIGIR '18, Jul. 8-12, 2018, Ann Arbor, MI, USA, arXiv:1808.05127v1, 4 pages. https://arxiv.org/pdf/1808.05127.pdf. [cited by applicant]
Starr, “How To Use Entity Markup To Appear In The Knowledge Graph”, SEO, Mar. 21, 2014, 16 pages. https://searchengineland.com/5-ways-optimize-markup-knowledge-graph-semantic-search-186755. [cited by applicant]
Mell et al., “The NIST Definition of Cloud Computing”, Recommendations of the National Institute of Standards and Technology, Special Publication 800-145, Sep. 2011, 7 pages. [cited by applicant]
Cited By (1)
US 12,524,421