IP Library › Granted Patent US 11,989,214
Granted Patent B2
US 11,989,214 · App. 17/513,460 · Granted May 21, 2024

Mapping natural language utterances to nodes in a knowledge graph

Inventors: Cynthia Joann Osmon (Sunnyvale, CA); Roger C. Meike (Redwood City, CA); Sricharan Kallur Palli Kumar (Mountain View, CA); Gregory Kenneth Coulombe (Sherwood Park, CA); Pavlo Malynin (Menlo Park, CA)
Assignee: Intuit Inc.
G06F16/3329G06F40/30G06N5/02G10L15/063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,989,214
App. No.
17/513,460
Filed
Oct 28, 2021
Granted
May 21, 2024
Kind
B2
Examiner
HE, JIALONG
Art Unit
2659
USPC
704/9
Abstract

Certain aspects of the present disclosure provide techniques for mapping natural language to stored information. The method generally includes receiving a long-tail query comprising a natural language utterance from a user of an application associated with a set of topics and providing the natural language utterance to a natural language model configured to identify nodes of a knowledge graph. The method further includes, based on output of the natural language model, identifying a node of a knowledge graph associated with the natural language utterance, wherein the output of the natural language model includes a node identifier for the node of the knowledge graph and providing the node identifier to the knowledge engine. The method further includes receiving a response associated with the node of the knowledge graph from the knowledge engine and transmitting the response to the user in response to the long-tail query.

Claims (77)

1. A method for mapping natural language to stored information, comprising:

receiving, at an automated response system, a query from a user device;

determining whether a query response to the query can be located by accessing a response database of the automated response system;

in response to the query response not being located by accessing the response database:

providing the query to a natural language model trained, using a training data set including strings obtained from all nodes of a knowledge graph and including pairs of text strings as training inputs paired with node identifiers as labels, to output a corresponding node identifier based on any text input;

receiving, from the natural language model, a node identifier in response to the query;

providing the node identifier to a knowledge engine, wherein the knowledge engine is configured to:

locate a given node of the knowledge graph based on a stored association in the knowledge graph between the node identifier and the given node;

access the given node of the knowledge graph based on the stored association; and

retrieve corresponding node data from the given node of the knowledge graph;

receiving, from the knowledge engine, node data from the node based on the node identifier;

determining a response based on the node data;

formatting the response to a text format of the automated response system; and

transmitting the response to the user device in response to the query.

2. The method of claim 1 , wherein the query comprises a natural language utterance and determining the node identifier comprises providing one or more inputs, based on the natural language utterance, to a natural language model trained to identify nodes of the knowledge graph associated with the set of topics when receiving one or more given inputs related to a given natural language utterance.

3. The method of claim 2 , wherein the natural language model is trained using only training data obtained from the knowledge graph.

4. The method of claim 2 , further comprising:

receiving a confidence value corresponding to the node identifier from the natural language model based on the one or more inputs; and

determining whether the natural language utterance is related to the node identifier based on the confidence value and a confidence threshold.

5. The method of claim 4 , further comprising:

identifying one or more additional node identifiers associated with confidence values above the confidence threshold;

obtaining one or more additional responses from the knowledge graph associated with the one or more additional node identifiers associated with the confidence values above the confidence threshold; and

transmitting the one or more additional responses to the user.

6. The method of claim 4 , further comprising:

determining that the natural language model cannot identify the natural language utterance; and

generating a crowdsourcing job to obtain additional training data for the natural language model.

7. The method of claim 1 , wherein the query is received via a chatbot application, wherein the response is transmitted to the user via the chatbot application.

8. The method of claim 7 , further comprising determining that a response database associated with the chatbot application does not store the response for the query.

9. A system for mapping natural language to stored information, the system comprising:

one or more processors; and

a memory comprising instructions that, when executed by the one or more processors, cause the system to:

receive, at an automated response system, a query from a user device;

determine whether a query response to the query can be located by accessing a response database of the automated response system;

in response to the query response not being located by accessing the response database:

provide query to a natural language model trained, using a training data set including all strings obtained from all nodes of a knowledge graph and including pairs of text strings as training inputs paired with node identifiers as labels, to output a corresponding node identifier based on any text input;

receive, from the natural language model, a node identifier in response to the query;

provide the node identifier to a knowledge engine, wherein the knowledge engine is configured to:

locate a given node of the knowledge graph based on a stored association in the knowledge graph between the node identifier and the given node;

access the given node of the knowledge graph based on the stored association; and

retrieve corresponding node data from the given node of the knowledge graph;

receive, from the knowledge engine, node data from the node based on the node identifier;

determine a response based on the node data;

format the response to a text format of the automated response system; and

transmit the response to the user device in response to the query.

10. The system of claim 9 , wherein the query comprises a natural language utterance and determining the node identifier comprises providing one or more inputs, based on the natural language utterance, to a natural language model trained to identify nodes of the knowledge graph associated with the set of topics when receiving one or more given inputs related to a given natural language utterance.

11. The system of claim 10 , wherein the natural language model is trained using only training data obtained from the knowledge graph.

12. The system of claim 10 , wherein the instructions, when executed by the one or more processors, further cause the system to:

receive a confidence value corresponding to the node identifier from the natural language model based on the one or more inputs; and

determine whether the natural language utterance is related to the node identifier based on the confidence value and a confidence threshold.

13. The system of claim 12 , wherein the instructions, when executed by the one or more processors, further cause the system to:

identify one or more additional node identifiers associated with confidence values above the confidence threshold;

obtain one or more additional responses from the knowledge graph associated with the one or more additional node identifiers associated with the confidence values above the confidence threshold; and

transmit the one or more additional responses to the user.

14. The system of claim 10 , wherein the instructions, when executed by the one or more processors, further cause the system to:

determine that the natural language model cannot identify the natural language utterance; and

generate a crowdsourcing job to obtain additional training data for the natural language model.

15. The system of claim 9 , wherein the query is received via a chatbot application, wherein the response is transmitted to the user via the chatbot application.

16. The system of claim 15 , further comprising determining that a response database associated with the chatbot application does not store the response for the query.

17. A method for generating a query response, comprising:

receiving, at an automated response system, a query from a user device of an application;

determining whether a query response to the query can be located by accessing a response database of the automated response system;

in response to the query response not being located by accessing the response database:

providing the query to a natural language model trained, using a training data set including strings obtained from nodes of a knowledge graph and including pairs of text strings as training inputs paired with node identifiers as labels, to output a corresponding node identifier based on any text input;

receiving, from the natural language model in response to the query, a node identifier corresponding to a node of a knowledge graph;

providing the node identifier to a knowledge engine, wherein the knowledge engine is configured to:

locate a given node of the knowledge graph based on a stored association in the knowledge graph between the node identifier and the given node;

access the given node of the knowledge graph based on the stored association; and

retrieve corresponding node data from the given node of the knowledge graph;

receiving, from the knowledge engine, node data from the node based on the node identifier;

generating the response to the query based on the node data;

formatting the response to a text format of the automated response system; and

transmitting the response to the user device in response to the query.

18. The method of claim 17 , wherein the query comprises a natural language utterance and determining the node identifier comprises providing one or more inputs, based on the natural language utterance, to a natural language model trained to identify nodes of the knowledge graph associated with a set of topics when receiving one or more given inputs related to a given natural language utterance.

19. The method of claim 18 , wherein the natural language model is trained using only training data obtained from the knowledge graph.

20. The method of claim 18 , further comprising:

receiving a confidence value corresponding to the node identifier from the natural language model based on the one or more inputs; and

determining whether the natural language utterance is related to the node identifier based on the confidence value and a confidence threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 28, 2021
From: OSMON, CYNTHIA J.; MEIKE, ROGERT C.; KUMAR, SRICHARAN KALLUR PALLI; COULOMBE, GREGORY KENNETH; MALYNIN, PAVLO
To: INTUIT, INC.
Reel/Frame 057952/0209 →
Continuity (2)
Continuation 16588873 · Sep 30, 2019
Related Publication 20220050864A1 · Feb 17, 2022