Granted Patent B2

US 10,796,100 · App. 16/248,433 · Granted Oct 6, 2020

Inventors: Srinivas Bangalore (Morristown, NJ); John Chen (Millburn, NJ)

Assignee: Interactions LLC

G06F40/30G06F16/3329G06F40/137G10L15/183G10L15/1815

Abstract

Claims (44)

1. A non-transitory computer-readable storage medium comprising instructions executable by a processor for identifying an intent associated with a natural language utterance, the instructions comprising:

instructions for receiving a natural language utterance of a user;

instructions for accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes;

instructions for accessing a plurality of intent understanding models, wherein each of the intent understanding models of the plurality of intent understanding models is associated with one of the leaf nodes of the domain hierarchy of intents or is associated with one of the ancestor nodes of the domain hierarchy of intents, and wherein each of the plurality of intent understanding models is trained, through supervised machine learning based on a training set of natural language utterances that are labeled with user intents, to determine a confidence score representative of a likelihood that a natural language utterance represents an intent corresponding to the associated ancestor node or the associated leaf node;

instructions for producing a plurality of node confidence scores by applying the plurality of intent understanding models of the leaf nodes to the natural language utterance;

instructions for determining that none of the node confidence scores corresponding to the leaf nodes exceeds a given minimum confidence threshold;

instructions for producing confidence scores for one or more ancestor nodes;

instructions for identifying a most applicable ancestor node based on the one or more produced confidence scores of ancestor nodes; and

instructions for identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

2. The non-transitory computer-readable storage medium of claim 1 , the instructions further comprising:

instructions for identifying, based on the identified general intent, a prompt for additional information; and

instructions for providing the prompt to the user over the computer network.

3. The non-transitory computer-readable storage medium of claim 1 , wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

4. The non-transitory computer-readable storage medium of claim 1 , wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

5. The non-transitory computer-readable storage medium of claim 1 , wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

7. The non-transitory computer-readable storage medium of claim 6 , wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

8. The non-transitory computer-readable storage medium of claim 6 , wherein the ancestor node confidence score is determined by determining a maximum confidence score of leaf nodes descended from the ancestor node.

9. The non-transitory computer-readable storage medium of claim 6 , the instructions further comprising:

instructions for applying, to the utterance, a plurality of hierarchy understanding models for a corresponding plurality of domain hierarchies to produce a corresponding plurality of scores for the utterance;

instructions for selecting a subset of the domain hierarchies based on the scores; and

instructions for identifying the most applicable ancestor node based on the selected subset.

10. A computer-implemented method for identifying an intent associated with a natural language utterance comprising:

receiving a natural language utterance of a user over a computer network;

accessing a domain hierarchy of intents comprising leaf nodes and ancestor nodes;

accessing a plurality of intent understanding models, wherein each of the intent understanding models of the plurality of intent understanding models is associated with one of the leaf nodes of the domain hierarchy of intents or is associated with one of the ancestor nodes of the domain hierarchy of intents, and wherein each of the plurality of intent understanding models is trained, through supervised machine learning based on a training set of natural language utterances that are labeled with user intents, to determine a confidence score representative of a likelihood that a natural language utterance represents an intent corresponding to the associated ancestor node or the associated leaf node;

producing a plurality of node confidence scores by applying the plurality of intent understanding models of the nodes to the natural language utterance;

determining that none of the node confidence scores corresponding to the leaf nodes exceeds a given minimum confidence threshold;

producing confidence scores for one or more ancestor nodes;

identifying a most applicable ancestor node based on the one or more produced confidence scores of ancestor nodes; and

identifying, as a general intent of the natural language utterance, an intent corresponding to the identified most applicable ancestor node.

11. The computer-implemented method of claim 10 , further comprising:

identifying, based on the identified general intent, a prompt for additional information; and

providing the prompt to the user over the computer network.

12. The computer-implemented method of claim 10 , wherein any descendant node in the hierarchy of intents corresponds to a more specific user intent than the intents corresponding to any ancestor nodes from which the descendant node is descended.

13. The computer-implemented method of claim 10 , wherein applying the intent understanding models comprises associating the natural language utterance with a verb from a domain hierarchy of actions and with a noun from a domain hierarchy of intents related to items.

14. The computer-implemented method of claim 10 , wherein applying the intent understanding models comprises associating the natural language utterance with two or more intents, each intent selected from its own intent hierarchy.

15. The computer-implemented method of claim 10 , wherein the ancestor node confidence score is determined by calculating a function of the confidence scores of other nodes in the same intent hierarchy.

16. The computer-implemented method of claim 15 , wherein the ancestor node confidence score is determined by calculating a geometric mean of the leaf node confidence scores of leaf nodes descended from the ancestor node.

17. The computer-implemented method of claim 15 , wherein the ancestor node confidence score is determined by determining a maximum confidence score of leaf nodes descended from the ancestor node.

18. The computer-implemented method of claim 15 , further comprising:

applying, to the utterance, a plurality of hierarchy understanding models for a corresponding plurality of domain hierarchies to produce a corresponding plurality of scores for the utterance;

selecting a subset of the domain hierarchies based on the scores; and

identifying the most applicable ancestor node based on the selected subset.

Assignments (12)

Cited By (5)

US 12,556,890 US 12,608,171 US 12,613,730 US 12,619,452 US 12,633,287