IP Library Granted Patent US 11,501,753
Granted Patent B2
US 11,501,753 · App. 16/728,672 · Granted Nov 15, 2022

System and method for automating natural language understanding (NLU) in skill development

Inventors: Yilin Shen (Santa Clara, CA); Avik Ray (Sunnyvale, CA); Hongxia Jin (San Jose, CA)
Assignee: Samsung Electronics Co., Ltd.
G10L15/063G06N20/00G10L15/04G10L15/1822G10L15/22G10L2015/0638
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,501,753
App. No.
16/728,672
Granted
Nov 15, 2022
Kind
B2
Abstract

A method includes receiving, from an electronic device, information defining a user utterance associated with a skill to be performed, where the skill is not recognized by a natural language understanding (NLU) engine. The method also includes receiving, from the electronic device, information defining one or more actions for performing the skill. The method further includes identifying, using at least one processor, one or more known skills having one or more slots that map to at least one word or phrase in the user utterance. The method also includes creating, using the at least one processor, a plurality of additional utterances based on the one or more mapped slots. In addition, the method includes training, using the at least one processor, the NLU engine using the plurality of additional utterances.

Claims (64)

1. A method comprising:

receiving, from an electronic device, information defining a user utterance associated with a user intent and a skill, the skill associated with one or more actions to be performed by the electronic device to satisfy the user intent, wherein the skill is not recognized by a natural language understanding (NLU) engine;

receiving, from the electronic device, information defining the one or more actions for performing the skill in order to satisfy the user intent;

identifying, using at least one processor, multiple known skills each having one or more slots that map to at least one word or phrase in the user utterance;

retrieving, using the at least one processor, training utterances associated with the multiple known skills, wherein at least two of the retrieved training utterances are associated with different known skills;

segmenting, using the at least one processor, the retrieved training utterances into segments;

creating, using the at least one processor, a plurality of additional utterances by selecting different combinations of at least some of the segments of the retrieved training utterances in place of different words or phrases in the user utterance, the plurality of additional utterances associated with the user intent and the skill; and

training, using the at least one processor, the NLU engine using the plurality of additional utterances so that the NLU engine is able to recognize the skill associated with the user intent.

2. The method of claim 1 , wherein identifying the multiple known skills comprises:

parsing the user utterance associated with the skill;

identifying at least one slot in the parsed user utterance; and

identifying the one or more slots of the multiple known skills as being associated with the at least one identified slot in the parsed user utterance.

3. The method of claim 1 , wherein:

each known skill is associated with annotated training utterances;

each training utterance is associated with both intent and slot annotations; and

the plurality of additional utterances comprises at least some of the slot annotations.

4. The method of claim 1 , wherein the information defining the one or more actions for performing the skill comprises one or more instructions for performing the one or more actions that are received from a user.

5. The method of claim 1 , wherein the information defining the one or more actions for performing the skill comprises a demonstration of one or more user interactions with at least one application for performing the skill.

6. An apparatus comprising:

at least one memory; and

at least one processor operatively coupled to the at least one memory and configured to:

receive, from an electronic device, information defining a user utterance associated with a user intent and a skill, the skill associated with one or more actions to be performed by the electronic device to satisfy the user intent, wherein the skill is not recognized by a natural language understanding (NLU) engine;

receive, from the electronic device, information defining the one or more actions for performing the skill in order to satisfy the user intent;

identify multiple known skills each having one or more slots that map to at least one word or phrase in the user utterance;

retrieve training utterances associated with the multiple known skills, wherein at least two of the retrieved training utterances are associated with different known skills;

segment the retrieved training utterances into segments;

create a plurality of additional utterances by selecting different combinations of at least some of the segments of the retrieved training utterances in place of different words or phrases in the user utterance, the plurality of additional utterances associated with the user intent and the skill; and

train the NLU engine using the plurality of additional utterances so that the NLU engine is able to recognize the skill associated with the user intent.

7. The apparatus of claim 6 , wherein, to identify the multiple known skills, the at least one processor is configured to:

parse the user utterance associated with the skill;

identify at least one slot in the parsed user utterance; and

identify the one or more slots of the multiple known skills as being associated with the at least one identified slot in the parsed user utterance.

8. The apparatus of claim 6 , wherein:

each known skill is associated with annotated training utterances;

each training utterance is associated with both intent and slot annotations; and

the plurality of additional utterances comprises at least some of the slot annotations.

9. The apparatus of claim 6 , wherein the information defining the one or more actions for performing the skill comprises one or more instructions for performing the one or more actions that are received from a user.

10. The apparatus of claim 6 , wherein the information defining the one or more actions for performing the skill comprises a demonstration of one or more user interactions with at least one application for performing the skill.

11. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of a host device to:

receive, from an electronic device, information defining a user utterance associated with a user intent and a skill, the skill associated with one or more actions to be performed by the electronic device to satisfy the user intent, wherein the skill is not recognized by a natural language understanding (NLU) engine;

receive, from the electronic device, information defining the one or more actions for performing the skill in order to satisfy the user intent;

identify multiple known skills each having one or more slots that map to at least one word or phrase in the user utterance;

retrieve training utterances associated with the multiple known skills, wherein at least two of the retrieved training utterances are associated with different known skills;

segment the retrieved training utterances into segments;

create a plurality of additional utterances by selecting different combinations of at least some of the segments of the retrieved training utterances in place of different words or phrases in the user utterance, the plurality of additional utterances associated with the user intent and the skill; and

train the NLU engine using the plurality of additional utterances so that the NLU engine is able to recognize the skill associated with the user intent.

12. The non-transitory machine-readable medium of claim 11 , wherein the instructions that when executed cause the at least one processor to identify the multiple known skills comprise:

instructions that when executed cause the at least one processor to:

parse the user utterance associated with the skill;

identify at least one slot in the parsed user utterance; and

identify the one or more slots of the multiple known skills as being associated with the at least one identified slot in the parsed user utterance.

13. The non-transitory machine-readable medium of claim 11 , wherein:

each known skill is associated with annotated training utterances;

each training utterance is associated with both intent and slot annotations; and

the plurality of additional utterances comprises at least some of the slot annotations.

14. The non-transitory machine-readable medium of claim 11 , wherein the information defining the one or more actions for performing the skill comprises at least one of:

one or more instructions for performing the one or more actions that are received from a user; and

a demonstration of one or more user interactions with at least one application for performing the skill.

15. The method of claim 1 , wherein the segments of the at least two retrieved training utterances associated with the different known skills include different combinations of keys.

16. The method of claim 1 , wherein the training utterances associated with the multiple known skills are retrieved from a database.

17. The apparatus of claim 6 , wherein the segments of the at least two retrieved training utterances associated with the different known skills include different combinations of keys.

18. The apparatus of claim 6 , wherein the at least one processor is configured to retrieve the training utterances associated with the multiple known skills from a database.

19. The non-transitory machine-readable medium of claim 11 , wherein the segments of the at least two retrieved training utterances associated with the different known skills include different combinations of keys.

20. The non-transitory machine-readable medium of claim 11 , wherein the instructions when executed cause the at least one processor to retrieve the training utterances associated with the multiple known skills from a database.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: SHEN, YILIN; RAY, AVIK; JIN, HONGXIA
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051377/0831 →
Continuity (2)
Provisional Application 62867019 · Jun 26, 2019
Related Publication 20200410986A1 · Dec 31, 2020
Cited By (1)
US 12,288,032