IP Library › Granted Patent US 10,789,953
Granted Patent B2
US 10,789,953 · App. 16/277,844 · Granted Sep 29, 2020

Voice and connection platform

Inventors: Gregory Renard (Menlo Park, CA); Mathias Herbaux (Lille, FR)
Assignee: XBrain, Inc.
G10L15/22G06F3/167G10L15/18G10L15/24G10L15/30G10L15/1822G10L2015/223G10L2015/226G10L2015/227G10L2015/228
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,789,953
App. No.
16/277,844
Granted
Sep 29, 2020
Kind
B2
Abstract

A system and method for providing a voice assistant including receiving, at a first device, a first audio input from a user requesting a first action; performing automatic speech recognition on the first audio input; obtaining a context of user; performing natural language understanding based on the speech recognition of the first audio input; and taking the first action based on the context of the user and the natural language understanding.

Claims (70)

1. A computer-implemented method comprising:

receiving, at a first user device, a first audio input from a user, the first audio input associated with a dialogue and requesting a first action;

determining, at the first user device, a first context of the user;

initiating the first action on the first user device;

subsequent to initiating the first action, receiving, at a second user device distinct from the first user device, a second audio input from the user continuing the dialogue and requesting a second action related to the first action;

determining a tuple describing user intent associated with the second action based on the second audio input, the tuple missing an element;

determining the missing element of the tuple using the first context to complete the tuple;

initiating the second action on the second user device based on the completed tuple;

determining, using a machine learning model, an action pattern of the user based on the first context of the user and one or more past contexts of the user;

predicting a third action of the user based on the action pattern of the user; and

proactively providing to the user a recommendation based on the predicted third action of the user.

2. The computer-implemented method of claim 1 , wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

3. The computer-implemented method of claim 1 , further comprising:

determining a dictionary based on the first context of the user; and

determining a first tuple describing user intent associated with the first action by performing Natural Language Understanding (NLU) on the first audio input using the dictionary; and

initiating the first action on the first user device based on the first tuple.

4. The computer-implemented method of claim 1 , further comprising:

determining a first tuple describing user intent associated with the first action based on the first context of the user and one or more past contexts of the user; and

initiating the first action on the first user device based on the first tuple.

5. The computer-implemented method of claim 1 , further comprising:

detecting an event; and

responsive to detecting the event, proactively initiating the dialogue, wherein the dialogue is between a voice assistant and the user and is performed using the first user device.

6. The computer-implemented method of claim 5 , wherein the event includes one or more of an internal event associated with the first user device, an external event associated with the user, an input of a wake-up phrase, an input of a user input, and an expiration of a timer.

7. The computer-implemented method of claim 1 , further comprising:

proactively initiating the dialogue based on a second action pattern.

8. The computer-implemented method of claim 1 , further comprising:

subsequent to initiating the first action, receiving a third audio input from the user requesting the third action, wherein the third action is unrelated to the first action;

initiating the third action;

subsequent to initiating the third action, receiving a fourth audio input from the user continuing the dialogue and requesting a fourth action related to the first action;

determining a fourth tuple describing user intent associated with the fourth action based on the fourth audio input, the fourth tuple missing another element;

determining the another element of the fourth tuple using the first context to complete the fourth tuple; and

initiating the fourth action based on the completed fourth tuple.

9. The computer-implemented method of claim 1 , further comprising:

creating an input shortcut based on the second audio input and the completed tuple.

10. A system comprising:

one or more processors; and

a memory storing instructions that when executed by the one or more processors, cause the system to perform steps including:

receive, at a first user device, a first audio input from a user, the first audio input associated with a dialogue and requesting a first action;

determine, at the first user device, a first context of the user;

initiate the first action on the first user device;

subsequent to initiating the first action, receive, at a second user device distinct from the first user device, a second audio input from the user continuing the dialogue and requesting a second action related to the first action;

determine a tuple describing user intent associated with the second action based on the second audio input, the tuple missing an element;

determine the missing element of the tuple using the first context to complete the tuple;

initiate the second action on the second user device based on the completed tuple;

determine, using a machine learning model, an action pattern of the user based on the first context of the user and one or more past contexts of the user;

predict a third action of the user based on the action pattern of the user; and

proactively provide to the user a recommendation based on the predicted third action of the user.

11. The system of claim 10 , wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

12. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

determine a dictionary based on the first context of the user; and

determine a first tuple describing user intent associated with the first action by performing Natural Language Understanding (NLU) on the first audio input using the dictionary; and

initiate the first action on the first user device based on the first tuple.

13. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

determine a first tuple describing user intent associated with the first action based on the first context of the user and one or more past contexts of the user; and

initiate the first action on the first user device based on the first tuple.

14. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

detect an event; and

responsive to detecting the event, proactively initiate the dialogue, wherein the dialogue is between a voice assistant and the user and is performed using the first user device.

15. The system of claim 14 , wherein the event includes one or more of an internal event associated with the first user device, an external event associated with the user, an input of a wake-up phrase, an input of a user input, and an expiration of a timer.

16. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

proactively initiate the dialogue based on a second action pattern.

17. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

subsequent to initiating the first action, receive a third audio input from the user requesting the third action, wherein the third action is unrelated to the first action;

initiate the third action;

subsequent to initiating the third action, receive a fourth audio input from the user continuing the dialogue and requesting a fourth action related to the first action;

determine a fourth tuple describing user intent associated with the fourth action based on the fourth audio input, the fourth tuple missing another element;

determine the another element of the fourth tuple using the first context to complete the fourth tuple; and

initiate the fourth action based on the completed fourth tuple.

18. The system of claim 10 , comprising instructions that, when executed by the one or more processors, cause the system to:

create an input shortcut based on the second audio input and the completed tuple.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2019
From: RENARD, GREGORY; HERBAUX, MATHIAS
To: XBRAIN, INC.
Reel/Frame 048375/0740 →
Continuity (3)
Continuation 14871272 · Sep 30, 2015
Provisional Application 62058508 · Oct 1, 2014
Related Publication 20190180750A1 · Jun 13, 2019