IP Library Granted Patent US 10,521,476
Granted Patent B2
US 10,521,476 · App. 15/888,049 · Granted Dec 31, 2019

Dynamically updatable offline grammar model for resource-constrained offline device

Inventors: Sangsoo Sung (Palo Alto, CA); Yuli Gao (Sunnyvale, CA); Prathab Murugesan (Mountain View, CA)
Assignee: GOOGLE LLC
G06F16/90324G06F16/2228G06F16/243G06F16/2425G10L15/19G10L15/30G10L17/22G06F16/3344G06F17/271G06F17/274
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,521,476
App. No.
15/888,049
Granted
Dec 31, 2019
Kind
B2
Abstract

An offline semantic processor of a resource-constrained voice-enabled device such as a mobile device utilizes an offline grammar model with reduced resource requirements to parse voice-based queries received by the device. The offline grammar model may be generated from a larger and more comprehensive grammar model used by an online voice-based query processor, and the generation of the offline grammar model may be based upon query usage data collected from one or more users to enable a subset of more popular voice-based queries from the online grammar model to be incorporated into the offline grammar model. In addition, such a device may collect query usage data and upload such data to an online service to enable an updated offline grammar model to be generated and downloaded back to the device and thereby enable a dynamic update of the offline grammar model to be performed.

Claims (59)

1. A method, comprising:

in a semantic processor of a resource-constrained offline device, processing voice-based queries issued by a user of the resource-constrained offline device using an offline grammar model stored in the resource-constrained offline device, wherein the offline grammar model maps a subset of queries from among a plurality of queries to one or more actions;

collecting query usage data for queries issued by the user with the resource-constrained offline device;

uploading the query usage data from the resource-constrained offline device to an online service for use in updating the offline grammar model;

receiving update data from the online service with the resource-constrained offline device, the update data generated based at least in part on the uploaded query usage data; and

updating the offline grammar model stored in the resource-constrained offline device using the update data such that a voice-based query issued by the user after updating the offline grammar model is processed using the updated offline grammar model.

2. The method of claim 1 , wherein the query usage data includes query usage data associated with voice-based queries or text queries issued by the user with the resource-constrained offline device.

3. The method of claim 1 , wherein the update data is further generated based at least in part on additional query usage data for queries issued by a plurality of users.

4. The method of claim 1 , wherein the update data includes the updated offline grammar model, and wherein updating the offline grammar model includes storing the updated offline grammar model in the resource-constrained offline device.

5. The method of claim 1 , wherein the updated offline grammar model is personalized for the user of the resource-constrained offline device.

6. The method of claim 1 , further comprising:

using, by the resource-constrained offline device, the updated offline grammar model, wherein using the updated offline grammar model comprises:

mapping text, outputted by a voice-to-text module of the resource-constrained offline device, to a corresponding action of the updated offline grammar model, and

causing the corresponding action to be performed based on mapping the text to the corresponding action of the updated offline grammar model.

7. The method of claim 6 , further comprising:

identifying, based on the text, attributes that constrain the corresponding action;

wherein causing the corresponding action to be performed comprises:

causing the corresponding action to be performed with the attributes that constrain the corresponding action.

8. The method of claim 7 , further comprising:

performing, by the resource-constrained offline device, the corresponding action with the attributes.

9. The method of claim 1 , further comprising:

determining lack of connectivity to an online voice-based query processor; and

in response to determining the lack of connectivity:

using, by the resource-constrained offline device, the updated offline grammar model in determining a corresponding action for a query issued with the resource-constrained offline device.

10. The method of claim 1 , further comprising:

using, by the resource-constrained offline device, the updated offline grammar model in determining a corresponding action for a query issued with the resource-constrained offline device.

11. The method of claim 1 , wherein the actions mapped by the updated offline grammar model include two or more of:

a setting an alarm action;

a setting a reminder action;

an initiating a communication action;

a playing a song action; and

a changing a device setting action.

12. A method, comprising:

maintaining an online grammar model used by an online voice-based query processor to parse online voice-based queries, the online grammar model mapping a plurality of queries to actions;

analyzing query usage data for at least a subset of the plurality of queries to identify a subset of queries from among the plurality of queries mapped by the online grammar model, wherein the query usage data includes query usage data collected for queries issued by a plurality of users;

building an offline grammar model that maps the subset of queries to actions among the actions for use by a resource-constrained offline device, wherein the offline grammar model has reduced resource requirements relative to the online grammar model;

communicating the offline grammar model to the resource-constrained offline device for storage by the resource-constrained offline device and for use by an offline semantic processor of the resource-constrained offline device, the offline semantic processor using the offline grammar model to locally map a query to a corresponding action of the offline grammar model, to thereby cause performance of the corresponding action in response to the query.

13. The method of claim 12 , wherein analyzing the query usage data includes, for each of multiple actions among the actions:

determining a corresponding distribution of queries from among a plurality of corresponding queries mapped to the action by the online grammar model using the collected query usage data; and

including, in the identified subset of queries, at least a top corresponding query from among the plurality of corresponding queries mapped to the action.

14. The method of claim 12 , wherein the actions mapped by the offline grammar model include two or more of:

a setting an alarm action;

a setting a reminder action;

an initiating a communication action;

a playing a song action; and

a changing a device setting action.

15. The method of claim 12 , further comprising:

using, by the resource-constrained offline device, the offline grammar model, wherein using the offline grammar model comprises:

mapping a query, provided at the resource-constrained offline device, to a corresponding action of the offline grammar model, and

identifying attributes, for the corresponding action, that constrain the corresponding action.

16. The method of claim 15 , further comprising:

causing, by the resource-constrained offline device, performance of the action with the attributes.

17. The method of claim 12 , wherein using the offline grammar model is in response to determining lack of connectivity to the online voice-based query processor.

18. The method of claim 12 , wherein the query usage data includes query usage data collected for voice-based queries issued by a user of the resource-constrained offline device and processed by the offline semantic processor of the resource-constrained offline device.

19. A resource-constrained offline device including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to:

process voice-based queries issued by a user of the resource-constrained offline device using an offline grammar model stored in the resource-constrained offline device, wherein the offline grammar model maps a subset of queries from among a plurality of queries to one or more actions;

upload query data, related to the voice-based queries issued by the user, to an online service for use in updating the offline grammar model;

receive update data from the online service with the resource-constrained offline device, the update data generated based at least in part on the uploaded query data; and

update the offline grammar model stored in the resource-constrained offline device using the update data such that a voice-based query issued by the user after updating the offline grammar model is processed using the updated offline grammar model.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2018
From: SUNG, SANGSOO; GAO, YULI; MURUGESAN, PRATHAB
To: GOOGLE INC.
Reel/Frame 044849/0958 →
CHANGE OF NAME Recorded Feb 7, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 046368/0400 →
Continuity (2)
Continuation 14723305 · May 27, 2015
Related Publication 20180157673A1 · Jun 7, 2018