IP Library › Granted Patent US 12,002,463
Granted Patent B2
US 12,002,463 · App. 17/728,614 · Granted Jun 4, 2024

Systems and methods for voice-based initiation of custom device actions

Inventors: Bo Wang (San Jose, CA); Venkat Kotla (Mountain View, CA); Chad Yoshikawa (Mountain View, CA); Chris Ramsdale (Mountain View, CA); Pravir Gupta (Los Altos, CA); Alfonso Gomez-Jordana (Mountain View, CA); Kevin Yeun (Mountain View, CA); Jae Won Seo (Mountain View, CA); Lantian Zheng (San Jose, CA); Sang Soo Sung (Palo Alto, CA)
Assignee: GOOGLE LLC
G10L15/22G06F3/167G10L15/1822G10L15/26G10L15/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,002,463
App. No.
17/728,614
Granted
Jun 4, 2024
Kind
B2
Abstract

Systems and methods for enabling voice-based interactions with electronic devices can include a data processing system maintaining a plurality of device action data sets and a respective identifier for each device action data set. The data processing system can receive, from an electronic device, an audio signal representing a voice query and an identifier. The data processing system can identify, using the identifier, a device action data set. The data processing system can identify a device action from device action data set based on content of the audio signal. The data processing system can then identify, from the device action dataset, a command associated with the device action and send the command to the for execution device for execution.

Claims (69)

1. A data processing system to provide content responsive to voice-based interactions, comprising:

a memory to store device action data including a plurality of device action-command pairs supported by a plurality of electronic devices associated with a device model, each device action-command pair including a respective device action of a plurality of device actions and a respective device executable command of a plurality of device executable commands to trigger performance of the respective device action, the respective device executable command being an executable command specific to the plurality of electronic devices associated with the device model;

a device action customization component to map a device model identifier indicative of the device model of the plurality of electronic devices to each of the plurality of device action-command pairs supported by the plurality of electronic devices associated with the device model;

a communications interface to receive, from an electronic device, an audio signal and the device model identifier, the audio signal obtained by the electronic device responsive to a voice-based query;

a natural language processor component to identify, using content associated with the audio signal and the device model identifier, a device action-command pair of the plurality of device action-command pairs;

the device action customization component to identify a context of the voice-based query based on the device action data or the device action-command pair;

a content selector component to select a digital component based on the context of the voice-based query, the digital component comprising audio or visual content; and

the communications interface to transmit the digital component and a device executable command associated with the device action-command pair to the electronic device, the device executable command to cause performance of the device action associated with the device action-command pair and the digital component for presentation by the electronic device, the digital component causing the electronic device to render the audio or visual content on the electronic device in connection with and based on the performance of the device action.

2. The data processing system of claim 1 , wherein the device action is a first device action and the data processing system comprising:

the device action customization component to:

identify a predefined sequence of device actions associated with the voice-based query using the first device action associated with the device-action command pair;

identify one or more second device actions following the first device action associated with the device-action command pair in the predefined sequence of device actions; and

identify an attribute of the context of the voice-based query based on the one or more second device actions following the device action.

3. The data processing system of claim 1 , wherein the digital component includes an audio digital component.

4. The data processing system of claim 1 , comprising:

the memory to maintain a plurality of responses for serving to the plurality of electronic devices, each response of the plurality of responses mapped to a corresponding device-action pair of the plurality of device-action pairs supported by the plurality of electronic devices;

the device action customization component to identify a response of the plurality of responses corresponding to the device action-command pair; and

the communications interface to transmit the response to the electronic device for presentation.

5. The data processing system of claim 1 , comprising:

the memory to maintain, for each device action-command pair of the plurality of device action-command pairs, one or more corresponding keywords; and

the device action customization component to identify the context of the voice-based query based on the one or more keywords associated with the device action-command pair identified by the natural language processor component.

6. The data processing system of claim 1 , wherein the plurality of device action-command pairs are specific to the device model.

7. The data processing system of claim 6 comprising:

the memory to maintain a description of the plurality of electronic devices associated with device model; and

the device action customization component to identify an attribute of the context of the voice-based query based on the description of the plurality of electronic devices associated with device model.

8. The data processing system of claim 1 , wherein the device-model identifier is associated with an application installed on the plurality of electronic devices and the plurality of device action-command pairs are specific to the application.

9. The data processing system of claim 8 comprising:

the memory to maintain a description of the application installed on the plurality of electronic devices; and

the device action customization component to identify an attribute of the context of the voice-based query based on the description of the application installed on the plurality of electronic devices.

10. The data processing system of claim 8 comprising:

the device action customization component to identify one or more parameter values of the device executable command associated with the device action-command pair; and

the device action customization component to identify an attribute of the context of the voice-based query based on the one or more parameter values of the device executable command associated with the device action-command pair.

11. The data processing system of claim 1 , comprising:

the device action customization component to provide a user interface to allow transmission of the device action data and the device-model identifier to the data processing system.

12. The data processing system of claim 1 , comprising:

the device action customization component to provide a restful application programming interface (API) to allow transmission of the device action data and the device-model identifier to the data processing system.

13. A method of providing content responsive to voice-based interactions, the method comprising:

storing, in a memory, device action data including a plurality of device action-command pairs supported by a plurality of electronic devices associated with a device model, each device action-command pair including a respective device action of a plurality of device actions and a respective device executable command of a plurality of device executable commands to trigger performance of the respective device action, the respective device executable command being an executable command specific to the plurality of electronic devices associated with the device model;

mapping a device model identifier indicative of the device model of the plurality of electronic devices to each of the plurality of device action-command pairs supported by the plurality of electronic devices associated with the device model;

receiving, from an electronic device, an audio signal and the device model identifier, the audio signal obtained by the electronic device responsive to a voice-based query;

identifying, using content associated with the audio signal and the device model identifier, a device action-command pair of the plurality of device action-command pairs;

identifying a context of the voice-based query based on the device action data or the device action-command pair;

selecting a digital component based on the context of the voice-based query, the digital component comprising audio or visual content; and

transmitting the digital component and a device executable command associated with the device action-command pair to the electronic device, the device executable command to cause performance of the device action associated with the device action-command pair and the digital component for presentation by the electronic device, the digital component causing the electronic device to render the audio or visual content on the electronic device in connection with and based on the performance of the device action.

14. The method of claim 13 , comprising:

maintaining, for each device action-command pair of the plurality of device action-command pairs, one or more corresponding keywords; and

identifying the context of the voice-based query based on the one or more keywords associated with the device action-command pair identified by the natural language processor component.

15. The method of claim 13 , wherein the plurality of device action-command pairs are specific to the device model.

16. The method of claim 15 comprising:

maintaining a description of the plurality of electronic devices associated with device model; and

identifying an attribute of the context of the voice-based query based on the description of the plurality of electronic devices associated with device model.

17. The method of claim 13 , wherein the device model identifier is associated with an application installed on the plurality of electronic devices and the plurality of device action-command pairs are specific to application.

18. The method of claim 17 comprising:

maintaining a description of the application installed on the plurality of electronic devices; and

identifying an attribute of the context of the voice-based query based on the description of the application installed on the plurality of electronic devices.

19. The method of claim 17 comprising:

identifying one or more parameter values of the device executable command associated with the device action-command pair; and

identifying an attribute of the context of the voice-based query based on the one or more parameter values of the device executable command associated with the device action-command pair.

20. A method of providing content responsive to voice-based interactions, the method comprising:

storing, in a memory, device action data including a plurality of device action-command pairs supported by a plurality of electronic devices, each device action-command pair including a respective device action of a plurality of device actions and a respective device executable command of a plurality of device executable commands to trigger performance of the respective device action;

mapping an identifier to each of the plurality of device action-command pairs supported by the plurality of electronic devices;

receiving, from an electronic device, an audio signal and the identifier, the audio signal obtained by the electronic device responsive to a voice-based query;

identifying, using content associated with the audio signal and the identifier, a device action-command pair of the plurality of device action-command pairs, wherein the device action associated with the device action-command pair is a first device action;

identifying a predefined sequence of device actions associated with the voice-based query using the first device action associated with the device-action command pair;

identifying one or more second device actions following the first device action associated with the device-action command pair in the predefined sequence of device actions;

identifying a context of the voice-based query based on the device action data or the device action-command pair;

identifying an attribute of the context of the voice-based query based on the one or more second device actions following the device action;

selecting a digital component based on the context of the voice-based query; and

transmitting the digital component and a device executable command associated with the device action-command pair to the electronic device, the device executable command to cause performance of the device action associated with the device action-command pair and the digital component for presentation by the electronic device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: WANG, BO; VENKATA, SUBBAIAH; YOSHIKAWA, CHAD; RAMSDALE, CHRIS; GUPTA, PRAVIR; GOMEZ-JORDANA, ALFONSO; YEUN, KEVIN; SEO, JAE WON; ZHENG, LANTIAN; SUNG, SANG SOO
To: GOOGLE LLC
Reel/Frame 060559/0527 →
Continuity (3)
Continuation 15781787
Provisional Application 62640007 · Mar 7, 2018
Related Publication 20220244910A1 · Aug 4, 2022