Granted Patent B2

US 12,646,512 · App. 17/925,472 · Granted Jun 2, 2026

Inventors: Surath Sundram (Santa Clara, CA); Rupika Chatterjee (Santa Clara, CA)

Assignee: GOOGLE LLC

G10L15/22G10L15/08G10L2015/088G10L2015/223

Abstract

Claims (69)

1 . A system to control delivery of supplementary content via digital assistants, comprising:

a data processing system comprising memory and one or more processors;

a digital assistant component of the data processing system to:

receive, via a network, data packets from a computing device comprising voice input detected by a microphone of the computing device;

process the data packets to generate an action data structure that is responsive to the voice input;

generate content selection criteria for input into a content selector component to select supplementary content provided by a third-party content provider, the content selection criteria comprising one or more keywords and a digital assistant content type;

transmit a request to the content selector component to select supplementary content based on the content selection criteria, in a manner that overlaps with the generation of the action data structure that is responsive to the voice input;

receive, responsive to the request and prior to generation of the action data structure, a supplementary content item selected by the content selector component based on the content selection criteria generated by the digital assistant component;

in response to receiving the supplementary content item prior to the generation of the action data structure, determine to delay delivery of the supplementary content item until completion of generation of the action data structure, wherein transmission of the supplementary content item is delayed so that the action data structure is transmitted prior to the supplementary content item, wherein transmission of the supplementary content item is delayed by adding a buffer in the transmission of the supplementary content item;

provide, responsive to generation of the action data structure, in a first graphical user interface slot via a display device coupled to the computing device, the action data structure responsive to the voice input; and

provide, responsive to generation of the action data structure, in a second graphical user interface slot via the display device coupled to the computing device, the supplementary content item selected by the content selector component.

2 . The system of claim 1 , comprising:

the data processing system to generate the second graphical user interface slot in a home screen of the computing device.

3 . The system of claim 1 , comprising:

the data processing system to generate the second graphical user interface slot with attributes configured for a mobile display device.

4 . The system of claim 1 , comprising the data processing system to:

receive an indication that the computing device receives power from a battery and is not connected to a charger; and

generate, responsive to the indication, the second graphical user interface slot with first attributes that reduce energy consumption relative to second attributes that consume a greater amount of energy.

5 . The system of claim 1 , comprising the data processing system to:

generate the content selection criteria comprising the digital assistant content type, wherein the digital assistant content type defines attributes for the supplementary content.

6 . The system of claim 5 , wherein the attributes for the digital assistant content type reduce an amount of energy used to provide the supplementary content via the computing device relative to second attributes of a different type of supplementary content that causes the computing device to use a greater amount of energy.

7 . The system of claim 1 , comprising the data processing system to:

transmit the request to the content selector component to select the supplementary content in a manner that overlaps with the generation of the action data structure that is responsive to the voice input;

receive, from the content selector component, the supplementary content item prior to generation of the action data structure; and

instruct the computing device to provide the supplementary content item in the second graphical user interface slot responsive to provision of the action data structure in the first graphical user interface slot.

8 . The system of claim 1 , wherein the second graphical user interface slot is configured to pin the supplementary content item on a home screen of the computing device.

9 . The system of claim 1 , wherein the supplementary content item comprises a supplementary digital assistant application provided by the third-party content provider.

10 . The system of claim 1 , comprising the content selector component to:

identify a plurality of candidate supplementary content items that correspond to the digital assistant content type; and

select the supplementary content item from the plurality of candidate supplementary content items based on the one or more keywords.

11 . The system of claim 1 , comprising the content selector component to:

identify, based on the one or more keywords, a first plurality of candidate supplementary content items that correspond to the digital assistant content type;

identify, based on the one or more keywords, a second plurality of candidate supplementary content items that have a second content type different from the digital assistant content type; and

increase a weight of the first plurality of candidate supplementary content items to improve a likelihood of selection of one of the first plurality of candidate supplementary content items relative to selection of one of the second plurality of candidate supplementary content items.

12 . The system of claim 1 , comprising the content selector component to:

receive one or more supplementary content items from the third-party content provider;

receive, for the one or more of the supplementary content items, an indication from the third-party content provider that indicates the one or more supplementary content items correspond to the digital assistant content type;

identify content parameters for the digital assistant content type; and

perform, responsive to the indication of the digital assistant content type, a validation process on the one or more supplementary content items to identify one or more valid supplementary content items that satisfy the content parameters established for the digital assistant content type.

13 . The system of claim 12 , comprising:

the content selector component to select the supplementary content item from the one or more valid supplementary content items.

14 . The system of claim 12 , wherein the content parameters comprise at least one of an image size, a brightness level, a file size, a network bandwidth utilization, a video duration, or an audio duration.

15 . A method of delivering supplementary content via digital assistants, comprising:

a data processing system comprising memory and one or more processors;

a digital assistant component of the data processing system to:

receiving, by a data processing system comprising memory and one or more processors, via a network, data packets from a computing device comprising voice input detected by a microphone of the computing device;

processing, by a digital assistant component of the data processing system, the data packets to generate an action data structure that is responsive to the voice input;

generating, by the digital assistant component, content selection criteria for input into a content selector component to select supplementary content provided by a third-party content provider, the content selection criteria comprising one or more keywords and a digital assistant content type;

transmitting, by the digital assistant component, a request to the content selector component to select supplementary content based on the content selection criteria, in a manner that overlaps with the generation of the action data structure that is responsive to the voice input;

receiving, by the digital assistant component and prior to generation of the action data structure, responsive to the request, a supplementary content item selected by the content selector component based on the content selection criteria generated by the digital assistant component;

in response to receiving the supplementary content item prior to the generation of the action data structure, determining to delay delivery of the supplementary content item until completion of generation of the action data structure, wherein transmission of the supplementary content item is delayed so that the action data structure is transmitted prior to the supplementary content item, wherein transmission of the supplementary content item is delayed by adding a buffer in the transmission of the supplementary content item;

providing, by the digital assistant component and responsive to generation of the action data structure, in a first graphical user interface slot via a display device coupled to the computing device, the action data structure responsive to the voice input; and

providing, by the digital assistant component and responsive to generation of the action data structure, in a second graphical user interface slot via the display device coupled to the computing device, the supplementary content item selected by the content selector component.

16 . The method of claim 15 , comprising:

generating, by the data processing system, the second graphical user interface slot in a home screen of the computing device with attributes configured for a mobile display device.

17 . The method of claim 15 , comprising:

generating, by the data processing system, the content selection criteria comprising the digital assistant content type, wherein the digital assistant content type defines attributes for the supplementary content and that reduce an amount of energy used to provide the supplementary content via the computing device relative to second attributes of a different type of supplementary content that causes the computing device to use a greater amount of energy.

18 . The method of claim 15 , comprising:

transmitting, by the data processing system, the request to the content selector component to select the supplementary content in a manner that overlaps with the generation of the action data structure that is responsive to the voice input;

receiving, by the data processing system from the content selector component, the supplementary content item prior to generation of the action data structure; and

instructing, by the data processing system, the computing device to provide the supplementary content item in the second graphical user interface slot responsive to provision of the action data structure in the first graphical user interface slot.

19 . The method of claim 15 , comprising:

receiving, by the content selector component, one or more supplementary content items from the third-party content provider;

receiving, by the content selector component, for the one or more of the supplementary content items, an indication from the third-party content provider that indicates the one or more supplementary content items correspond to the digital assistant content type;

identifying, by the content selector component, content parameters for the digital assistant content type, wherein the content parameters comprise at least one of an image size, a brightness level, a file size, a network bandwidth utilization, a video duration, or an audio duration; and

performing, by the content selector component, responsive to the indication of the digital assistant content type, a validation process on the one or more supplementary content items to identify one or more valid supplementary content items that satisfy the content parameters established for the digital assistant content type.

20 . The system of claim 1 , wherein the buffer is determined based on historical performance of the content selector component and an application programming interface (API).

21 . The system of claim 1 , wherein the buffer is preconfigured.

22 . The system of claim 1 , wherein the buffer is based on average time difference between receiving the supplementary content item and generating the action data structure.

Assignments (1)

Continuity (1)

References Cited (12)