IP Library Granted Patent US 11,582,169
Granted Patent B2
US 11,582,169 · App. 17/094,361 · Granted Feb 14, 2023

Modification of audio-based computer program output

Inventors: Laura Eidem (Mountain View, CA); Alex Jacobson (Mountain View, CA)
Assignee: GOOGLE LLC
H04L51/02G06F3/16G06F3/167G06F40/211G06F40/253G10L13/08H04L67/306G06F21/6218
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,582,169
App. No.
17/094,361
Granted
Feb 14, 2023
Kind
B2
Abstract

Modifying computer program output in a voice or non-text input activated environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify a computer program to invoke. The computer program can identify a dialog data structure. The system can modify the identified dialog data structure to include a content item. The system can provide the modified dialog data structure to a computing device for presentation.

Claims (61)

1. A system to modify computer program output, comprising:

a data processing system having one or more processors and memory to:

receive, from a computing device, a digital file corresponding to a first acoustic signal carrying voice content detected by a microphone of the computing device;

select, responsive to the voice content of the digital file, a computer program comprising a chatbot from a plurality of computer programs comprising chatbots;

identify a dialog data structure generated by the chatbot responsive to the voice content of the digital file;

determine, based on the dialog data structure generated by the chatbot, to select content via a content selection process for provision with the dialog data structure;

select, via the content selection process, a content item for provision with the dialog data structure; and

provide, to the chatbot, the content item selected via the content selection process to cause the computing device to generate a second acoustic signal corresponding to the dialog data structure with the content item, wherein the computing device plays the content item with an acoustic fingerprint corresponding to the chatbot.

2. The system of claim 1 , comprising the data processing system to:

identify a tag in the dialog data structure generated by the chatbot; and

determine, responsive to the tag, to select content via the content selection process.

3. The system of claim 1 , comprising the data processing system to:

identify, based on the dialog data structure, an indication to generate a request for content; and

determine, responsive to the request, to select content via the content selection process.

4. The system of claim 1 , comprising the data processing system to:

generate, based on the dialog data structure, a request for content selected via the content selection process; and

determine, responsive to the request, to select content via the content selection process.

5. The system of claim 1 , comprising the data processing system to:

identify a placeholder field in the dialog data structure;

generate a request for content responsive to identification of the placeholder field; and

determine, responsive to the request, to select content via the content selection process.

6. The system of claim 5 , comprising the data processing system to:

insert the content item into the placeholder field of the dialog data structure.

7. The system of claim 1 , wherein the computing device plays the content item with the acoustic fingerprint matching the acoustic fingerprint the chatbot.

8. The system of claim 1 , wherein the content item comprises a parameterized format configured for a parametrically driven text to speech technique, and the computing device executes the parametrically driven text to speech technique to play the content item in the acoustic fingerprint corresponding to the chatbot.

9. The system of claim 1 , comprising the data processing system to:

select, responsive to a third acoustic signal, a second chatbot from the plurality of computer programs comprising chatbots;

select a second content item for insertion into a placeholder field in a second dialog data structure to be provided via the second chatbot; and

provide the second content item to the computing device to cause the computing device to play the second content item with a second acoustic fingerprint corresponding to the second chatbot, wherein the second acoustic fingerprint is different from the acoustic fingerprint corresponding to the chatbot.

10. The system of claim 1 , wherein each of the plurality of computer programs comprising chatbots is associated with a different acoustic fingerprint.

11. The system of claim 1 , comprising the data processing system to:

use a natural language processing technique to process the dialog data structure and identify a portion of the dialog data structure at which to insert content selected via the content selection process; and

provide an indication to insert content selected via the content selection process at the portion of the dialog data structure.

12. A method of modifying computer program output, comprising:

receiving, by a data processing system having one or more processors and memory, from a computing device, a digital file corresponding to a first acoustic signal carrying voice content detected by a microphone of the computing device;

selecting, by the data processing system responsive to the voice content of the digital file, a computer program comprising a chatbot from a plurality of computer programs comprising chatbots;

identifying, by the data processing system, a dialog data structure generated by the chatbot responsive to the voice content of the digital file;

determining, by the data processing system based on the dialog data structure generated by the chatbot, to select content via a content selection process for provision with the dialog data structure;

selecting, by the data processing system via the content selection process, a content item for provision with the dialog data structure; and

providing, by the data processing system to the chatbot, the content item selected via the content selection process to cause the computing device to generate a second acoustic signal corresponding to the dialog data structure with the content item, wherein the computing device plays the content item with an acoustic fingerprint corresponding to the chatbot.

13. The method of claim 12 , comprising:

identifying, by the data processing system, a tag in the dialog data structure generated by the chatbot; and

determining, by the data processing system responsive to the tag, to select content via the content selection process.

14. The method of claim 12 , comprising:

identifying, by the data processing system based on the dialog data structure, an indication to generate a request for content; and

determining, by the data processing system responsive to the request, to select content via the content selection process.

15. The method of claim 12 , comprising:

generating, by the data processing system based on the dialog data structure, a request for content selected via the content selection process; and

determining, by the data processing system responsive to the request, to select content via the content selection process.

16. The method of claim 12 , comprising:

identifying, by the data processing system, a placeholder field in the dialog data structure;

generating, by the data processing system, a request for content responsive to identification of the placeholder field; and

determining, by the data processing system responsive to the request, to select content via the content selection process.

17. The method of claim 16 , comprising:

inserting, by the data processing system, the content item into the placeholder field of the dialog data structure.

18. The method of claim 12 , wherein the computing device plays the content item with the acoustic fingerprint matching the acoustic fingerprint the chatbot.

19. The method of claim 12 , wherein the content item comprises a parameterized format configured for a parametrically driven text to speech technique, and the computing device executes the parametrically driven text to speech technique to play the content item in the acoustic fingerprint corresponding to the chatbot.

20. The method of claim 12 , comprising:

selecting, by the data processing system responsive to a third acoustic signal, a second chatbot from the plurality of computer programs comprising chatbots;

selecting, by the data processing system, a second content item for insertion into a placeholder field in a second dialog data structure to be provided via the second chatbot; and

providing, by the data processing system, the second content item to the computing device to cause the computing device to play the second content item with a second acoustic fingerprint corresponding to the second chatbot, wherein the second acoustic fingerprint is different from the acoustic fingerprint corresponding to the chatbot.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2020
From: EIDEM, LAURA; JACOBSON, ALEX
To: GOOGLE INC.
Reel/Frame 054326/0814 →
CHANGE OF NAME Recorded Nov 10, 2020
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 054373/0157 →
Continuity (3)
Continuation 16694573 · Nov 25, 2019
Continuation 15618842 · Jun 9, 2017
Related Publication 20210058347A1 · Feb 25, 2021