IP Library Granted Patent US 10,657,173
Granted Patent B2
US 10,657,173 · App. 15/618,873 · Granted May 19, 2020

Validate modification of audio-based computer program output

Inventors: Laura Eidem (Mountain View, CA); Alex Jacobson (Mountain View, CA)
Assignee: Google LLC
G06F16/683G06F3/167G06F12/1018H04L9/3236H04L43/10H04L51/02H04L63/0457H04L63/0823H04L67/141H04L67/20G06F16/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,657,173
App. No.
15/618,873
Granted
May 19, 2020
Kind
B2
Abstract

Modifying computer program output in a voice or non-text input activated environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify a computer program to invoke. The computer program can identify a dialog data structure. The system can modify the identified dialog data structure to include a content item. The system can provide the modified dialog data structure to a computing device for presentation. The system can validate the dialog data structure output by the computing device for presentation.

Claims (69)

1. A system to validate modification of computer program output, comprising:

a data processing system having one or more processors and memory to:

establish a communication channel with a third-party server that provides a computer program comprising a chatbot, the computer program comprising the chatbot selected based on an acoustic signal detected by a microphone of a computing device;

receive, from the third-party server, a request for content in a parameterized format configured for a parametrically driven text to speech technique, the request triggered by identification of a placeholder field in a dialog data structure identified by the chatbot;

select, via a content selection process responsive to the request, a content item for insertion into the placeholder field of the dialog data structure, the content item in the parameterized format configured for the parametrically driven text to speech technique;

transmit, to the third-party server different from the data processing system, the content item in the parameterized format selected via the content selection process for provision to the chatbot to cause the computing device to perform the parametrically driven text to speech technique to generate a second acoustic signal corresponding to the dialog data structure modified with the content item;

receive, from the chatbot, an indication of the content item;

determine, based on a comparison of the indication of the content item with the content item, that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server; and

set, based on the determination that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server, a validation parameter for the third-party server.

2. The system of claim 1 , comprising the data processing system to:

ping the chatbot subsequent to transmission of the content item; and

receive, from the chatbot, the indication of the content item responsive to the ping.

3. The system of claim 1 , comprising the data processing system to:

ping, subsequent to transmission of the content item, the chatbot based on a timing function; and

receive, from the chatbot, the indication of the content item responsive to the ping.

4. The system of claim 1 , comprising the data processing system to:

receive an indication of a second content item from the chatbot;

determine, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and

set the validation parameter based on the determination.

5. The system of claim 1 , comprising the data processing system to:

receive an indication of a second content item from the chatbot;

determine, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and

disable, based on the determination, provision of content items to chatbots provided by the third-party server.

6. The system of claim 1 , comprising the data processing system to:

determine, based on the comparison, that the indication of the content item received from the chatbot matches the content item transmitted by the data processing system to the third-party server; and

record, responsive to the determination, network activity information associated with an interaction of the content item via the computing device.

7. The system of claim 1 , comprising the data processing system to:

encrypt the content item with a hash function and salts to generate an encrypted content item; and

transmit the encrypted content item to the third-party server.

8. The system of claim 1 , comprising:

encrypt the content item with a hash function and salts to generate an encrypted content item; and

transmit the encrypted content item to the third-party server.

9. The system of claim 1 , comprising the data processing system to:

determine, based on an identifier of the computing device and via a lookup in a data repository, that the computing device is authorized to access the computer program comprising the chatbot; and

select the content item responsive to the determination that the computing device is authorized to access the chatbot.

10. A method of validating a modification of computer program output, comprising:

establishing, by a data processing system, a communication channel with a third-party server that provides a computer program comprising a chatbot, the computer program comprising the chatbot selected based on an acoustic signal detected by a microphone of a computing device;

receiving, by the data processing system from the third-party server, a request for content in a parameterized format configured for a parametrically driven text to speech technique, the request triggered by identification of a placeholder field in a dialog data structure identified by the chatbot;

selecting, by the data processing system via a content selection process responsive to the request, a content item for insertion into the placeholder field of the dialog data structure, the content item in the parameterized format configured for the parametrically driven text to speech technique;

transmitting, by the data processing system to the third-party server, the content item in the parameterized format selected via the content selection process for provision to the chatbot to cause the computing device to perform the parametrically driven text to speech technique to generate a second acoustic signal corresponding to the dialog data structure modified with the content item;

receiving, by the data processing system from the chatbot, an indication of the content item;

determining, by the data processing system based on a comparison of the indication of the content item with the content item, that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server; and

setting, by the data processing system, based on the determination that the content item received by the chatbot from the third-party server matches the content item transmitted by the data processing system to the third-party server, a validation parameter for the third-party server.

11. The method of claim 10 , comprising:

pinging the chatbot subsequent to transmission of the content item; and

receiving, from the chatbot, the indication of the content item responsive to the pinging.

12. The method of claim 10 , comprising:

pinging, subsequent to transmission of the content item, the chatbot based on a timing function; and

receiving, from the chatbot, the indication of the content item responsive to the pinging.

13. The method of claim 10 , comprising:

receiving, by the data processing system, an indication of a second content item from the chatbot;

determining, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and

setting the validation parameter based on the determination.

14. The method of claim 10 , comprising:

receiving, by the data processing system, an indication of a second content item from the chatbot;

determining, based on a comparison, that the indication of the second content item received from the chatbot does not match the content item transmitted by the data processing system to the third-party server; and

disabling, based on the determination, provision of content items to chatbots provided by the third-party server.

15. The method of claim 10 , comprising:

determining, based on the comparison, that the indication of the content item received from the chatbot matches the content item transmitted by the data processing system to the third-party server; and

recording, responsive to the determination, network activity information associated with an interaction of the content item via the computing device.

16. The method of claim 10 , comprising:

encrypting the content item with a hash function and salts to generate an encrypted content item; and

transmitting the encrypted content item to the third-party server.

17. The method of claim 10 , comprising:

encrypting the content item with a hash function and salts to generate an encrypted content item; and

transmitting the encrypted content item to the third-party server.

18. The method of claim 10 , comprising:

determining, based on an identifier of the computing device and via a lookup in a data repository, that the computing device is authorized to access the computer program comprising the chatbot; and

selecting the content item responsive to the determination that the computing device is authorized to access the chatbot.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2017
From: EIDEM, LAURA; JACOBSON, ALEX
To: GOOGLE INC.
Reel/Frame 043249/0618 →
Continuity (1)
Related Publication 20180357310A1 · Dec 13, 2018