IP Library Granted Patent US 10,109,270
Granted Patent B2
US 10,109,270 · App. 15/653,872 · Granted Oct 23, 2018

Adaptive text-to-speech outputs

Inventors: Matthew Sharifi (Kilchberg, CH); Jakob Nicolaus Foerster (San Francisco, CA)
Assignee: Google LLC
G10L13/043G06F17/274G06F17/2775G10L13/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,109,270
App. No.
15/653,872
Granted
Oct 23, 2018
Kind
B2
Abstract

In some implementations, a language proficiency of a user of a client device is determined by one or more computers. The one or more computers then determines a text segment for output by a text-to-speech module based on the determined language proficiency of the user. After determining the text segment for output, the one or more computers generates audio data including a synthesized utterance of the text segment. The audio data including the synthesized utterance of the text segment is then provided to the client device for output.

Claims (58)

1. A method performed by one or more computers, the method comprising:

receiving, by the one or more computers, context data from a client device of a user;

selecting, by the one or more computers, a user context corresponding to the context data from the client device, the user context being selected from among a plurality of user contexts, and the user context indicating a level of complexity of speech that the user is likely able to comprehend at a given time when the context data was received;

selecting, by the one or more computers and from among a plurality of candidate text segments that correspond to different levels of complexity of speech, the text segment for text-to-speech synthesis by a text-to-speech module that best matches the selected user context;

generating, by the one or more computers, audio data comprising a synthesized utterance of the selected text segment using the text-to-speech module; and

providing, by the one or more computers and to the client device, the audio data comprising the synthesized utterance of the selected text segment.

2. The method of claim 1 , further comprising:

receiving, by the one or more computers, data indicating a voice query detected by the client device; and

generating, by the one or more computers, the plurality of candidate text segments, each candidate text segment responsive to the voice query,

wherein receiving the context data comprises receiving context data indicating a current context when the voice query was detected by the client device, and

wherein providing the audio data comprises providing the audio data to the client device for output as a response to the voice query.

3. The method of claim 1 , further comprising generating each of the plurality of candidate text segments from one or more search results identified by a search engine in response to a query from the user.

4. The method of claim 1 , wherein receiving the context data comprises receiving data indicating a location, speed, or movement pattern of the client device; and

wherein selecting the user context comprises selecting the user context based on the location, speed, or movement pattern of the client device indicated by the context data.

5. The method of claim 1 , wherein the context data indicates a GPS data indicating a current location associated with the user; and

wherein selecting the user context comprises selecting the user context based on the GPS data indicating the current location associated with the user.

6. The method of claim 1 , wherein the context data includes sensor data from a mobile device of the user; and

wherein selecting the user context comprises selecting the user context based on the sensor data from the mobile device of the user.

7. The method of claim 1 , further comprising:

generating, by the one or more computers, the plurality of candidate text segments in response to a query from the user;

ranking, by the one or more computers, the plurality of candidate text segments based on a likelihood that each candidate text segment is likely to be responsive to the query; and

re-ranking, by the one or more computers, the plurality of candidate text segments based on the selected user context.

8. The method of claim 7 , wherein the selected text segment that best matches the selected user context corresponds to the candidate text segment having the highest rank after the re-ranking.

9. The method of claim 1 , wherein the client device displays a mobile application that uses a text-to-speech interface.

10. The method of claim 1 , wherein the user context is selected based on one or more queries that were previously submitted by the user, data indicating a current task of the user, or an indication that the user failed to complete a task.

11. A system comprising:

one or more computers; and

a non-transitory computer-readable medium coupled to the one or more computers having instructions stored thereon, which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

receiving context data from a client device of a user;

selecting a user context corresponding to the context data from the client device, the user context being selected from among a plurality of user contexts, and the user context indicating a level of complexity of speech that the user is likely able to comprehend at a given time when the context data was received;

selecting, from among a plurality of candidate text segments that correspond to different levels of complexity of speech, the text segment for text-to-speech synthesis by a text-to-speech module that best matches the selected user context;

generating audio data comprising a synthesized utterance of the selected text segment using the text-to-speech module; and

providing, to the client device, the audio data comprising the synthesized utterance of the selected text segment.

12. The system of claim 11 , wherein the operations further comprise:

receiving data indicating a voice query detected by the client device; and

generating the plurality of candidate text segments, each candidate text segment responsive to the voice query,

wherein receiving the context data comprises receiving context data indicating a current context when the voice query was detected by the client device, and

wherein providing the audio data comprises providing the audio data to the client device for output as a response to the voice query.

13. The system of claim 11 , wherein the operations further comprise generating each of the plurality of candidate text segments from one or more search results identified by a search engine in response to a query from the user.

14. The system of claim 11 , wherein:

receiving the context data comprises receiving data indicating a location, speed, or movement pattern of the client device; and

selecting the user context comprises selecting the user context based on the location, speed, or movement pattern of the client device indicated by the context data.

15. A non-transitory computer-readable storage device encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

receiving context data from a client device of a user;

selecting a user context corresponding to the context data from the client device, the user context being selected from among a plurality of user contexts, and the user context indicating a level of complexity of speech that the user is likely able to comprehend at a given time when the context data was received;

selecting from among a plurality of candidate text segments that correspond to different levels of complexity of speech, the text segment for text-to-speech synthesis by a text-to-speech module that best matches the selected user context;

generating, by the one or more computers, audio data comprising a synthesized utterance of the selected text segment using the text-to-speech module; and

providing, to the client device, the audio data comprising the synthesized utterance of the selected text segment.

16. The non-transitory computer-readable storage device of claim 15 , wherein the operations further comprise:

receiving data indicating a voice query detected by the client device; and

generating the plurality of candidate text segments, each candidate text segment responsive to the voice query,

wherein receiving the context data comprises receiving context data indicating a current context when the voice query was detected by the client device, and

wherein providing the audio data comprises providing the audio data to the client device for output as a response to the voice query.

17. The non-transitory computer-readable storage device of claim 15 , wherein the operations further comprise generating each of the plurality of candidate text segments from one or more search results identified by a search engine in response to a query from the user.

18. The non-transitory computer-readable storage device of claim 15 , wherein:

receiving the context data comprises receiving data indicating a location, speed, or movement pattern of the client device; and

selecting the user context comprises selecting the user context based on the location, speed, or movement pattern of the client device indicated by the context data.

19. The non-transitory computer-readable storage device of claim 15 , wherein the user context is selected based on one or more queries that were previously submitted by the user, data indicating a current task of the user, or an indication that the user failed to complete a task.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 19, 2017
From: SHARIFI, MATTHEW; FOERSTER, JAKOB NICOLAUS
To: GOOGLE INC.
Reel/Frame 043044/0956 →
Continuity (3)
Continuation 15477360 · Apr 3, 2017
Continuation 15009432 · Jan 28, 2016
Related Publication 20170316774A1 · Nov 2, 2017
Cited By (2)
US 12,198,671 US 12,387,725