IP Library › Granted Patent US 10,206,014
Granted Patent B2
US 10,206,014 · App. 14/488,213 · Granted Feb 12, 2019

Clarifying audible verbal information in video content

Inventors: Ingrid McAulay Trollope (Richmond, GB); Ant Oztaskent (Sutton, GB); Yaroslav Volovich (Cambridge, GB)
Assignee: GOOGLE LLC
H04N21/8133G06F17/30796G10L15/265H04N21/233H04N21/235H04N21/2393
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,206,014
App. No.
14/488,213
Granted
Feb 12, 2019
Kind
B2
Abstract

A method at a server includes: receiving a user request to clarify audible verbal information associated with a media content item playing in proximity to a client device, where the user request includes an audio sample of the media content item and a user query, and the audio sample corresponds to a portion of the media content item proximate in time to issuance of the user query; in response to the user request: identifying the media content item and a first playback position in the media content corresponding to the audio sample; in accordance with the first playback position and identity of the media content item, obtaining textual information corresponding to the user query for a respective portion of the media content item; and transmitting to the client device at least a portion of the textual information.

Claims (66)

1. A method, comprising:

at a server:

receiving from a first client device a user request to clarify audible verbal information of a media content item playing on a second client device in proximity to the first client device,

wherein the user request includes an audio sample of the media content item and a user query issued to the first client device by a user, the user query regarding a subset, not all, of audio content of the media content item; and

in response to the user request:

identifying the media content item based on the audio sample;

identifying a portion of the media content containing the subset of audio content based on the audio sample and content of the user query;

in accordance with the identity of the media content item, obtaining textual information responsive to the user query for the portion of the media content item; and

transmitting to the first client device at least a portion of the textual information for presentation at the first client device.

2. The method of claim 1 , wherein the textual information comprises song lyrics.

3. The method of claim 1 , wherein the textual information comprises a transcription of speech.

4. The method of claim 1 , wherein the audible verbal information is associated with a plurality of speakers; and the user request comprises a request to clarify the audible verbal information with respect to a respective speaker.

5. The method of claim 1 , wherein the textual information comprises subtitle data associated with the media content item.

6. The method of claim 1 , wherein the textual information comprises information obtained from an online document.

7. The method of claim 1 , further comprising:

receiving from the first client device a second user request to clarify audible verbal information associated with the media content item playing on the second media device in proximity to the first client device, wherein the second user request includes a second audio sample of the media content item and a second user query issued to the first client device by the user, wherein the second audio sample corresponds to a second portion of the audio content of the media content item output during the playback of the media content item by the second client device proximate in time to the issuance of the second user query and recorded by the first client device; and

in response to the second user request:

identifying a second playback position in the media content corresponding to the second audio sample;

in accordance with the second playback position and the identity of the media content item, obtaining second textual information corresponding to the second user query for a second portion of the media content item; and

transmitting to the first client device at least a portion of the second textual information for presentation at the first client device.

8. The method of claim 1 , wherein the textual information comprises a translation of the audible verbal information corresponding to the respective portion of the media content item.

9. The method of claim 1 , further comprising:

obtaining or generating a translation of the textual information; and

transmitting to the client device at least a portion of the translation, wherein the at least a portion of the translation corresponds to the at least a portion of the textual information.

10. The method of claim 1 , wherein the media content item is live television content.

11. The method of claim 1 , wherein the media content item is previously broadcast television content.

12. The method of claim 1 , wherein the media content item is recorded content.

13. The method of claim 1 , wherein the media content item is streaming content.

14. A server system, comprising:

memory;

one or more processors; and

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for:

receiving from a first client device a user request to clarify audible verbal information of a media content item playing on a second client device in proximity to the first client device,

wherein the user request includes an audio sample of the media content item and a user query issued to the first client device by a user, the user query regarding a subset, not all, of audio content of the media content item; and

in response to the user request:

identifying the media content item based on the audio sample;

identifying a portion of the media content containing the subset of audio content based on the audio sample and content of the user query;

in accordance with the identity of the media content item, obtaining textual information responsive to the user query for the portion of the media content item; and

transmitting to the first client device at least a portion of the textual information for presentation at the first client device.

15. The system of claim 14 , wherein the audible verbal information is associated with a plurality of speakers; and the user request comprises a request to clarify the audible verbal information with respect to a respective speaker.

16. The system of claim 14 , further comprising instructions for:

receiving from the first client device a second user request to clarify audible verbal information associated with the media content item playing on the second media device in proximity to the first client device, wherein the second user request includes a second audio sample of the media content item and a second user query issued to the first client device by the user, wherein the second audio sample corresponds to a second portion of the audio content of the media content item output during the playback of the media content item by the second client device proximate in time to the issuance of the second user query and recorded by the first client device; and

in response to the second user request:

identifying a second playback position in the media content corresponding to the second audio sample;

in accordance with the second playback position and the identity of the media content item, obtaining second textual information corresponding to the second user query for a second portion of the media content item; and

transmitting to the first client device at least a portion of the second textual information for presentation at the first client device.

17. The system of claim 14 , further comprising instructions for:

obtaining or generating a translation of the textual information; and

transmitting to the client device at least a portion of the translation, wherein the at least a portion of the translation corresponds to the at least a portion of the textual information.

18. A non-transitory computer readable storage medium storing one or more programs to be executed by a computer system with memory and one or more processors, the one or more programs comprising:

instructions for receiving from a first client device a user request to clarify audible verbal information of a media content item playing on a second client device in proximity to the first client device, wherein the user request includes an audio sample of the media content item and a user query issued to the first client device by a user, the user query regarding a subset, not all, of audio content of the media content item; and

in response to the user request:

instructions for identifying the media content item based on the audio sample;

instructions for identifying a portion of the media content containing the subset of audio content based on the audio sample and content of the user query;

instructions for, in accordance with the identity of the media content item, obtaining textual information responsive to the user query for the portion of the media content item; and

instructions for transmitting to the first client device at least a portion of the textual information for presentation at the first client device.

19. The computer readable storage medium of claim 18 , wherein the audible verbal information is associated with a plurality of speakers; and the user request comprises a request to clarify the audible verbal information with respect to a respective speaker.

20. The computer readable storage medium of claim 18 , further comprising:

instructions for receiving from the first client device a second user request to clarify audible verbal information associated with the media content item playing on the second media device in proximity to the first client device, wherein the second user request includes a second audio sample of the media content item and a second user query issued to the first client device by the user, wherein the second audio sample corresponds to a second portion of the audio content of the media content item output during the playback of the media content item by the second client device proximate in time to the issuance of the second user query and recorded by the first client device; and

in response to the second user request:

instructions for identifying a second playback position in the media content corresponding to the second audio sample;

instructions for, in accordance with the second playback position and the identity of the media content item, obtaining second textual information corresponding to the second user query for a second portion of the media content item; and

instructions for transmitting to the first client device at least a portion of the second textual information for presentation at the first client device.

21. The computer readable storage medium of claim 18 , further comprising:

instructions for obtaining or generating a translation of the textual information; and

instructions for transmitting to the client device at least a portion of the translation, wherein the at least a portion of the translation corresponds to the at least a portion of the textual information.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2015
From: TROLLOPE, INGRID MCAULAY; OZTASKENT, ANT; VOLOVICH, YAROSLAV
To: GOOGLE INC.
Reel/Frame 035697/0495 →
Continuity (4)
Continuation In Part 14311204 · Jun 20, 2014
Continuation In Part 14311211 · Jun 20, 2014
Continuation In Part 14311218 · Jun 20, 2014
Related Publication 20150373428A1 · Dec 24, 2015
Cited By (1)
US 12,726,674