Granted Patent B2

US 12,216,700 · App. 17/234,084 · Granted Feb 4, 2025

Inventors: Alexander Taboriskiy (Zurich, CH); Emmanuel Mogenet (Zollikerberg, CH); Oliver Heckmann (Baech, CH); Matsvei Zhdanovich (Zurich, CH); Gokhan Hasan Bakir (Zurich, CH); Behshad Behzadi (Zurich, CH); Karoly Csalogany (Zurich, CH)

Assignee: GOOGLE LLC

G06F16/438G06F16/433G06F16/48H04N21/42203H04N21/4722

Abstract

Claims (76)

1. A method, comprising:

receiving, using a mobile device, a plurality of input terms;

determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device, the query including at least a portion of the plurality of input terms other than the trigger term;

in response to determining that the trigger term has been included in the plurality of input terms, requesting, using the mobile device, media playback information from the media playback device;

receiving, at the mobile device from the media playback device and in response to the request, the media playback information including media content identification information related to the media content and timing information including a first timestamp corresponding to a first time at which the trigger term was received and a second timestamp corresponding to a second time at which a last term in the query was received;

in response to receiving the media playback information the mobile device enabled to cause a search to be performed at a search server that includes the query, the timing information, and the media content identification information; and

receiving and presenting, on a display of the mobile device, at least a portion of a search result responsive to the search, including the query.

2. The method of claim 1 , wherein the at least the portion of the search result is determined to be presented on the display of the mobile device based on search popularity information associated with the search result.

3. The method of claim 1 , further comprising:

retrieving the media playback information prior to receiving the query; and

associating the media playback information with the query in response to receiving the trigger term.

4. The method of claim 1 , further comprising:

receiving a request to associate with the media playback device, wherein the media playback device is presenting the media content and wherein the media playback device is executing a media application that exchanges the media playback information with a mobile application executing on the mobile device;

associating the mobile device with the media playback device; and

activating an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to the plurality of input terms.

5. The method of claim 4 , further comprising:

determining whether the audio input device associated with the mobile device is capable of receiving the plurality of ambient sounds; and

activating an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the mobile device is not capable of receiving the plurality of ambient sounds.

6. The method of claim 4 , further comprising extracting the trigger term and the query from the plurality of input terms comprises processing the plurality of ambient sounds using a speech recognition system.

7. The method of claim 4 , further comprising:

requesting additional media playback information from the media application executing on the media playback device; and

transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result.

8. The method of claim 4 , further comprising:

determining whether the search result is to be presented on the display of the mobile device; and

presenting at least a second portion of the search result on a display associated with the media playback device in response to determining that the at least the second portion of the search result is not to be presented on the display of the mobile device.

9. The method of claim 4 , further comprising:

receiving requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content;

activating audio capturing devices associated with at least a portion of the plurality of mobile devices and the media playback device to receive the plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device;

determining that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the presented media content;

obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries; and

causing the plurality of search results to be presented.

10. The method of claim 9 , further comprising:

associating at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received; and

causing the at least the portion of the search result to be presented on the respective mobile device.

11. A system, comprising:

a hardware processor of a mobile device configured to:

receive a plurality of input terms;

determine whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device, the query including at least a portion of the plurality of input terms other than the trigger term;

in response to determining that the trigger term has been included in the plurality of input terms, requesting, using the mobile device, media playback information from the media playback device;

receive, at the mobile device from the media playback device and in response to the request, the media playback information including media content identification information related to the media content and timing information including a first timestamp corresponding to a first time at which the trigger term was received and a second timestamp corresponding to a second time at which a last term in the query was received;

cause, using the mobile device, a search to be performed at a search server that includes the query, the timing information, and the media content identification information; and

receive and present, on a display of the device, at least a portion of a search result responsive to the search, including the query.

12. The system of claim 11 , wherein the at least the portion of the search result is determined to be presented on the display of the mobile device based on search popularity information associated with the search result.

13. The system of claim 11 , wherein the hardware processor is further configured to:

retrieve the media playback information prior to receiving the query; and

associate the media playback information with the query in response to receiving the trigger term.

14. The system of claim 11 , wherein the hardware processor is further configured to:

receive a request to associate with the media playback device, wherein the media playback device is presenting the media content and wherein the media playback device is executing a media application that exchanges the media playback information with a mobile application executing on the mobile device;

associate the mobile device with the media playback device; and

activate an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to the plurality of input terms.

15. The system of claim 14 , wherein the hardware processor is further configured to:

determine whether the audio input device associated with the mobile device is capable of receiving the plurality of ambient sounds; and

activate an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the mobile device is not capable of receiving the plurality of ambient sounds.

16. The system of claim 14 , wherein the hardware processor is further configured to extract the trigger term and the query from the plurality of input terms comprises processing the plurality of ambient sounds using a speech recognition system.

17. The system of claim 14 , wherein the hardware processor is further configured to:

request additional media playback information from the media application executing on the media playback device; and

transmit the query, the media playback information, and the additional media playback information to a search system to receive the search result.

18. The system of claim 14 , wherein the hardware processor is further configured to:

determine whether the search result is to be presented on the display of the mobile device; and

present at least a second portion of the search result on a display associated with the media playback device in response to determining that the at least the second portion of the search result is not to be presented on the display of the mobile device.

19. The system of claim 11 , wherein the hardware processor is further configured to:

receive requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content;

activate audio capturing devices associated with at least a portion of the plurality of mobile devices and the media playback device to receive a plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device;

determine that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the presented media content;

obtain a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries; and

cause the plurality of search results to be presented.

20. The system of claim 19 , wherein the hardware processor is further configured to:

associate at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received; and

cause at least the portion of the search result to be presented on the respective mobile device.

21. A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform method for providing media guidance, the method comprising:

receiving, using a mobile device, a plurality of input terms;

determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device;

in response to determining that the trigger term has been included in the plurality of input terms, requesting, using the mobile device, media playback information from the media playback device;

receiving, at the mobile device from the media playback device and in response to the request, the media playback information including media content identification information related to the media content and timing information including a first timestamp corresponding to a first time at which the trigger term was received and a second timestamp corresponding to a second time at which a last term in the query was received;

causing, using the mobile device, a search to be performed at a search server that includes the query, the timing information, and the media content identification information, the query including at least a portion of the plurality of input terms other than the trigger term; and

receiving and presenting, on a display associated with the mobile device, at least a portion of a search result responsive to the search, including the query.

Assignments (2)

Continuity (3)

References Cited (93)

Cited By (2)

US 12,556,783 US 12,574,608