IP Library Granted Patent US 12,316,893
Granted Patent B2
US 12,316,893 · App. 18/238,788 · Granted May 27, 2025

Systems and methods that match search queries to television subtitles

Inventors: Zbigniew Skolicki (Cracow, PL); Yaroslav Volovich (Cambridge, GB); Ant Oztaskent (London, GB); Varun Rajgarhia (London, GB)
Assignee: GOOGLE LLC
H04N21/252G06F16/245G06F16/738G06F16/7844H04N21/251H04N21/4524H04N21/4532H04N21/4884H04N21/8133G06F16/7834
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,316,893
App. No.
18/238,788
Granted
May 27, 2025
Kind
B2
Abstract

A process identifies a search query spike from queries submitted by users during a first span of time, which is less than a predefined duration. The spike corresponds to a set of queries identified as equivalent. The frequency of submitting queries from the set during the first time span exceeds the frequency of submitting queries from the set during an average span of time. The process correlates the spike to a broadcast video program by matching terms from the set of search queries to subtitle terms appearing in the video program at a first location. The first location in the video program was broadcast within a predefined time before the first span of time. The process receives notification from a user device indicating user interest in the video program. The process transmits to the user device search results corresponding to some search queries from the set of search queries.

Claims (37)

1. A method for providing video program information, the method comprising:

collecting, over a network, search queries from a plurality of different computing devices;

identifying, using a hardware processor, for a first time period, a search query spike from the search queries;

matching, using a classifier trained on data from previously stored subtitles and search query groups, a portion of search terms from the search queries against a plurality of subtitle terms to identify a media content item associated with the search query spike from among a plurality of media content items being presented during the first time period;

receiving, at a second time period, an indication of user interest in the media content item from a computing device, the indication including that the media content item is currently being presented on a display device; and

in response to receiving the indication of user interest in the media content item from the computing device, causing, using the hardware processor, at least one search result corresponding to a search query from the search queries to be transmitted to and displayed on the computing device at a same time that the media content item is currently being presented on the display device.

2. The method of claim 1 , wherein the search query spike is associated with the media content item being presented during the first time period by matching the portion of search terms from the search queries to the portion of subtitle terms appearing in the media content item at a first location, wherein the first location in the media content item was presented within a predefined time before the first time period.

3. The method of claim 2 , wherein the at least one search result is concurrently presented at the computing device with the first location in the media content item when the media content item is presented during the second time period.

4. The method of claim 1 , wherein a first search query and a second search query from the search queries are identified as being equivalent when an ordered sequence of search terms from the first search query is substantially identical to an ordered sequence of search terms from the second search query.

5. The method of claim 1 , wherein a first search query and a second search query from the search queries are identified as being equivalent when a linguistic concept expressed using search terms from the first search query is substantially a same linguistic concept expressed using search terms from the second search query.

6. The method of claim 1 , wherein receiving the indication of user interest in the media content item from the computing device comprises receiving the indication that the media content item is currently being presented on the display device proximal to the computing device.

7. The method of claim 1 , wherein receiving the indication of user interest in the media content item from the computing device comprises receiving an audio stream from the computing device and correlating the audio stream to the media content item.

8. The method of claim 1 , wherein matching the portion of search terms from the search queries to the portion of subtitle terms associated with the media content item comprises matching an ordered sequence of terms from a search query in the search queries to a substantially identical ordered sequence of subtitle terms.

9. The method of claim 1 , wherein matching the portion of search terms from the search queries to the portion of subtitle terms associated with the media content item comprises matching a linguistic concept expressed using search terms from a search query with a substantially identical linguistic concept expressed using the subtitle terms.

10. The method of claim 1 , wherein the computing device is a second screen device and the media content item is presented on the display device proximal to the computing device.

11. A system for providing video program information, the system comprising:

a hardware processor of a server device, wherein the hardware processor is configured to:

collect, over a network, search queries from a plurality of different computing devices;

identify for a first time period, a search query spike from the search queries;

match, using a classifier trained on data from previously stored subtitles and search query groups, a portion of search terms from the search queries against a plurality of subtitle terms to identify a media content item associated with the search query spike from among a plurality of media content items being presented during the first time period;

receive, at a second time period, an indication of user interest in the media content item from a computing device, the indication including that the media content item is currently being presented on a display device; and

in response to receiving the indication of user interest in the media content item from the computing device, cause at least one search result corresponding to a search query from the search queries to be transmitted to and displayed on the computing device at a same time that the media content item is currently being presented on a display device.

12. The system of claim 11 , wherein the search query spike is associated with the media content item being presented during the first time period by matching the portion of search terms from the search queries to the portion of subtitle terms appearing in the media content item at a first location, wherein the first location in the media content item was presented within a predefined time before the first time period.

13. The system of claim 12 , wherein the at least one search result is concurrently presented at the computing device with the first location in the media content item when the media content item is presented during the second time period.

14. The system of claim 11 , wherein a first search query and a second search query from the search queries are identified as being equivalent when an ordered sequence of search terms from the first search query is substantially identical to an ordered sequence of search terms from the second search query.

15. The system of claim 11 , wherein a first search query and a second search query from the search queries are identified as being equivalent when a linguistic concept expressed using search terms from the first search query is substantially a same linguistic concept expressed using search terms from the second search query.

16. The system of claim 11 , wherein receiving the indication of user interest in the media content item from the computing device comprises receiving the indication that the media content item is currently being presented on the display device proximal to the computing device.

17. The system of claim 11 , wherein receiving the indication of user interest in the media content item from the computing device comprises receiving an audio stream from the computing device and correlating the audio stream to the media content item.

18. The system of claim 11 , wherein matching the portion of search terms from the search queries to the portion of subtitle terms associated with the media content item comprises matching an ordered sequence of terms from a search query in the search queries to a substantially identical ordered sequence of subtitle terms.

19. The system of claim 11 , wherein matching the portion of search terms from the search queries to the portion of subtitle terms associated with the media content item comprises matching a linguistic concept expressed using search terms from a search query with a substantially identical linguistic concept expressed using the subtitle terms.

20. The system of claim 11 , wherein the computing device is a second screen device and the media content item is presented on the display device proximal to the computing device.

21. A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for providing video program information, the method comprising:

collecting, over a network, search queries from a plurality of different computing devices;

identifying, for a first time period, a search query spike from the search queries;

matching, using a classifier trained on data from previously stored subtitles and search query groups, a portion of search terms from the search queries against a plurality of subtitle terms to identify a media content item associated with the search query spike from among a plurality of media content items being presented during the first time period;

receiving, at a second time period, an indication of user interest in the media content item from a computing device, the indication including that the media content item is currently being presented on a display device; and

in response to receiving the indication of user interest in the media content item from the computing device, causing at least one search result corresponding to a search query from the search queries to be transmitted to and displayed on the computing device at a same time that the media content item is currently being presented on the display device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 26, 2024
From: SKOLICKI, ZBIGNIEW; VOLOVICH, YAROSLAV; OZTASKENT, ANT; RAJGARHIA, VARUN
To: GOOGLE INC.
Reel/Frame 068098/0861 →
ENTITY CONVERSION Recorded Jul 26, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 068174/0243 →
Continuity (7)
Continuation 17328196 · May 24, 2021
Continuation 16717909 · Dec 17, 2019
Continuation 16148972 · Oct 1, 2018
Continuation 15438583 · Feb 21, 2017
Continuation 14335834 · Jul 18, 2014
Provisional Application 61982761 · Apr 22, 2014
Related Publication 20230403420A1 · Dec 14, 2023
References Cited (52)
US 4823306A · Barbic et al. · 1989 [cited by applicant]
US 6961954B1 · Maybury et al. · 2005 [cited by applicant]
US 7542967B2 · Hurst-Hiller et al. · 2009 [cited by applicant]
US 8122011B1 · Garg et al. · 2012 [cited by applicant]
US 8176068B2 · Kunjithapatham et al. · 2012 [cited by applicant]
US 8370380B1 · Kuraoka · 2013 [cited by examiner]
US 20070050406A1 · Byers · 2007 [cited by examiner]
US 20080186810A1 · Kumaran · 2008 [cited by examiner]
US 20090254415A1 · Bhavsar · 2009 [cited by applicant]
US 20110289098A1 · Oztaskent et al. · 2011 [cited by applicant]
US 20110289532A1 · Yu et al. · 2011 [cited by applicant]
US 20120019732A1 · Lee et al. · 2012 [cited by applicant]
US 20120066059A1 · Berger · 2012 [cited by applicant]
US 20120150907A1 · Chowdhury et al. · 2012 [cited by applicant]
US 20120189273A1 · Folgner et al. · 2012 [cited by applicant]
US 20120291072A1 · Maddison · 2012 [cited by examiner]
US 20120310628A1 · Rathod et al. · 2012 [cited by applicant]
US 20130024443A1 · Achuthan · 2013 [cited by examiner]
US 20130070163A1 · Connors · 2013 [cited by applicant]
US 20130212113A1 · Acharya et al. · 2013 [cited by applicant]
US 20130298146A1 · Conrad et al. · 2013 [cited by applicant]
US 20140101551A1 · Sherrets et al. · 2014 [cited by applicant]
US 20140140679A1 · Khader et al. · 2014 [cited by applicant]
CN 1640137 · 2005 [cited by applicant]
CN 101887439 · 2010 [cited by applicant]
EP 1909195 · 2008 [cited by applicant]
KR 20100132376 · 2010 [cited by applicant]
WO WO2011102886 · 2011 [cited by applicant]
Decision to Refuse a European Patent Application dated Dec. 13, 2019 in European Patent Application No. 15726482.1. [cited by applicant]
Examination Report dated Dec. 20, 2021 in EP Patent Application No. 20158856.3. [cited by applicant]
Extended European Search Report dated Apr. 21, 2020 in EP Patent Application No. 20158856.3. [cited by applicant]
International Search Report and Written Opinion dated Aug. 4, 2015 in International Patent Application No. PCT/US2015/031621. [cited by applicant]
Nichols, J. et al., “Summarizing Sporting Events Using Twitter”, in Proceedings of ACM International Conference on Intelligent User Interfaces, Lisbon, PT, Feb. 14-17, 2012, pp. 1-10. [cited by applicant]
Notice of Allowance dated Jan. 26, 2021 in U.S. Appl. No. 16/717,909. [cited by applicant]
Notice of Allowance dated Apr. 12, 2023 in U.S. Appl. No. 17/328,196. [cited by applicant]
Notice of Allowance dated May 24, 2018 in U.S. Appl. No. 15/438,583. [cited by applicant]
Notice of Allowance dated Jul. 1, 2016 in U.S. Appl. No. 14/335,834. [cited by applicant]
Notice of Allowance dated Jul. 30, 2018 in KR Patent Application No. 10-2016-7035070. [cited by applicant]
Notice of Allowance dated Aug. 15, 2019 in U.S. Appl. No. 16/148,972. [cited by applicant]
Notice of Allowance dated Aug. 18, 2016 in U.S. Appl. No. 14/335,851. [cited by applicant]
Office Action dated Jan. 3, 2019 in CN Patent Application No. 201580025873.0. [cited by applicant]
Office Action dated Jan. 12, 2018 in U.S. Appl. No. 15/438,583. [cited by applicant]
Office Action dated Apr. 22, 2016 in U.S. Appl. No. 14/335,834. [cited by applicant]
Office Action dated May 16, 2016 in U.S. Appl. No. 14/335,851. [cited by applicant]
Office Action dated Jun. 27, 2019 in U.S. Appl. No. 16/148,972. [cited by applicant]
Office Action dated Jun. 28, 2017 in U.S. Appl. No. 15/438,583. [cited by applicant]
Office Action dated Sep. 3, 2020 in U.S. Appl. No. 16/717,909. [cited by applicant]
Office Action dated Sep. 18, 2015 in U.S. Appl. No. 14/335,834. [cited by applicant]
Office Action dated Sep. 20, 2017 in U.S. Appl. No. 15/438,583. [cited by applicant]
Office Action dated Sep. 29, 2022 in U.S. Appl. No. 17/328,196. [cited by applicant]
Summons to Attend Oral Proceedings dated Mar. 21, 2023 in EP Patent Application No. 20158856.3. [cited by applicant]
Summons to Attend Oral Proceedings dated Apr. 16, 2019 in EP Patent Application No. 15726482.1. [cited by applicant]