IP Library Granted Patent US 11,853,370
Granted Patent B2
US 11,853,370 · App. 17/975,176 · Granted Dec 26, 2023

Scene aware searching

Inventor: Carlos Santiago (Aurora, CO)
Assignee: TiVo Corporation
G06F16/951G06F16/783G06N5/022H04N21/4394H04N21/44008H04N21/4828H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,853,370
App. No.
17/975,176
Granted
Dec 26, 2023
Kind
B2
Abstract

Novel tools and techniques are provided for scene aware searching. A system may include a media player configured to play a video stream, a database, and a server configured to host an artificial intelligence (AI) engine. The server may further include a processor and a non-transitory computer readable medium comprising a set of instructions that, when executable by the processor to receive, from the media device, a search query from a user. The AI engine may further be configured to obtain the video stream associated with the search query, identify one or more objects in the video stream, derive contextual data associated with the one or more objects, identify one or more matches based on the contextual data, and determine a result of the search query.

Claims (47)

1. A method, comprising:

receiving, at an artificial intelligence (AI) engine, a first search query from a user;

determining whether the first search query is related to a video stream that is currently being consumed by the user;

in response to determining that the first search query is related to the video stream that is currently being consumed by the user: determining context related to the first search query, wherein the determination comprises analyzing one or more objects in a frame of the video stream currently being consumed by the user;

refining, by the AI engine, the context determined based on the first search query in response to receiving a second search query, wherein the second search query is related to the first search query;

identifying, by the AI engine, one or more matches based on the refined context, wherein the one or more matches are entries in one or more data lakes of a database; and

displaying results of the identified matches.

2. The method of claim 1 , further comprising:

identifying that the frame of the video stream currently being consumed by the user includes a first and a second object, from the one or more objects;

determining that the second search query identifies the first object; and

in response to the determination, eliminating contextual data relating to the second object.

3. The method of claim 2 , wherein refining the context determined based on the first search query includes, removing the second object from being considered in determining the refined context.

4. The method of claim 1 , further comprising:

identifying results of matches based on context determined for the first search query; and

determining accuracy of the identified results based on the received second search query.

5. The method of claim 4 , further comprising, using data collected based on the accuracy determination to refine future searches.

6. The method of claim 1 , wherein the second search query identifies an object from the one or more objects in the frame of the video stream currently being consumed by the user.

7. The method of claim 1 , further comprising: receiving, at the AI engine, the video stream and information associated to the video stream, wherein information associated to the video stream includes at least one of an audio stream, frame stream, closed captioning stream, and metadata.

8. The method of claim 1 , wherein determining whether the first search query is related to the video stream that is currently being consumed by the user comprises analyzing a combination of textual, audio, and touch input.

9. The method of claim 1 , wherein the first search query is related to a request to purchase the one or more objects in the frame of the video stream currently being consumed by the user.

10. The method of claim 1 , wherein the first search query is related to identify a location of the one or more objects in the frame of the video stream currently being consumed by the user.

11. A system, comprising:

a media player configured to play a video stream;

a database coupled to a plurality of media players including the media player, the database configured to host one or more data lakes comprising a collection of one or more video streams from each of the plurality of media players;

a server configured to host an artificial intelligence (AI) engine, the server coupled to the media player via a network, the server comprising:

at least one processor;

a non-transitory computer readable medium in communication with the at least one processor, the non-transitory computer readable medium having stored thereon computer software comprising a set of instructions that, when executed by the at least one first processor, causes the at least one first processor to:

receive a first search query from a user;

determine whether the first search query is related to a video stream that is currently being consumed by the user;

in response to determining that the first search query is related to the video stream that is currently being consumed by the user: determine context related to the first search query, wherein the determination comprises analyzing one or more objects in a frame of the video stream currently being consumed by the user;

refine the context determined based on the first search query in response to receiving a second search query, wherein the second search query is related to the first search query;

identify one or more matches based on the refined context, wherein the one or more matches are entries in one or more data lakes of a database; and

display results of the identified matches.

12. The system of claim 11 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

identify that the frame of the video stream currently being consumed by the user includes a first and a second object, from the one or more objects;

determine that the second search query identifies the first object; and

in response to the determination, eliminate contextual data relating to the second object.

13. The system of claim 12 , wherein refining the context determined based on the first search query includes, instructions executable by the at least one processor to remove the second object from being considered in determining the refined context.

14. The system of claim 11 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

identify results of matches based on context determined for the first search query; and

determine accuracy of the identified results based on the received second search query.

15. The system of claim 14 , wherein the set of instructions further comprise instructions executable by the at least one processor to use data collected based on the accuracy determination to refine future searches.

16. The system of claim 11 , wherein the second search query identifies an object from the one or more objects in the frame of the video stream currently being consumed by the user.

17. The system of claim 11 , wherein the set of instructions further comprise instructions executable by the at least one processor to receive the video stream and information associated to the video stream, wherein information associated to the video stream includes at least one of an audio stream, frame stream, closed captioning stream, and metadata.

18. The system of claim 11 , wherein determining whether the first search query is related to the video stream that is currently being consumed by the user comprises instructions executable by the at least one processor to analyze a combination of textual, audio, and touch input.

19. The system of claim 11 , wherein the first search query is related to a request to purchase the one or more objects in the frame of the video stream currently being consumed by the user.

20. The system of claim 11 , wherein the first search query is related to identify a location of the one or more objects in the frame of the video stream currently being consumed by the user.

Assignments (6)
CHANGE OF NAME Recorded Mar 31, 2026
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 075304/0073 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO CORPORATION
To: TIVO LLC
Reel/Frame 069083/0270 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO LLC
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 069083/0339 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2023
From: SANTIAGO, CARLOS
To: CENTURYLINK INTELLECTUAL PROPERTY LLC
Reel/Frame 062846/0343 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2023
From: CENTURYLINK INTELLECTUAL PROPERTY LLC
To: TIVO CORPORATION
Reel/Frame 062846/0370 →
Continuity (4)
Continuation 16846630 · Apr 13, 2020
Continuation 15859131 · Dec 29, 2017
Provisional Application 62516529 · Jun 7, 2017
Related Publication 20230185862A1 · Jun 15, 2023
Cited By (1)
US 12,608,391