IP Library Granted Patent US 11,514,112
Granted Patent B2
US 11,514,112 · App. 16/846,630 · Granted Nov 29, 2022

Scene aware searching

Inventor: Carlos Santiago (Aurora, CO)
Assignee: TIVO CORPORATION
G06F16/951G06F16/783G06N5/022H04N21/4394H04N21/44008H04N21/4828H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,514,112
App. No.
16/846,630
Granted
Nov 29, 2022
Kind
B2
Abstract

Novel tools and techniques are provided for scene aware searching. A system may include a media player configured to play a video stream, a database, and a server configured to host an artificial intelligence (AI) engine. The server may further include a processor and a non-transitory computer readable medium comprising a set of instructions that, when executable by the processor to receive, from the media device, a search query from a user. The AI engine may further be configured to obtain the video stream associated with the search query, identify one or more objects in the video stream, derive contextual data associated with the one or more objects, identify one or more matches based on the contextual data, and determine a result of the search query.

Claims (79)

1. A method, comprising:

receiving, with an artificial intelligence (AI) engine, a search query from a user;

identifying, with the AI engine, one or more objects in a video stream associated with the search query;

deriving, with the AI engine, contextual data associated with the one or more objects from one or more frames of the video stream, wherein the contextual data includes data determined by the AI engine to be related to the search query and deriving the contextual data from the video stream further comprises:

analyzing, via the AI engine, an audio stream or a segment of the audio stream associated with the video stream;

converting, via the AI engine, one or more audio objects in the audio stream to text; and

generating, via the AI engine, at least one of a keyword or audio signal as the contextual data based on the audio stream;

identifying, with the AI engine, one or more matches based on a search for the contextual data, wherein the one or more matches are entries in one or more data lakes of a database; and

determining, with the AI engine, a result of the search query.

2. The method of claim 1 , further comprising:

receiving, at the AI engine, feedback indicative of a qualify of the result; and

updating, via the AI engine, a belief of the AI engine regarding at least one of the search query, contextual data, or one or more matches based on the feedback.

3. The method of claim 1 , further comprising:

determining, via the AI engine, a first time associated with the search query, wherein the first time is indicative of a timestamp of the video stream when the search query was initiated by a user;

wherein the identification of one or more objects of the data stream and the determination of contextual data is associated with a segment of the video stream within a threshold time of the first time.

4. The method of claim 1 , wherein deriving the contextual data from the video stream further comprises:

analyzing, via the AI engine, one or more frames of a frame stream, wherein the frame stream is a series of images of the video stream;

identifying, via the AI engine, one or more objects in one or more frames of the frame stream; and

generating, via the AI engine, at least one of a keyword or image as the contextual data, based on the frame stream.

5. The method of claim 1 , wherein deriving the contextual data from the video stream further comprises:

analyzing, via the AI engine, a closed captioning stream, wherein the closed captioning stream comprises text for closed captioning associated with the video stream;

identifying, via the AI engine, text associated with the search query; and

generating, via the AI engine, at least one of a keyword as the contextual data, based on the closed captioning stream.

6. The method of claim 1 , further comprising:

receiving, via the AI engine, the video stream and associated information, wherein associated information includes at least one of an audio stream, frame stream, closed captioning stream, and metadata.

7. The method of claim 1 , further comprising:

causing, via the AI engine, a media device to execute a linked application based, at least in part, on the search query.

8. An apparatus, comprising:

at least one processor; and

a non-transitory computer readable medium in communication with the at least one processor, the non-transitory computer readable medium having stored thereon computer software comprising a set of instructions that, when executed by the at least one processor, causes the at least one processor to:

receive, via a media device, a search query from a user;

obtain a video stream associated with the search query;

identify one or more objects in the video stream associated with the search query;

derive contextual data associated with the one or more objects from one or more frames of the video stream, wherein the contextual data includes data determined by the AI engine to be related to the search query and deriving the contextual data from the video stream further comprises instructions executable by the at least one processor to:

analyze an audio stream or a segment of the audio stream associated with the video stream;

convert one or more audio objects in the audio stream to text; and

generate at least one of a keyword or audio signal as the contextual data based on the audio stream;

identify, via a database, one or more matches based on a search for the contextual data, wherein the one or more matches are entries in one or more data lakes of the database; and

determine a result of the search query.

9. The apparatus of claim 8 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

receive, via a media device, feedback indicative of a qualify of the result; and

update a belief regarding at least one of the search query, contextual data, or one or more matches based on the feedback.

10. The apparatus of claim 8 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

analyze one or more frames of a frame stream, wherein the frame stream is a series of images of the video stream;

identify one or more objects in one or more frames of the frame stream; and

generate at least one of a keyword or image as the contextual data, based on the frame stream.

11. The apparatus of claim 8 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

analyze a closed captioning stream, wherein the closed captioning stream comprises text for closed captioning associated with the video stream;

identify text associated with the search query; and

generate at least one of a keyword as the contextual data, based on the closed captioning stream.

12. The apparatus of claim 8 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

receive the video stream and associated information, wherein associated information includes at least one of an audio stream, frame stream, closed captioning stream, and metadata.

13. A system, comprising:

a media player configured to play a video stream;

a database coupled to a plurality of media players including the media player, the database configured to host one or more data lakes comprising a collection of one or more data streams from each of the plurality of media players;

a server configured to host an artificial intelligence (AI) engine, the server coupled to the media player via a network, the server comprising:

at least one processor;

a non-transitory computer readable medium in communication with the at least one processor, the non-transitory computer readable medium having stored thereon computer software comprising a set of instructions that, when executed by the at least one first processor, causes the at least one first processor to:

receive, from the media device, a search query from a user;

obtain the video stream associated with the search query;

identify, via the AI engine, one or more objects in the video stream associated with the search query;

derive, via the AI engine, contextual data associated with the one or more objects from one or more frames of the video stream, wherein the contextual data includes data determined by the AI engine to be related to the search query and wherein the set of instructions further comprise instructions executable by the at least one processor to:

analyze, via the AI engine, one or more frames of a frame stream, wherein the frame stream is a series of images of the video stream;

identify, via the AI engine, one or more objects in one or more frames of the frame stream; and

generate, via the AI engine, at least one of a keyword or image as the contextual data, based on the frame stream;

identify, via a database, one or more matches based on a search for the contextual data, wherein the one or more matches are entries in the one or more data lakes of the database; and

determine, via the AI engine, a result of the search query.

14. The system of claim 13 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

receive, via the media device, feedback indicative of a qualify of the result; and

update, via the AI engine, a belief regarding at least one of the search query, contextual data, or one or more matches based on the feedback.

15. The system of claim 13 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

analyze, via the AI engine, an audio stream or a segment of the audio stream associated with the video stream;

convert, via the AI engine, one or more audio objects in the audio stream to text; and

generate, via the AI engine, at least one of a keyword or audio signal as the contextual data based on the audio stream.

16. The system of claim 13 , wherein the set of instructions further comprise instructions executable by the at least one processor to:

analyze, via the AI engine, a closed captioning stream, wherein the closed captioning stream comprises text for closed captioning associated with the video stream;

identify, via the AI engine, text associated with the search query; and

generate, via the AI engine, at least one of a keyword as the contextual data, based on the closed captioning stream.

17. The system of claim 13 , wherein the database is a relational database.

Assignments (6)
CHANGE OF NAME Recorded Mar 31, 2026
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 075304/0073 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO CORPORATION
To: TIVO LLC
Reel/Frame 069083/0270 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO LLC
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 069083/0339 →
SECURITY INTEREST Recorded May 19, 2023
From: ADEIA GUIDES INC.; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063707/0884 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2021
From: CENTURYLINK INTELLECTUAL PROPERTY LLC
To: TIVO CORPORATION
Reel/Frame 057803/0959 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2020
From: SANTIAGO, CARLOS
To: CENTURYLINK INTELLECTUAL PROPERTY LLC
Reel/Frame 052389/0830 →
Continuity (3)
Continuation 15859131 · Dec 29, 2017
Provisional Application 62516529 · Jun 7, 2017
Related Publication 20200242172A1 · Jul 30, 2020