IP Library Granted Patent US 11,589,120
Granted Patent B2
US 11,589,120 · App. 16/799,232 · Granted Feb 21, 2023

Deep content tagging

Inventors: Utkarsh Gaur (Millbrae, CA); Adil Ilyas Jagmag (Hollis, NH); Gaurav Arora (Cupertino, CA)
Assignee: Synaptics Incorporated
H04N21/4666G06F3/167H04N21/4755
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,589,120
App. No.
16/799,232
Granted
Feb 21, 2023
Kind
B2
Abstract

A method and apparatus for deep content tagging. A media device receives one or more first frames of a content item, where the one or more first frames spans a duration of a scene in the content item. The media device detects one or more objects or features in each of the first frames using a neural network model and identifies one or more first genres associated with the first frames based at least in part on the detected objects or features in each of the first frames. The media device further controls playback of the content item based at least in part on the identified first genres.

Claims (44)

1. A method of playing back media content, comprising:

receiving a sequence of frames of a content item, the sequence of frames spanning a duration associated with a plurality of scenes in the content item;

detecting one or more objects or features in each frame in the sequence based on a neural network model;

identifying a respective genre associated with each of the plurality of scenes in the content item based at least in part on the detected objects or features in each frame in the sequence;

generating a genre map associated with the plurality of scenes in the content item, wherein the genre map includes a timeline and a mapping of each of the identified genres to a respective portion of the timeline associated with one of the plurality of scenes;

detecting a genre transition based on differences between the genre associated with a first scene of the plurality of scenes and the genre associated with a second scene of the plurality of scenes, wherein the second scene immediately follows the first scene in the timeline;

modifying the content item to include additional media content between the first scene and the second scene based at least in part on the detected genre transition; and

playing back the modified content item.

2. The method of claim 1 , wherein the one or more objects or features are detected based on at least one of an image, audio, optical flow, or subtitle component of each frame in the sequence.

3. The method of claim 1 , further comprising:

generating recommendations for other media content based at least in part on the genre map.

4. The method of claim 3 , further comprising:

determining an overall genre for the content item based at least in part on the genre map, wherein the recommendations include only media content having an overall genre that matches the overall genre of the content item.

5. The method of claim 1 , further comprising:

selectively initiating playback at one or more frames of the content item based at least in part on the genre map.

6. The method of claim 1 , wherein the modifying of the content item further comprises:

dynamically jumping to or skipping one or more frames of the content item based at least in part on the genre map.

7. The method of claim 1 , wherein the modifying of the content item further comprises:

detecting a user initiating the playback of the content item;

determining one or more preferences of the user; and

dynamically editing the content item based at least in part on the genre map and the one or more preferences of the user.

8. The method of claim 7 , wherein the dynamically editing comprises:

identifying objectionable content in the plurality of scenes in the content item based at least in part on the one or more preferences of the user; and

filtering the objectionable content from the content item.

9. The method of claim 7 , wherein the dynamically editing comprises:

determining an age or demographic group of the user, wherein the editing of the content item is further based on the age or demographic group of the user.

10. The method of claim 7 , wherein the dynamically editing comprises:

determining a source or origin of the content item, wherein the editing of the content item is further based on the source or origin of the content item.

11. The method of claim 7 , wherein the dynamically editing comprises:

determining a time at which the user initiates the playback of the content item, wherein the editing of the content item is further based on the time at which the user initiates the playback of the content item.

12. A media device, comprising:

processing circuitry; and

memory storing instructions that, when executed by the processing circuitry, cause the media device to:

receive a sequence of frames of a content item, the sequence of frames spanning a duration associated with a plurality of scenes in the content item;

detect one or more objects or features in each frame in the sequence based on a neural network model;

identify a respective genre associated with each of the plurality of scenes in the content item based at least in part on the detected objects or features in each frame in the sequence;

generate a genre map associated with the plurality of scenes in the content item, wherein the genre map includes a timeline and a mapping of each of the identified genres to a respective portion of the timeline associated with one of the plurality of scenes;

detect a genre transition based on differences between the genre associated with a first scene of the plurality of scenes and the genre associated with a second scene of the plurality of scenes, wherein the second scene immediately follows the first scene in the timeline;

modify the content item to include additional media content between the first scene and the second scene based at least in part on the detected genre transition; and

playback the modified content item.

13. The media device of claim 12 , wherein execution of the instructions for modifying the content item further causes the media device to:

detect a user initiating the playback of the content item;

determine one or more preferences of the user; and

dynamically edit the content item based at least in part on the genre map and the one or more preferences of the user.

Assignments (2)
SECURITY INTEREST Recorded Mar 12, 2021
From: SYNAPTICS INCORPORATED
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 055581/0737 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2020
From: GAUR, UTKARSH; JAGMAG, ADIL ILYAS; ARORA, GAURAV
To: SYNAPTICS INCORPORATED
Reel/Frame 052766/0162 →