IP Library Granted Patent US 10,007,838
Granted Patent B2
US 10,007,838 · App. 15/375,868 · Granted Jun 26, 2018

Media content enrichment using an adapted object detector

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,007,838
App. No.
15/375,868
Filed
Dec 12, 2016
Granted
Jun 26, 2018
Kind
B2
Examiner
GRAY, RYAN M
Art Unit
2611
USPC
382/118
Abstract

Disclosed herein are a system, method and architecture for media content enrichment. A visual object detector is trained using a training data set and an existing visual object detector. The newly-adapted visual object detector may be used to detect a visual object belonging to a class of visual object. The existing object detector that is used to train the adapted object detector detects a class of visual objects different from the visual object class detected by the adapted object detector. A media content item depicting a visual object detected using the adapted object detector may be associated with metadata, tag or other information about the detected visual object to enrich the media content item.

Claims (46)

1. A method comprising:

obtaining, by a computing device, an existing object detector used to detect a first object class;

splitting, by the computing device, the existing object detector into portions, the portions of the object detector comprising a head portion, a torso potion and a tail portion, the torso potion of the object detector detecting stable object variations;

generating, by the computing device, an adapted head portion for use in forming an adapted object detector, generation of the adapted head portion comprising retraining the head portion of the existing object detector;

generating, by the computing device, an adapted tail portion for use in forming the adapted object detector, generation of the adapted tail portion comprising retraining the tail potion of the existing object detector;

forming, by the computing device, the adapted object detector using the adapted head portion, the adapted tail portion and the torso portion, the adapted object detector comprising the adapted head portion generated by retraining the head portion of the existing object detector, the adapted tail portion generated by retraining the tail portion of the existing object detector, and the torso portion of the existing object detector as the torso portion of the adapted object detector;

analyzing, by the computing device, a media content item using the adapted object detector to determine whether the media content item depicts a visual object belonging to a visual object class detected by the adapted object detector.

2. The method of claim 1 , the existing object detector is a human face detector.

3. The method of claim 2 , the adapted object detector is a non-human face detector.

4. The method of claim 3 , the adapted object detector is a non-human face detector for use in detecting a face of an animated character.

5. The method of claim 1 , the existing object detector comprising a cascade of classifiers.

6. The method of claim 5 , splitting the existing object detector into a head portion, a torso potion and a tail portion further comprising:

splitting, by the computing device, the cascade of classifiers of the existing object detector into three subsets, the head portion of the existing object detector comprising a first subset of the classifiers of the cascade, the torso portion comprising a second subset of the classifiers of the cascade, and the tail portion comprising a third subset of the classifiers of the cascade.

7. The method of claim 1 , the torso portion of the existing object detector is used in the adapted object detector to capture stable variations in visual objects belonging to the visual object class being detected by the adapted object detector.

8. The method of claim 1 , the tail portion of the adapted object detector comprising a probabilistic generative model that is based on a non-negative matrix factorization.

9. The method of claim 1 , the adapted head portion of the adapted object detector being used to increase recall of the adapted object detector and the adapted tail portion of the adapted object detector being used to increase precision of the adapted object detector.

10. The method of claim 1 , a training data set of media content items is used in the generation of the adapted head portion and in the generation of the adapted tail portion, each media content item of the training data set having a number of associated tags indicating that the media content item depicts at least one visual object belonging to the visual object class detected by the adapted object detector.

11. The method of claim 1 , the analyzing further comprising:

determining, by the computing device using the adapted object detector, that the media content item depicts the visual object belonging to the visual object class detected by the adapted object detector; and

associating, by the computing device, metadata with the media content item identifying the visual object detected by the adapted object detector.

12. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:

obtaining an existing object detector used to detect a first object class;

splitting the existing object detector into three portions, the three portions of the object detector comprising a head portion, a torso potion and a tail portion, the torso portion of the object detector detecting stable variations;

generating an adapted head portion for use in forming an adapted object detector, generation of the adapted head portion comprising retraining the head portion of the existing object detector;

generating an adapted tail portion for use in forming the adapted object detector, generation of the adapted tail portion comprising retraining the tail potion of the existing object detector;

forming the adapted object detector using the adapted head portion, the adapted tail portion and the torso portion, the adapted object detector comprising the adapted head portion generated by retraining the head portion of the existing object detector, the adapted tail portion generated by retraining the tail portion of the existing object detector, and the torso portion of the existing object detector as the torso portion of the adapted object detector;

analyzing a media content item using the adapted object detector to determine whether the media content item depicts a visual object belonging to a visual object class detected by the adapted object detector.

13. The non-transitory computer-readable storage medium of claim 12 , the existing object detector is a human face detector and the adapted object detector is a non-human face detector for use in detecting a face of an animated character.

14. The non-transitory computer-readable storage medium of claim 12 , splitting the existing object detector into a head portion, a torso potion and a tail portion further comprising:

splitting, by the computing device, a cascade of classifiers of the existing object detector into three subsets, the head portion of the existing object detector comprising a first subset of the classifiers of the cascade, the torso portion comprising a second subset of the classifiers of the cascade, and the tail portion comprising a third subset of the classifiers of the cascade.

15. The non-transitory computer-readable storage medium of claim 12 , the torso portion of the existing object detector is used in the adapted object detector to capture stable variations in visual objects belonging to the visual object class being detected by the adapted object detector.

16. The non-transitory computer-readable storage medium of claim 12 , the tail portion of the adapted object detector comprising a probabilistic generative model that is based on a non-negative matrix factorization.

17. The non-transitory computer-readable storage medium of claim 12 , the adapted head portion of the adapted object detector being used to increase recall of the adapted object detector and the adapted tail portion of the adapted object detector being used to increase precision of the adapted object detector.

18. The non-transitory computer-readable storage medium of claim 12 , a training data set of media content items is used in the generation of the adapted head portion and in the generation of the adapted tail portion, each media content item of the training data set having a number of associated tags indicating that the media content item depicts at least one visual object belonging to the visual object class detected by the adapted object detector.

19. The non-transitory computer-readable storage medium of claim 12 , the analyzing further comprising:

determining, by the computing device using the adapted object detector, that the media content item depicts the visual object belonging to the visual object class detected by the adapted object detector; and

associating, by the computing device, metadata with the media content item identifying the visual object detected by the adapted object detector.

20. A computing device comprising:

a processor;

a non-transitory storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:

obtaining logic executed by the processor for obtaining an existing object detector used to detect a first object class;

splitting logic executed by the processor for splitting the existing object detector into portions, the portions of the object detector comprising a head portion, a torso potion and a tail portion, the torso portion of the object detector detecting stable object variations;

generating logic executed by the processor for generating an adapted head portion for use in forming the adapted object detector, generation of an adapted head portion comprising retraining the head portion of the existing object detector;

generating logic executed by the processor for generating an adapted tail portion for use in forming the adapted object detector, generation of the adapted tail portion comprising retraining the tail potion of the existing object detector;

forming logic executed by the processor for forming the adapted object detector using the adapted head portion, the adapted tail portion and the torso portion, the adapted object detector comprising the adapted head portion generated by retraining the head portion of the existing object detector, the adapted tail portion generated by retraining the tail portion of the existing object detector, and the torso portion of the existing object detector as the torso portion of the adapted object detector;

analyzing logic executed by the processor for analyzing a media content item using the adapted object detector to determine whether the media content item depicts a visual object belonging to a visual object class detected by the adapted object detector.

Assignments (6)
PATENT SECURITY AGREEMENT (FIRST LIEN) Recorded Sep 29, 2022
From: YAHOO ASSETS LLC
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 061571/0773 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: YAHOO AD TECH LLC (FORMERLY VERIZON MEDIA INC.)
To: YAHOO ASSETS LLC
Reel/Frame 058982/0282 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 045240/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 10, 2017
From: JAIN, VIDIT; FARFADE, SACHIN SUDHAKAR
To: YAHOO! INC.
Reel/Frame 043820/0182 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2017
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 042963/0211 →