IP Library Granted Patent US 9,465,995
Granted Patent B2
US 9,465,995 · App. 14/061,043 · Granted Oct 11, 2016

Identifying video content via color-based fingerprint matching

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,465,995
App. No.
14/061,043
Granted
Oct 11, 2016
Kind
B2
Abstract

Systems and methods of identifying unknown video content are described. An example method may include receiving a first fingerprint and a second fingerprint. The first fingerprint may be a color-based fingerprint derived from colors in a portion of the unknown video content, and the second fingerprint may be at least partially based on a feature other than the colors of the unknown video content. A reference database of reference fingerprints may then be queried using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints. The candidate group of fingerprints may then be queried using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint. The unknown video content may then be identified using the at least one query fingerprint. For example, the second fingerprint may be a luminance-based fingerprint derived from luminance in the unknown video content.

Claims (36)

1. A method of identifying unknown video content, the method comprising:

receiving, in an identification query from a client device, a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the same portion of the unknown video content,

responsive to the identification query, querying a reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried;

responsive to the identification query, querying the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

identifying the unknown video content using the at least one query fingerprint.

2. The method of claim 1 , wherein the second fingerprint is a luminance-based fingerprint derived from luminance in the unknown video content.

3. The method of claim 1 , wherein the reference database is queried with the first fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the second fingerprint to identify the at least one query fingerprint.

4. The method of claim 1 , wherein the reference database is queried with the second fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the first fingerprint to identify the at least one query fingerprint.

5. The method of claim 1 , wherein the query fingerprint is based on a value calculated for multiple patches of a frame of video content using an integral image technique and includes a value based on multiple colors within the frame of video content.

6. The method of claim 1 , wherein the query fingerprint is based on a value calculated for multiple Haar-like features of the frame of video content and includes a value based on multiple colors within the frame of video content.

7. The method of claim 1 , wherein the query fingerprint is based on:

a value calculated for multiple Haar-like features of the frame of video content, wherein each of the Haar-like features is associated with a unique location of a unique scale within the frame of video content; and

a value based on multiple colors within the frame of video content.

8. The method of claim 1 , wherein the color-based fingerprint is based on a quantization of eight dominant colors within at least a portion of a frame of the unknown video content.

9. The method of claim 1 , wherein the color-based fingerprint is based on a quantization of sixteen dominant colors within at least a portion of a frame of the unknown video content.

10. The method of claim 1 , wherein querying a reference database using one of the first fingerprint or the second fingerprint uses a fingerprint defined by a lower number of bits than that used to query the candidate group.

11. The method of claim 1 , wherein querying the reference database using the first fingerprint comprises querying an index of 8-bit numbers that are associated with colors within frames of known video content.

12. The method of claim 1 , wherein querying the reference database of reference fingerprints using color-based fingerprint includes querying an index of 16-bit numbers that are associated with colors within the frames of known video content.

13. The method of claim 1 , wherein querying the reference database of reference fingerprints using color-based fingerprint includes querying an index of 8-bit numbers that are associated with colors within the frames of known video content; and

querying the candidate group of reference fingerprints includes querying an index of 24-bit numbers that are associated with single frames of the reference fingerprints.

14. The method of claim 1 , wherein the querying the database of reference fingerprints using the first fingerprint includes querying an index of 16-bit numbers that are associated with colors within the frames of known video content, and wherein querying the group of candidate reference fingerprints using the second fingerprint includes querying an index of 24-bit numbers that are associated with single frames of the reference fingerprints.

15. The method of claim 1 , wherein querying the reference database with the color-based fingerprint comprises determining that a similarity between the color-based fingerprint and the reference fingerprints satisfies a predetermined threshold associated with a Tanimoto distance measurement.

16. The method of claim 1 , wherein querying the reference database with the color-based fingerprint comprises determining that a similarity between the color-based fingerprint and the reference fingerprints satisfies a predetermined threshold associated with a Manhattan distance measurement.

17. The method of claim 1 , wherein the first and the second fingerprint correspond to one of a plurality of patches, the patches being defined by regions of a frame of the unknown video content.

18. A system, comprising:

a hardware processor-based fingerprint module configured to receive an identification query including a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the same portion of the unknown video content;

a hardware processor-based query module configured to perform the following operations responsive to the identification query:

query a reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried;

query the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

a hardware processor-based identification module configured to identify the unknown video content using the at least one query fingerprint.

19. The system of claim 18 , wherein the second fingerprint is a luminance-based fingerprint derived from luminance in the unknown video content.

20. A non-transitory computer-readable medium embodying instructions which, when executed by a machine, cause the machine to perform operations comprising:

receive, in an identification query from a client device, a first fingerprint and a second fingerprint, the first fingerprint being a color-based fingerprint derived from colors in a portion of the unknown video content, the second fingerprint being at least partially based on a feature other than the colors of the same portion of the unknown video content;

responsive to the identification query, query a reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried;

responsive to the identification query, query the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint; and

identify the unknown video content using the at least one query fingerprint.

Assignments (14)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
RELEASE (REEL 042262 / FRAME 0601) Recorded Oct 13, 2022
From: CITIBANK, N.A.
To: GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC
Reel/Frame 061748/0001 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: GRACENOTE, INC.
To: ROKU, INC.
Reel/Frame 056103/0786 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Apr 13, 2017
From: GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE DIGITAL VENTURES, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 042262/0601 →
RELEASE OF SECURITY INTEREST IN PATENT RIGHTS Recorded Feb 8, 2017
From: JPMORGAN CHASE BANK, N.A.
To: GRACENOTE, INC.; CASTTV INC.; TRIBUNE MEDIA SERVICES, LLC; TRIBUNE DIGITAL VENTURES, LLC
Reel/Frame 041656/0804 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2014
From: HARRON, WILSON; CREMER, MARKUS K
To: GRACENOTE, INC.
Reel/Frame 032789/0556 →
SECURITY INTEREST Recorded Mar 19, 2014
From: GRACENOTE, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 032480/0272 →