IP Library Granted Patent US 9,906,831
Granted Patent B2
US 9,906,831 · App. 15/138,678 · Granted Feb 27, 2018

Fingerprinting media content using hashing

Inventor: Juikun Chen (West Jordan, UT)
Assignee: Sorenson Media, Inc.
H04N21/4345G06F17/3002G06F17/30038G06F17/3053G06F17/30097H04N21/84G06F17/30109G06F17/30256G06K9/6215H04N2201/3233
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,906,831
App. No.
15/138,678
Granted
Feb 27, 2018
Kind
B2
Abstract

A server device includes memory to store: a database of frame fingerprints ordered sequentially by media item number and frame number for media items, and a hash table containing (key, value) entries each including a hashed frame fingerprint and index number(s) to the database. A processor coupled to the memory is to, for an input set of hashed frame fingerprints, select a most recent hashed frame fingerprint. The processor performs a lookup with the most recent hashed frame fingerprint to identify a plurality of index numbers from a matching entry. The processor retrieves a target plurality of frame fingerprints from the database using each of the plurality of index numbers as a starting point; calculates a rank distance between the input set and each respective target plurality of frame fingerprints; and selects the target plurality of frame fingerprints for which the rank distance is shortest, to identify a media item.

Claims (49)

1. A server device comprising:

a non-transitory memory storing instructions and configured to store:

a database of frame fingerprints ordered sequentially by media item number and frame number for a plurality of media items; and

a hash table containing a plurality of (key, value) pair entries to the database, each entry including a hashed frame fingerprint and one or more index numbers, each index number mapping the hashed frame fingerprint to a corresponding index slot in the database;

a network interface configured to receive an input set of hashed frame fingerprints of an unidentified media item from a client device, the hashed frame fingerprints ordered chronologically by frame number in the input set; and

a processor operatively coupled to the non-transitory memory and the network interface and configured to execute the instructions, the executed instructions causing the processor to perform operations comprising:

identifying a most recent hashed frame fingerprint in the input set of hashed frame fingerprints;

querying the hash table using the most recent hashed frame fingerprint to identify a plurality of index numbers from the (key, value) pair entry of the hash table that matches the most recent hashed frame fingerprint;

for each identified index number, retrieving a corresponding target block of target frame fingerprints from the database, the target frame fingerprints ordered chronologically by frame number and comprising a most recent target frame fingerprint and one or more preceding target frame fingerprints ordered sequentially backwards in time from the most recent target frame fingerprint, the most recent target frame fingerprint having an index slot in the database corresponding to the identified index number;

determining a corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints;

retrieving a media item identifier from the database for the target block of target frame fingerprints associated with a shortest rank distance from the input set of hashed frame fingerprints, the retrieved media item identifier identifying the media item associated with the input set of hashed frame fingerprints; and

sending the retrieved media item identifier to the client device via the network interface, the client device in response to receiving the retrieved media item identifier, requesting overlay content from an overlay decision engine for display on a viewer in communication with the client device.

2. The server device of claim 1 , wherein a number of frame fingerprints within each corresponding target block of the target frame fingerprints corresponds to a number of the frame fingerprints in the input set of hashed frame fingerprints.

3. The server device of claim 1 , wherein determining the corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints comprises calculating a sum of an absolute difference between ordinal values of respective hashed frame fingerprints of the input set and the target frame fingerprints for each retrieved target block.

4. The server device of claim 1 , wherein the operations further comprise:

when retrieving the media item identifier from the database, retrieving a frame number for the most recent target frame fingerprint from the target block of target frame fingerprints associated with the shortest rank distance; and

when sending the retrieved media item identifier, sending the frame number for the most recent target frame fingerprint to the client device.

5. The server device of claim 1 , wherein the plurality of media items comprise videos or television programs.

6. A method comprising:

storing, by a server device of an automatic content recognition (ACR) system, frame fingerprints of a plurality of media items ordered sequentially by frame number within a database of non-transitory memory of the ACR system;

storing, by the server device, a plurality of (key, value) pair entries within a hash table of the non-transitory memory, each entry including a hashed frame fingerprint and one or more index numbers, each index number mapping the hashed frame fingerprint to a corresponding index slot in the database;

receiving, at the server device, an input set of hashed frame fingerprints of an unidentified media item from a media device in communication with the server device, the hashed frame fingerprints ordered chronologically by frame number in the input set;

identifying, by the server device, a most recent hashed frame fingerprint in the input set of hashed frame fingerprints;

querying, by the server device, the hash table using the most recent hashed frame fingerprint to identify a plurality of index numbers from the (key, value) pair entry of the hash table that matches the most recent hashed frame fingerprint;

for each identified index number, retrieving, by the server device, a corresponding target block of target frame fingerprints from the database, the target frame fingerprints ordered chronologically by frame number and comprising a most recent target frame fingerprint and one or more preceding target frame fingerprints ordered sequentially backwards in time from the most recent target frame fingerprint, the most recent target frame fingerprint having an index slot in the database corresponding to the identified index number;

determining, by the server device, a corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints;

retrieving, by the server device, a media item identifier from the database for the target block of target frame fingerprints associated with a shortest rank distance from the input set of hashed frame fingerprints, the retrieved media item identifier identifying the media item associated with the input set of hashed frame fingerprints; and

sending the retrieved media item identifier from the server device to the media device, the media device in response to receiving the retrieved media item identifier, requesting overlay content from an overlay decision engine for display on a viewer in communication with the media device.

7. The method of claim 6 , wherein a number of frame fingerprints within each corresponding target block of the target frame fingerprints corresponds to a number of the frame fingerprints in the input set of hashed frame fingerprints.

8. The method of claim 6 , wherein determining the corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints comprises calculating a sum of an absolute difference between ordinal values of respective hashed frame fingerprints of the input set and the target frame fingerprints for each retrieved target block.

9. The method of claim 6 , further comprising:

when retrieving the media item identifier from the database, retrieving, by the server device, a frame number for the most recent target frame fingerprint from the target block of target frame fingerprints associated with the shortest rank distance; and

when sending the retrieved media item identifier, sending the frame number for the most recent target frame fingerprint to the media device.

10. The method of claim 6 , where the media item comprises a video or television program.

11. An automatic content recognition (ACR) system comprising a non-transitory computer-readable medium storing instructions and a processing device configured to execute the instructions, the executed instructions causing the processing device to perform operations comprising:

storing frame fingerprints of a plurality of media items ordered sequentially by frame number within a database of the non-transitory computer-readable medium;

storing a plurality of (key, value) pair entries within a hash table of the non-transitory computer-readable medium, each entry including a hashed frame fingerprint and one or more index numbers, each index number mapping the hashed frame fingerprint to a corresponding index slot in the database;

receiving an input set of hashed frame fingerprints of an unidentified media item from a media device in communication with the processing device, the hashed frame fingerprints ordered chronologically by frame number in the input set;

identifying a most recent hashed frame fingerprint in the input set of hashed frame fingerprints;

querying the hash table using the most recent hashed frame fingerprint to identify a plurality of index numbers from the (key, value) pair entry of the hash table that matches the most recent hashed frame fingerprint;

for each identified index number, retrieving a corresponding target block of target frame fingerprints from the database, the target frame fingerprints ordered chronologically by frame number and comprising a most recent target frame fingerprint and one or more preceding target frame fingerprints ordered sequentially backwards in time from the most recent target frame fingerprint, the most recent target frame fingerprint having an index slot in the database corresponding to the identified index number;

determining a corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints;

retrieving a media item identifier from the database for the target block of target frame fingerprints associated with a shortest rank distance from the input set of hashed frame fingerprints, the retrieved media item identifier identifying the media item associated with the input set of hashed frame fingerprints; and

sending the retrieved media item identifier to the media device via the network interface, the media device in response to receiving the retrieved media item identifier, requesting overlay content from an overlay decision engine for display on a viewer in communication with the media device.

12. The non-transitory computer-readable medium of claim 11 , wherein a number of frame fingerprints within each corresponding target block of the target frame fingerprints corresponds to a number of the frame fingerprints in the input set of hashed frame fingerprints.

13. The non-transitory computer-readable medium of claim 11 , wherein determining the corresponding rank distance from the input set of hashed frame fingerprints and each of the retrieved target blocks of target frame fingerprints comprises calculating a sum of an absolute difference between ordinal values of respective hashed frame fingerprints of the input set and the target frame fingerprints for each retrieved target block.

14. The non-transitory computer-readable medium of claim 11 , wherein the operations further comprise:

when retrieving the media item identifier from the database, retrieving a frame number for the most recent target frame fingerprint from the target block of target frame fingerprints associated with the shortest rank distance; and

when sending the retrieved media item identifier, sending the frame number for the most recent target frame fingerprint to the media device.

Assignments (12)
SECURITY INTEREST Recorded Sep 18, 2024
From: ROKU, INC.
To: CITIBANK, N.A.
Reel/Frame 068982/0377 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT (REEL/FRAME 056982/0194) Recorded Feb 22, 2023
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROKU, INC.; ROKU DX HOLDINGS, INC.
Reel/Frame 062826/0664 →
PATENT SECURITY AGREEMENT SUPPLEMENT Recorded Jun 29, 2021
From: ROKU, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 056982/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 30, 2021
From: THE NIELSEN COMPANY (US), LLC
To: ROKU, INC.
Reel/Frame 056106/0376 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Apr 20, 2021
From: CITIBANK, N.A.
To: THE NIELSEN COMPANY (US), LLC; GRACENOTE, INC.
Reel/Frame 056973/0280 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2019
From: SORENSON MEDIA, INC.
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 048355/0949 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF INVENTOR PREVIOUSLY RECORDED ON REEL 038387 FRAME 0540. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Recorded Apr 7, 2017
From: CHEN, JUIKUN
To: SORENSON MEDIA, INC.
Reel/Frame 042190/0969 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 26, 2016
From: CHEN, JUI-KUN
To: SORENSON MEDIA, INC.
Reel/Frame 038387/0540 →
Continuity (2)
Provisional Application 62299455 · Feb 24, 2016
Related Publication 20170244999A1 · Aug 24, 2017