IP Library › Granted Patent US 12,639,364
Granted Patent B2
US 12,639,364 · App. 18/740,241 · Granted May 26, 2026

Indexing fingerprints

Inventor: Matthew James Wilkinson (Emeryville, CA)
Assignee: Gracenote, Inc.
G06F16/44G06F16/41G06F16/48H04N21/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,639,364
App. No.
18/740,241
Filed
Jun 11, 2024
Granted
May 26, 2026
Kind
B2
Examiner
WU, TONY
Art Unit
2166
USPC
707/722
Abstract

Example methods and systems for indexing fingerprints are described. Fingerprints may be made up of sub-fingerprints, each of which corresponds to a frame of the media, which is a smaller unit of time than the fingerprint. In some example embodiments, multiple passes are performed. For example, a first pass may be performed that compares the sub-fingerprints of the query fingerprint with every thirty-second sub-fingerprint of the reference material to identify likely matches. In this example, a second pass is performed that compares the sub-fingerprints of the query fingerprint with every fourth sub-fingerprint of the likely matches to provide a greater degree of confidence. A third pass may be performed that uses every sub-fingerprint of the most likely matches, to help distinguish between similar references or to identify with greater precision the timing of the match. Each of these passes is amenable to parallelization.

Claims (39)

1 . A computer-implemented method comprising:

obtaining a query fingerprint comprising a plurality of query sub-fingerprints;

providing the query fingerprint to a plurality of devices, wherein each device in the plurality of devices is associated with a corresponding plurality of segments of reference fingerprints divided from a set of reference fingerprints, wherein each segment includes a plurality of reference sub-fingerprints, wherein each device in the plurality of devices creates one or more thread groups, wherein each thread in the one or more thread groups accesses a portion of an associated segment of reference fingerprints and compares a portion of the query sub-fingerprints with a portion of the reference sub-fingerprints in the accessed portion of the associated segment of reference fingerprints to determine a set of first bit error rates;

identifying a predetermined number of lowest bit error rates based on aggregating the set of first bit error rates determined by each thread in the one or more thread groups, wherein each of the predetermined number of lowest bit error rates corresponds to a reference fingerprint of the set of reference fingerprints;

in response to identifying the predetermined number of lowest bit error rates, for each reference fingerprint of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining a second bit error rate by comparing the query fingerprint with the reference fingerprint;

in response to determining the second bit error rate for one or more of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining at least one response candidate based on the second bit error rates; and

providing the at least one response candidate in a response to a query including the query fingerprint.

2 . The computer-implemented method of claim 1 , further comprising down-sampling the segments of reference fingerprints at a first down-sampling rate for a first comparison pass.

3 . The computer-implemented method of claim 1 , further comprising storing the predetermined number of lowest bit error rates.

4 . The computer-implemented method of claim 1 , wherein each segment of reference fingerprints is divided into one or more chunks, wherein each thread group is associated with a chunk of the one or more chunks.

5 . The computer-implemented method of claim 1 , wherein each thread group forms a warp.

6 . The computer-implemented method of claim 2 , further comprising down-sampling the segments of reference fingerprints at a second down-sampling rate for a second comparison pass.

7 . The computer-implemented method of claim 6 , wherein the first down-sampling rate is greater than the second down-sampling rate.

8 . A tangible, non-transitory computer readable medium comprising instructions that, when executed, cause at least one processor to perform a set of operations comprising:

obtaining a query fingerprint comprising a plurality of query sub-fingerprints;

providing the query fingerprint to a plurality of devices, wherein each device in the plurality of devices is associated with a corresponding plurality of segments of reference fingerprints divided from a set of reference fingerprints, wherein each segment includes a plurality of reference sub-fingerprints, wherein each device in the plurality of devices creates one or more thread groups, wherein each thread in the one or more thread groups accesses a portion of an associated segment of reference fingerprints and compares a portion of the query sub-fingerprints with a portion of the reference sub-fingerprints in the accessed portion of the associated segment of reference fingerprints to determine a set of first bit error rates; identifying a predetermined number of lowest bit error rates based on aggregating the set of first bit error rates determined by each thread in the one or more thread groups, wherein each of the predetermined number of lowest bit error rates corresponds to a reference fingerprint of the set of reference fingerprints;

in response to identifying the predetermined number of lowest bit error rates, for each reference fingerprint of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining a second bit error rate by comparing the query fingerprint with the reference fingerprint;

in response to determining the second bit error rate for one or more of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining at least one response candidate based on the second bit error rates; and

providing the at least one response candidate in a response to a query including the query fingerprint.

9 . The tangible, non-transitory computer readable medium of claim 8 , wherein the set of operations further comprises down-sampling the segments of reference fingerprints at a first down-sampling rate for a first comparison pass.

10 . The tangible, non-transitory computer readable medium of claim 8 , f wherein the set of operations further comprises storing the predetermined number of lowest bit error rates.

11 . The tangible, non-transitory computer readable medium of claim 8 , wherein each segment of reference fingerprints is divided into one or more chunks, wherein each thread group is associated with a chunk of the one or more chunks.

12 . The tangible, non-transitory computer readable medium of claim 8 , wherein each thread group forms a warp.

13 . The tangible, non-transitory computer readable medium of claim 9 , wherein the set of operations further comprises down-sampling the segments of reference fingerprints at a second down-sampling rate for a second comparison pass.

14 . The tangible, non-transitory computer readable medium of claim 13 , wherein the first down-sampling rate is greater than the second down-sampling rate.

15 . A computing device comprising:

at least one processors; and

tangible, non-transitory computer readable medium comprising instructions that, when executed, cause the at least one processor to perform a set of operations comprising:

obtaining a query fingerprint comprising a plurality of query sub-fingerprints;

providing the query fingerprint to a plurality of devices, wherein each device in the plurality of devices is associated with a corresponding plurality of segments of reference fingerprints divided from a set of reference fingerprints, wherein each segment includes a plurality of reference sub-fingerprints, wherein each device in the plurality of devices creates one or more thread groups, wherein each thread in the one or more thread groups accesses a portion of an associated segment of reference fingerprints and compares a portion of the query sub-fingerprints with a portion of the reference sub-fingerprints in the accessed portion of the associated segment of reference fingerprints to determine a set of first bit error rates;

identifying a predetermined number of lowest bit error rates based on aggregating the set of first bit error rates determined by each thread in the one or more thread groups, wherein each of the predetermined number of lowest bit error rates corresponds to a reference fingerprint of the set of reference fingerprints;

in response to identifying the predetermined number of lowest bit error rates, for each reference fingerprint of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining a second bit error rate by comparing the query fingerprint with the reference fingerprint;

in response to determining the second bit error rate for one or more of the reference fingerprints corresponding to the predetermined number of lowest bit error rates, determining at least one response candidate based on the second bit error rates; and

providing the at least one response candidate in a response to a query including the query fingerprint.

16 . The computing device of claim 15 , wherein the set of operations further comprises down-sampling the segments of reference fingerprints at a first down-sampling rate for a first comparison pass.

17 . The computing device of claim 15 , wherein each segment of reference fingerprints is divided into one or more chunks, wherein each thread group is associated with a chunk of the one or more chunks.

18 . The computing device of claim 15 , wherein each thread group forms a warp.

19 . The computing device of claim 16 , wherein the set of operations further comprises down-sampling the segments of reference fingerprints at a second down-sampling rate for a second comparison pass.

20 . The computing device of claim 19 , wherein the first down-sampling rate is greater than the second down-sampling rate.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2024
From: WILKINSON, MATTHEW JAMES
To: GRACENOTE, INC.
Reel/Frame 067706/0886 →
Continuity (5)
Continuation 17901726 · Sep 1, 2022
Continuation 16811626 · Mar 6, 2020
Continuation 15445615 · Feb 28, 2017
Provisional Application 62301372 · Feb 29, 2016
Related Publication 20240330356A1 · Oct 3, 2024
References Cited (61)
US 7240207B2 · Weare · 2007 [cited by applicant]
US 8069176B1 · Ioffe · 2011 [cited by examiner]
US 8165414B1 · Yagnik · 2012 [cited by applicant]
US 8290425B2 · Albrett · 2012 [cited by applicant]
US 8364703B2 · Ramanathan et al. · 2013 [cited by applicant]
US 8411977B1 · Baluja et al. · 2013 [cited by applicant]
US 8594392B2 · Bilobrov · 2013 [cited by applicant]
US 8660296B1 · Loffe · 2014 [cited by applicant]
US 8838609B1 · Sharifi et al. · 2014 [cited by applicant]
US 9143784B2 · Yagnik et al. · 2015 [cited by applicant]
US 9158842B1 · Yagnik et al. · 2015 [cited by applicant]
US 9286902B2 · Han et al. · 2016 [cited by applicant]
US 9438940B2 · Nelson · 2016 [cited by applicant]
US 9558407B1 · McSorley · 2017 [cited by applicant]
US 9684715B1 · Ross et al. · 2017 [cited by applicant]
US 10606879B1 · Wilkinson · 2020 [cited by applicant]
US 20060013451A1 · Haitsma · 2006 [cited by applicant]
US 20070192087A1 · Kim · 2007 [cited by examiner]
US 20070250716A1 · Brunk et al. · 2007 [cited by applicant]
US 20080263360A1 · Haitsma et al. · 2008 [cited by applicant]
US 20090290764A1 · Fiebrink et al. · 2009 [cited by applicant]
US 20100205174A1 · Jiang · 2010 [cited by applicant]
US 20100257129A1 · Lyon et al. · 2010 [cited by applicant]
US 20110022633A1 · Bemosky et al. · 2011 [cited by applicant]
US 20110078754A1 · Howcroft et al. · 2011 [cited by applicant]
US 20110116719A1 · Bilobrov · 2011 [cited by examiner]
US 20110153417A1 · Bemosky · 2011 [cited by applicant]
US 20120011128A1 · He · 2012 [cited by applicant]
US 20120116883A1 · Asam et al. · 2012 [cited by applicant]
US 20120173552A1 · Zhu et al. · 2012 [cited by applicant]
US 20120215329A1 · Jiang et al. · 2012 [cited by applicant]
US 20130197913A1 · Bilobrov · 2013 [cited by applicant]
US 20130279740A1 · Bauer et al. · 2013 [cited by applicant]
US 20130345840A1 · Lempel et al. · 2013 [cited by applicant]
US 20140195548A1 · Harron · 2014 [cited by applicant]
US 20140280265A1 · Wang · 2014 [cited by applicant]
US 20140280304A1 · Scherf · 2014 [cited by examiner]
US 20150007318A1 · Van De Van · 2015 [cited by applicant]
US 20160124966A1 · Cohen · 2016 [cited by applicant]
US 20160148620A1 · Bilobrov · 2016 [cited by applicant]
US 20160217799A1 · Han et al. · 2016 [cited by applicant]
US 20170118538A1 · Ashbacher · 2017 [cited by applicant]
US 20170155933A1 · Del Strother · 2017 [cited by applicant]
US 20200210470A1 · Wilkinson · 2020 [cited by applicant]
KR 20130055115 · 2013 [cited by applicant]
KR 101286862 · 2013 [cited by applicant]
WO 2010135082 · 2010 [cited by applicant]
Haitsma, et al., “A Highly Robust Audio Fingerprinting System,” ISMIR, 2002, 9 pages. [cited by applicant]
Kiao, “A Study on Music Retrieval Based on Audio Fingerprinting,” Department of Information Science and Intelligent Systems Graduate School of Advanced Technology and Science, Sep. 2013, 78 pages. [cited by applicant]
Baluja, et al, “Audio Fingerprinting: Combining Computer Vision & Data Stream Processing,” Google, Inc. 2007, 4 pages. [cited by applicant]
Bhatia, et al., “An Efficient Hash Count Indexing and Searching Scheme for Audio Fingerprinting,” International Journal of Science and Research (IJSR), vol. 3, Issue 9, Sep. 2014, 6 pages. [cited by applicant]
Rafii, “Audio Fingerprinting—EECS 352: Machine Perception of Music & Audio,”2014, 50 pages. [cited by applicant]
Lee, et al, “Robust Video Fingerprinting for Content-Based Video Identification,” IEEE Transaction on Circuits and Systems for Video Technology, vol. 18, No. 7, Jul. 2008, 6 pages. [cited by applicant]
Esmaeili, et al., “A Robust and Fast Video Copy Detection System Using Content=Based Fingerprinting,” IEEE Transactions on Information Forensics and Security, vol. 6, No. 1, Mar. 2011, 14 pages. [cited by applicant]
Liu, et al., “DCT Based Multiple Hashing Technique for Robust Audio Fingerprinting,” School of Electrical Engineering and INMC, Apr. 2009, 4 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 15/445,615, mailed Dec. 3, 2018, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” issued in connection with U.S. Appl. No. 15/445,615, mailed Jul. 3, 2019, 15 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,,” issued in connection with U.S. Appl. No. 15/445,615, mailed Nov. 22, 2019, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 16/811,626, mailed May 12, 2021, 12 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” issued in connection with U.S. Appl. No. 16/811,626, mailed Nov. 22, 2021, 15 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 16/811,626, mailed May 2, 2022, 8 pages. [cited by applicant]