Media identification system for efficient matching of media items having common content
View Patent ↗To identify a media item from a database of media items that have common content, a region of interest is defined to include a plurality of frames of a test fingerprint that correspond to different segments of a media item. A media identification system queries a database of reference fingerprints to identify candidate reference fingerprints that contain a frame that matches a frame of the test fingerprint. When a candidate reference fingerprint is found, additional matching frames are determined and the region of interest is reduced to eliminate the matched frames of the test fingerprint. This continues until the region of interest is empty or there are no further matching candidates. Once the set of candidate reference fingerprints are identified, the media identification system compares the test fingerprint to the candidates to determine a closest match, thereby identifying the media item associated with the test fingerprint.
1. A method for identifying a media item, the method comprising:
receiving, by a processor, a test fingerprint for a media item to be identified, the test fingerprint comprising a plurality of frames corresponding to segments of the media item, wherein a region of interest is defined for the test fingerprint to include the frames of the test fingerprint;
accessing, by the processor, a database of reference fingerprints, each reference fingerprint comprising a plurality of frames corresponding to segments of known media items;
performing, by the processor, a candidate search process a plurality of times, the candidate search process comprising:
querying the database to match one or more frames of the test fingerprint that are within the region of interest to one or more frames of a reference fingerprint in the database,
identifying the matching reference fingerprint as a candidate fingerprint, and
redefining the region of interest to eliminate the matched frames of the test fingerprint from the region of interest; and
returning, by the processor, one or more of the candidate fingerprints as being a possible match for the test fingerprint.
2. The method of claim 1 , further comprising:
comparing the test fingerprint with the identified candidate fingerprints; and
identifying the unknown media item based at least in part on the comparing.
3. The method of claim 2 , wherein comparing the test fingerprint with the identified candidate fingerprints comprises, for each candidate fingerprint, performing a bitwise comparison of at least a portion of the test fingerprint with at least a portion of the candidate fingerprint.
4. The method of claim 2 , wherein identifying the unknown media item comprises obtaining metadata for the known media item corresponding to the candidate fingerprint that most closely matches the test fingerprint based on the comparing.
5. The method of claim 1 , wherein the media item comprises a sampled audio signal.
6. The method of claim 1 , wherein the frames of the test fingerprint correspond to time segments of the media item.
7. The method of claim 1 , wherein the frames of the test fingerprint have a plurality of dimensions.
8. The method of claim 1 , wherein the plurality of dimensions include spatial and time dimensions.
9. The method of claim 1 , wherein querying the database in the candidate search process comprises:
identifying a first frame of a reference fingerprint in the database that matches a frame of the test fingerprint; and
identifying additional frames of the reference fingerprint that are adjacent to the first frame and match corresponding adjacent frames of the test fingerprint.
10. The method of claim 1 , wherein the candidate search process is repeated until at least: (a) all of the frames of the test fingerprint have been eliminated from the region of interest, or (b) the database contains no more reference fingerprints having frames that match a frame of the test fingerprint within the region of interest.
11. The method of claim 1 , wherein querying the database comprises accessing an index for the database stored in a computer memory, and comparing the test fingerprint with the identified candidate fingerprints comprises accessing the candidate fingerprints from a storage memory.
12. The method of claim 1 , wherein querying the database in the candidate search process comprises a dicotomic search algorithm that looks for a matching frame around the middle of a segment of the region of interest.
13. The method of claim 1 , wherein the region of interest is defined by a list of masked frames of the test fingerprint that are considered outside the region of interest.
14. The method of claim 1 , wherein querying the database in the candidate search process comprises indexing frames of the reference fingerprints in the database only within the region of interest, and then querying using the indexed frames.
15. The method of claim 1 , further comprising:
defining a region of interest for one or more of the reference fingerprints in the database, wherein querying the database in the candidate search process ignores frames in the reference fingerprints that are outside of the region of interest defined for each reference fingerprint.
16. The method of claim 1 , wherein the database comprises a single layer distributed database, wherein the querying comprises sending a query packet from a control computer to a plurality of distributed node computers, and upon finding a match by a node computer, sending a broadcast message from the node computer to the other node computers with instructions to modify the region of interest, and wherein redefining the region of interest comprises, responsive to receiving the broadcast message in each of the other nodes, reducing the region of interest to eliminate the matched frames.
17. The method of claim 1 , wherein the database comprises a multilayer database, wherein a first layer of the database is configured to identify and filter out reference fingerprints for media items that frequently appear in the searches, and a second layer of the database is configured to identify the candidate reference fingerprints without loading and verifying reference fingerprints that were filtered out by the first layer.
18. A method for identifying a media item, the method comprising:
receiving, by a processor, a test fingerprint for a media item to be identified, the test fingerprint comprising a plurality of frames corresponding to segments of the media item;
accessing, by the processor, a database of reference fingerprints, each reference fingerprint comprising a plurality of frames generated based on segments of known media items;
querying, by the processor, the database to match one or more frames of the test fingerprint to one or more frames of a first reference fingerprint in the database;
defining, by the processor, a region of interest for the test fingerprint to include the frames of the test fingerprint for which a match with the first reference fingerprint was not found;
querying, by the processor, the database to match one or more frames of the test fingerprint that are within the region of interest to one or more frames of a second reference fingerprint in the database;
comparing, by the processor, the test fingerprint with the first and second reference fingerprints; and
identifying, by the processor, the unknown media item based at least in part on the comparing.
19. The method of claim 18 , wherein querying the database comprises: identifying a first frame of a reference fingerprint in the database that matches a frame of the test fingerprint; and identifying additional frames of the reference fingerprint that are adjacent to the first frame and match corresponding adjacent frames of the test fingerprint.
20. The method of claim 18 , wherein comparing the test fingerprint with the first and second reference fingerprints comprises, for each candidate fingerprint, performing a bitwise comparison of at least a portion of the test fingerprint with at least a portion of the candidate fingerprint.
21. The method of claim 18 , wherein identifying the unknown media item comprises obtaining metadata for the known media item corresponding to the candidate fingerprint that most closely matches the test fingerprint based on the comparing.
22. The method of claim 18 , wherein querying the database comprises accessing an index for the database stored in a computer memory, and comparing the test fingerprint with the first and second reference fingerprints comprises accessing the candidate fingerprints from a storage memory.
23. A non-transitory computer readable storage medium for identifying a media item, the computer readable storage medium comprising computer program code executed by a processor for:
receiving, by the processor, a test fingerprint for a media item to be identified, the test fingerprint comprising a plurality of frames corresponding to segments of the media item, wherein a region of interest is defined for the test fingerprint to include the frames of the test fingerprint;
accessing, by the processor, a database of reference fingerprints, each reference fingerprint comprising a plurality of frames corresponding to segments of known media items;
performing, by the processor, a candidate search process a plurality of times, the candidate search process comprising:
querying the database to match one or more frames of the test fingerprint that are within the region of interest to one or more frames of a reference fingerprint in the database,
identifying the matching reference fingerprint as a candidate fingerprint, and
redefining the region of interest to eliminate the matched frames of the test fingerprint from the region of interest; and
returning, by the processor, one or more of the candidate fingerprints as being a possible match for the test fingerprint.
24. The non-transitory computer readable storage medium of claim 23 , further comprising computer program code for:
comparing the test fingerprint with the identified candidate fingerprints; and
identifying the unknown media item based at least in part on the comparing.
25. The non-transitory computer readable storage medium of claim 23 , wherein querying the database comprises:
identifying a first frame of a reference fingerprint in the database that matches a frame of the test fingerprint; and
identifying additional frames of the reference fingerprint that are adjacent to the first frame and match corresponding adjacent frames of the test fingerprint.
26. The non-transitory computer readable storage medium of claim 23 , wherein comparing the test fingerprint with the with the first and second reference fingerprints comprises, for each candidate fingerprint, performing a bitwise comparison of at least a portion of the test fingerprint with at least a portion of the candidate fingerprint.
27. The non-transitory computer readable storage medium of claim 23 , wherein identifying the unknown media item comprises obtaining metadata for the known media item corresponding to the candidate fingerprint that most closely matches the test fingerprint based on the comparing.
28. The non-transitory computer readable storage medium of claim 23 , wherein querying the database comprises accessing an index for the database stored in a computer memory, and comparing the test fingerprint with the first and second reference fingerprints comprises accessing the candidate fingerprints from a storage memory.