Identifying video content via color-based fingerprint matching
Systems and methods of identifying unknown video content are described. An example method may include receiving a first fingerprint and a second fingerprint. The first fingerprint may be a color-based fingerprint derived from colors in a portion of the unknown video content, and the second fingerprint may be at least partially based on a feature other than the colors of the unknown video content. A reference database of reference fingerprints may then be queried using one of the first fingerprint or the second fingerprint to obtain a candidate group of fingerprints. The candidate group of fingerprints may then be queried using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint. The unknown video content may then be identified using the at least one query fingerprint.
1. A method comprising:
determining a first fingerprint for unknown video content, the first fingerprint being a color-based fingerprint based on a set of colors in a frame of the unknown video content;
determining a second fingerprint for the unknown video content, the second fingerprint being a patch-based fingerprint based on a plurality of patch-based values, wherein determining the second fingerprint comprises:
dividing the frame of the unknown video content into a plurality of regions;
selecting a plurality of patches in the frame, wherein selecting the plurality of patches in the frame comprises selecting at least two patches in each region of the plurality of regions; and
deriving a patch-based value for each respective patch of the plurality of patches to obtain the plurality of patch-based values;
initiating a query of at least one reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried; and
responsive to the query, initiating a query of the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint, wherein the unknown video content is identified using the at least one query fingerprint.
2. The method of claim 1 , further comprising receiving the unknown video content at a media playback device including a hardware processor, the query of the at least one reference database and the query of the candidate group of fingerprints being performed by the hardware processor.
3. The method of claim 2 , wherein the at least one reference database is stored in memory of the media playback device.
4. The method of claim 1 , wherein initiating the query of the at least one reference database of reference fingerprints comprises sending the first fingerprint and the second fingerprint to a remote server system to execute the query, the at least one reference database being part of the remote server system.
5. The method of claim 1 , wherein the at least one reference database comprises:
a color-based fingerprint database including reference fingerprints that are color-based; and
a patch-based fingerprint database including reference fingerprints that are patch-based.
6. The method of claim 1 , wherein the set of colors includes more than 8 colors.
7. The method of claim 6 , wherein the set of colors includes 16 colors.
8. The method of claim 1 , wherein the at least one reference database is queried with the first fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the second fingerprint to identify the at least one query fingerprint.
9. The method of claim 1 , wherein the at least one reference database is queried with the second fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the first fingerprint to identify the at least one query fingerprint.
10. The method of claim 1 , wherein the color-based fingerprint is based on a plurality of color-based values, wherein each color-based value is determined for a respective color in the set of colors.
11. The method of claim 1 , wherein initiating the query of the at least one reference database using one of the first fingerprint or the second fingerprint uses a fingerprint defined by a lower number of bits than that used to query the candidate group.
12. The method of claim 1 , wherein initiating the query of the at least one reference database comprises querying the at least one reference database of reference fingerprints using the color-based fingerprint.
13. The method of claim 12 , wherein querying the at least one reference database using the color-based fingerprint comprises determining that a similarity between the color-based fingerprint and the reference fingerprints satisfies a predetermined threshold associated with vector distance measurement.
14. The method of claim 1 , wherein the at least two patches in each region of the plurality of regions include a patch of left-right features in the region and a patch of top-bottom features in the region.
15. The method of claim 1 , wherein the plurality of regions is defined by a 4×3 grid, and wherein selecting the plurality of patches in the frame comprises selecting a predetermined number of patches in the frame.
16. A computer comprising:
one or more hardware processors; and
memory including instructions that, when executed by the one or more processors, cause the computer to perform operations comprising:
determining a first fingerprint for unknown video content, the first fingerprint being a color-based fingerprint based on a set of colors in a frame of the unknown video content;
determining a second fingerprint for the unknown video content, the second fingerprint being a patch-based fingerprint based on a plurality of patch-based values, wherein determining the second fingerprint comprises:
dividing the frame of the unknown video content into a plurality of regions;
selecting a plurality of patches in the frame, wherein selecting the plurality of patches in the frame comprises selecting at least two patches in each region of the plurality of regions; and
deriving a patch-based value for each respective patch of the plurality of patches to obtain the plurality of patch-based values;
initiating a query of at least one reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried; and
responsive to the query, initiating a query of the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint, wherein the unknown video content is identified using the at least one query fingerprint.
17. The computer of claim 16 , wherein the computer is a media playback device including the at least one reference database, the media playback device configured to receive the unknown video content and query the at least one reference database and the candidate group of fingerprints to identify the at least one query fingerprint.
18. The computer of claim 16 , wherein the at least one reference database is queried with the second fingerprint to obtain the candidate group of fingerprints, and the candidate group of fingerprints is then queried with the first fingerprint to identify the at least one query fingerprint.
19. A non-transitory computer-readable storage medium embodying instructions which, when executed by a machine, cause the machine to perform operations comprising:
determining a first fingerprint for unknown video content, the first fingerprint being a color-based fingerprint based on a set of colors in a frame of the unknown video content;
determining a second fingerprint for the unknown video content, the second fingerprint being a patch-based fingerprint based on a plurality of patch-based values, wherein determining the second fingerprint comprises:
dividing the frame of the unknown video content into a plurality of regions;
selecting a plurality of patches in the frame, wherein selecting the plurality of patches in the frame comprises selecting at least two patches in each region of the plurality of regions; and
deriving a patch-based value for each respective patch of the plurality of patches to obtain the plurality of patch-based values;
initiating a query of at least one reference database of reference fingerprints using one of the first fingerprint to obtain a candidate group of fingerprints or the second fingerprint to obtain the candidate group of fingerprints, the candidate group of fingerprints being a subset of the reference fingerprints queried; and
responsive to the query, initiating a query of the candidate group of fingerprints using the other of the first fingerprint and the second fingerprint to identify at least one query fingerprint, wherein the unknown video content is identified using the at least one query fingerprint.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the machine is a media playback device, further comprising instructions to receive the unknown video content at the media playback device including a hardware processor, and query the at least one reference database and the candidate group of fingerprints using the hardware processor.