Video fingerprinting
A query fingerprint of a set of frames of video content captured at a client device may be generated. Multiple patches of the set of frames of video content may be selected and a value calculated for each of the selected multiple patches. The value for each patch may be indicated as a single bit along with an additional 1-bit value to indicate whether the patch value is weak. A database of known reference fingerprints may be queried using the generated query fingerprint. Matches between the query fingerprint and the reference fingerprints may be identified. Weak bits may be given reduced weight in identifying the match of fingerprints. Based on the matches, an identifier for the video content may be returned to the client device. The client device may use the received identifier to access the supplemental content.
1. A method comprising:
accessing a query fingerprint that includes a first component based on a plurality of patches of a first frame of a set of frames of video content captured at a client device, the first component composed of bits of data, the bits of data representing at least two patches of the plurality of patches;
querying a database of reference fingerprints associated with frames of known video content, each reference fingerprint having a plurality of components, each component of the plurality of components of each reference fingerprint based on a corresponding frame of known video content;
determining that the query fingerprint matches at least one of the reference fingerprints, a number of frames in the set of frames of video content on which the query fingerprint is based being fewer than a number of frames used to generate the at least one matching reference fingerprint;
identifying, by a processor of a machine, the video content captured at the client device based on the determination that the query fingerprint matches at least one of the reference fingerprints; and
sending an identifier to the client device, the identifier corresponding to the identified video content.
2. The method of claim 1 , wherein:
the querying of the database of reference fingerprints identifies a set of candidate fingerprints; and
the determining that the query fingerprint matches at least one of the reference fingerprints determines that the query fingerprint matches at least one of the candidate fingerprints.
3. The method of claim 2 , wherein:
the query fingerprint includes a second component based on a plurality of patches of a second frame of video content captured at the client device; and
the determining that the query fingerprint matches at least one of the candidate fingerprints determines that the first component of the query fingerprint matches a first component of the at least one matching candidate fingerprint and the second component of the query fingerprint matches a second component of the at least one matching candidate fingerprint.
4. The method of claim 3 , wherein the identifying of the set of candidate fingerprints does not compare the second component of the query fingerprint to any of the reference fingerprints.
5. The method of claim 1 , wherein the bits of data represent at least five patches of the plurality of patches.
6. The method of claim 1 , wherein each bit of the bits of data corresponds to a different patch of the plurality of patches.
7. The method of claim 1 , wherein querying a database of reference fingerprints includes querying an index of quantized patch values calculated for the reference fingerprints.
8. The method of claim 1 , wherein determining the query fingerprint matches at least one of the reference fingerprints includes determining that a similarity between the query fingerprint and the at least one of the reference fingerprint satisfies a predetermined threshold associated with a Tanimoto distance measurement.
9. The method of claim 1 , wherein determining the query fingerprint matches at least one of the reference fingerprints includes determining that a similarity between the query fingerprint and the at least one of the reference fingerprints satisfies a predetermined threshold associated with a Manhattan distance measurement.
10. The method of claim 1 , wherein the identifier sent to the client device is associated with metadata to be presented along with the video content via the client device.
11. The method of claim 1 , wherein the identifier sent to the client device is associated with event-based information to be presented along with the video content via the client device.
12. The method of claim 1 wherein the number of frames in the set of frames of video content on which the query fingerprint is based and the number of frames used to generate the at least one matching reference fingerprint correspond to an equal amount of time in the video content on which the query fingerprint is based and in the identified video content.
13. A system comprising:
a memory having instructions embodied thereon; and
one or more processors configured by the instructions to perform operations comprising:
accessing a query fingerprint that includes a first component based on a plurality of patches of a first frame of a set of frames of video content captured at a client device, the first component composed of bits of data, the bits of data representing at least two patches of the plurality of patches;
querying a database of reference fingerprints associated with frames of known video content, each reference fingerprint having a plurality of components, each component of the plurality of components of each reference fingerprint based on a corresponding frame of known video content;
determining that the query fingerprint matches at least one of the reference fingerprints, a number of frames in the set of frames of video content on which the query fingerprint is based being fewer than a number of frames used to generate the at least one matching reference fingerprint;
identifying the video content captured at the client device based on the determination that the query fingerprint matches at least one of the reference fingerprints; and
sending an identifier to the client device, the identifier corresponding to the identified video content.
14. The system of claim 13 , wherein:
the querying of the database of reference fingerprints identifies a set of candidate fingerprints; and
the determining that the query fingerprint matches at least one of the reference fingerprints determines that the query fingerprint matches at least one of the candidate fingerprints.
15. The system of claim 14 , wherein:
the query fingerprint includes a second component based on a plurality of patches of a second frame of video content captured at the client device; and
the determining that the query fingerprint matches at least one of the candidate fingerprints determines that the first component of the query fingerprint matches a first component of the at least one matching candidate fingerprint and the second component of the query fingerprint matches a second component of the at least one matching candidate fingerprint.
16. The system of claim 15 , wherein the identifying of the set of candidate fingerprints does not compare the second component of the query fingerprint to any of the reference fingerprints.
17. The system of claim 13 , wherein the bits of data represent at least five patches of the plurality of patches.
18. The system of claim 13 , wherein each bit of the bits of data corresponds to a different patch of the plurality of patches.
19. The system of claim 13 wherein the number of frames in the set of frames of video content on which the query fingerprint is based and the number of frames used to generate the at least one matching reference fingerprint correspond to an equal amount of time in the video content on which the query fingerprint is based and in the identified video content.
20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
accessing a query fingerprint that includes a first component based on a plurality of patches of a first frame of a set of frames of video content captured at a client device, the first component composed of bits of data, the bits of data representing at least two patches of the plurality of patches;
querying a database of reference fingerprints associated with frames of known video content, each reference fingerprint having a plurality of components, each component of the plurality of components of each reference fingerprint based on a corresponding frame of known video content;
determining that the query fingerprint matches at least one of the reference fingerprints, a number of frames in the set of frames of video content on which the query fingerprint is based being fewer than a number of frames used to generate the at least one matching reference fingerprint;
identifying the video content captured at the client device based on the determination that the query fingerprint matches at least one of the reference fingerprints; and
sending an identifier to the client device, the identifier corresponding to the identified video content.