Search and identification of video content
View Patent ↗Methods and systems for searching for and retrieving video content via fingerprint matching are described. In some example embodiments, the methods and systems access an index of reference fingerprints, such as quantized values associated with multiple values calculated from patches of a frame, that are associated with sequential frames of known video content. The methods and systems may receive a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device, access the index, compare the group of query fingerprints to reference fingerprints of the index, determine that the group of query fingerprints match the reference fingerprints of the index, and identify the video content presented at the client device based on the determined match.
1. A method, comprising:
receiving a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query fingerprints being sequential;
accessing, by a processor of a machine, an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more first regions of the corresponding frame of known video content and an associated second value calculated from one or more second regions of the corresponding frame of known video content;
identifying two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints;
selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to the second values associated with the identified match candidates; and
identifying the video content presented at the client device based on the selected match candidates.
2. The method of claim 1 , further comprising:
accessing, using an identifier of the identified video content, supplemental content from a database; and
causing to be presented, at the client device, the supplemental content.
3. The method of claim 2 , wherein the supplemental content is one or more of a listing, guide information, metadata associated with the identified video content, and information associated with the identified video content.
4. The method of claim 1 , wherein the one or more first regions and the one or more second regions are selected from a plurality of regions based on size.
5. The method of claim 1 , wherein the one or more first regions and the one or more second regions are selected from a plurality of regions based on the presence of Haar-like features in the one or more first regions and the one or more second regions.
6. The method of claim 1 , wherein the query fingerprints and the reference fingerprints are generated using integral image techniques.
7. The method of claim 1 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of quantized values for each of the sequential frames of known video content.
8. The method of claim 1 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of 32-bit vectors associated with one or more different patches of a frame of video content.
9. The method of claim 1 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with frames of video content, an integer being calculated by hashing three to five different Haar-like features of the frame of video content.
10. The method of claim 1 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with a frame of video content, an integer being calculated by hashing a first four Haar-like features of the frame of video content.
11. The method of claim 1 , wherein:
the accessing of the index of reference fingerprints includes accessing an index of reference fingerprints associated with frames of video content at a first frame rate;
the receiving of the search query includes a receiving a group of query fingerprints presented at a client device having a second frame rate faster than the first frame rate; and
the comparing of the group of query fingerprints to the reference fingerprints of the index includes comparing a portion of query fingerprints having the second frame rate to the reference fingerprints associated with frames of video content at the first frame rate.
12. A system comprising:
a memory having instructions embodied thereon;
a processor coupled to the memory and configured to execute the instructions to perform operations comprising:
receiving a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query fingerprints being sequential;
accessing, by a processor of a machine, an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more first regions of the corresponding frame of known video content and an associated second value calculated from one or more second regions of the corresponding frame of known video content;
identifying two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints;
selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to the second values associated with the identified match candidates; and
identifying the video content presented at the client device based on the selected match candidates.
13. The system of claim 12 , wherein the operations further comprise:
accessing, using an identifier of the identified video content, supplemental content from a database; and
causing to be presented, at the client device, the supplemental content.
14. The system of claim 13 , wherein the supplemental content is one or more of a listing, guide information, metadata associated with the identified video content, and information associated with the identified video content.
15. The system of claim 12 , wherein the one or more first regions and the one or more second regions are selected from a plurality of regions based on size.
16. The system of claim 12 , wherein the one or more first regions and the one or more second regions are selected from a plurality of regions based on the presence of Haar-like features in the one or more first regions and the one or more second regions.
17. The system of claim 12 , wherein the query fingerprints and the reference fingerprints are generated using integral image techniques.
18. The system of claim 12 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of quantized values for each of the sequential frames of known video content.
19. The system of claim 12 , wherein the accessing of the index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of 32-bit vectors associated with one or more different patches of a frame of video content.
20. A non-transitory machine-readable medium having instructions embodied thereon which, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
receiving a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query fingerprints being sequential;
accessing, by a processor of a machine, an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more first regions of the corresponding frame of known video content and an associated second value calculated from one or more second regions of the corresponding frame of known video content;
identifying two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints;
selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to the second values associated with the identified match candidates; and
identifying the video content presented at the client device based on the selected match candidates.