Search and identification of video content
View Patent ↗Methods and systems for searching for and retrieving video content via fingerprint matching are described. In some example embodiments, the methods and systems access an index of reference fingerprints, such as quantized values associated with multiple values calculated from patches of a frame, that are associated with sequential frames of known video content. The methods and systems may receive a search query that includes a group of query fingerprints associated with sequential frames of video content presented at a client device, access the index, compare the group of query fingerprints to reference fingerprints of the index, determine that the group of query fingerprints match the reference fingerprints of the index, and identify the video content presented at the client device based on the determined match.
1. A method, comprising:
receiving a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query fingerprints being sequential;
accessing an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more large regions of the corresponding frame of known video content, each reference fingerprint of the reference fingerprints having an associated second value calculated from one or more small regions of the corresponding frame of known video content;
comparing the group of query fingerprints to reference fingerprints of the index;
identifying two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints;
selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to the second values associated with the identified match candidates; and
identifying the video content presented at the client device based on the selected match candidates.
2. The method of claim 1 , wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of quantized values for each of the sequential frames of known video content.
3. The method of claim 1 , wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of 32 bit vectors associated with one or more different patches of a frame of video content.
4. The method of claim 1 , wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with frames of video content, an integer being calculated by hashing three to five different Haar-like features of the frame of video content.
5. The method of claim 1 , wherein accessing an index of reference fingerprints that are associated with sequential frames of known video content includes accessing an index of integers associated with a frame of video content, an integer being calculated by hashing a first four Haar-like features of the frame of video content.
6. The method of claim 1 , wherein accessing an index of reference fingerprints includes accessing an index of reference fingerprints associated with frames of video content at a first frame rate;
wherein receiving a search query includes a receiving a group of query fingerprints presented at a client device having a second frame rate faster than the first frame rate; and
wherein comparing the group of query fingerprints to the reference fingerprints of the index includes comparing a portion of query fingerprints having the second frame rate to the reference fingerprints associated with frames of video content at the first frame rate.
7. A system, comprising:
a processor-implemented index module that is configured to:
receive a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query finger prints being sequential, and
query an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more large regions of the corresponding frame of known video content, each reference fingerprint of the reference fingerprints having an associated second value calculated from one or more small regions of the corresponding frame of known video content;
a fingerprint match module that is configured to:
identify two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints,
select one or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints, and
an identification module that is configured to identify the video content presented at the client device based on the selected match candidates.
8. The system of claim 7 , wherein the index module is configured to query an index of quantized values for each of the sequential frames of known video content.
9. The system of claim 7 , wherein the index module is configured to query an index of 32 bit vectors associated with 32 different patches of a frame of video content.
10. The system of claim 7 , wherein the index module is configured to query an index of integers associated with frames of video content, an integer being calculated by hashing a first four Haar-like features of the frame of video content.
11. The system of claim 7 , wherein the index module is configured to query an index of integers associated with frames of video content, an integer being calculated by hashing three to five different Haar-like features of the frame of video content.
12. A non-transitory computer-readable storage medium whose contents, when executed by a computing system, cause the computing system to perform operations, comprising:
receiving a search query that includes a group of query fingerprints, each fingerprint of the group of query fingerprints being associated with a frame of video content presented at a client device, the frames of video content associated with the group of query fingerprints being sequential;
accessing an index of reference fingerprints, each reference fingerprint of the reference fingerprints corresponding to a sequential frame of known video content, each reference fingerprint of the reference fingerprints having an associated first value calculated from one or more large regions of the corresponding frame of known video content, each reference fingerprint of the reference fingerprints having an associated second value calculated from one or more small regions of the corresponding frame of known video content;
comparing the group of query fingerprints to reference fingerprints of the index;
identifying two or more match candidates based on a comparison of the group of query fingerprints to the first values associated with the reference fingerprints;
selecting one or more of the identified match candidates based on a comparison of the group of query fingerprints to the second values associated with the identified match candidates; and
identifying the video content presented at the client device based on the selected match candidates.