Method and apparatus for multi-dimensional content search and video identification
A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.
1. A computer-implemented method for storing information associated with videos in a reference database using hash values as traversal indexes, the computer-implemented method comprising:
for each of multiple video sequences:
obtaining, by a processor, data associated with the video sequence,
determining, by the processor, a multi-dimensional vector signature of a region of a frame of the video sequence,
determining, by the processor, a hash value for the frame of the video sequence based on the multi-dimensional vector signature, wherein the hash value comprises a bit vector, and wherein individual bit values of the bit vector represent respective dimensions of the multi-dimensional vector signature, and
storing the data associated with the video sequence in association with the hash value.
2. The computer-implemented method of claim 1 , wherein the region comprises multiple sectors, and wherein the multi-dimensional vector signature represents each sector.
3. The computer-implemented method of claim 2 , wherein determining the multi-dimensional vector signature comprises comparing features within each sector to a threshold value to generate a value for the sector.
4. The computer-implemented method of claim 2 , wherein the region is a rectangular grid.
5. The computer-implemented method of claim 1 , wherein:
storing the data associated with the video sequence comprises storing the data associated with the video sequence at a leaf node of a plurality of leaf nodes, and
the leaf nodes of the plurality of leaf nodes are addressable using the hash values.
6. The computer-implemented method of claim 1 , wherein the multi-dimensional vector signature is a shape signature of a keypoint.
7. The computer-implemented method of claim 1 , wherein the multi-dimensional vector signature is determined based on a weighted combination of multiple features of the region, and wherein the multiple features comprise color similarity.
8. The computer-implemented method of claim 1 , further comprising, for at least one of the multiple video sequences: selecting the frame based on a comparison of the frame with other frames of the video sequence.
9. The computer-implemented method of claim 1 , wherein the associated data comprises a video name and an indication of a frame number of the frame.
10. A multi-dimensional content search system comprising:
one or more processors; and
a computer-readable medium having stored therein instructions that are executable by the one or more processors to cause the multi-dimensional content search system to perform functions comprising:
for each of multiple video sequences:
obtaining data associated with the video sequence,
determining a multi-dimensional vector signature of a region of a frame of the video sequence,
determining a hash value for the frame of the video sequence based on the multi-dimensional vector signature, wherein the hash value comprises a bit vector, and wherein individual bit values of the bit vector represent respective dimensions of the multi-dimensional vector signature, and
storing the data associated with the video sequence in association with the hash value.
11. The multi-dimensional content search system of claim 10 , wherein the region comprises multiple sectors, and wherein the multi-dimensional vector signature represents each sector.
12. The multi-dimensional content search system of claim 11 , wherein determining the multi-dimensional vector signature comprises comparing features within each sector to a threshold value to generate a value for the sector.
13. The multi-dimensional content search system of claim 11 , wherein the region is a rectangular grid.
14. The multi-dimensional content search system of claim 10 , wherein the functions further comprise storing the multi-dimensional vector signature in association with the hash value.
15. The multi-dimensional content search system of claim 10 , wherein the multi-dimensional vector signature is determined based on a weighted combination of multiple features of the region, and wherein the multiple features comprise color similarity.
16. The multi-dimensional content search system of claim 10 , wherein the functions further comprise, for at least one of the multiple video sequences: selecting the frame based on a comparison of the frame with other frames of the video sequence.
17. The multi-dimensional content search system of claim 10 , wherein the associated data comprises a video name and an indication of a frame number of the frame.
18. A non-transitory computer-readable medium having stored therein instructions that are executable by one or more processors to cause a client media device to perform functions comprising:
for each of multiple video sequences:
obtaining data associated with the video sequence,
determining a multi-dimensional vector signature of a region of a frame of the video sequence,
determining a hash value for the frame of the video sequence based on the multi-dimensional vector signature, wherein the hash value comprises a bit vector, and wherein individual bit values of the bit vector represent respective dimensions of the multi-dimensional vector signature, and
storing the data associated with the video sequence in association with the hash value.
19. The non-transitory computer-readable medium of claim 18 , wherein the region comprises multiple sectors, and wherein the multi-dimensional vector signature represents each sector.
20. The non-transitory computer-readable medium of claim 19 , wherein determining the multi-dimensional vector signature comprises comparing features within each sector to a threshold value to generate a value for the sector.