Method and apparatus for multi-dimensional content search and video identification
A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.
1. A method of generating a likelihood score of a query video sequence matching an original video sequence, the method comprising:
generating a similarity measure between frames of a query video sequence and an original video sequence based on frame similarity scores that exceed a threshold, wherein the original video is an entry in a video database;
generating a time correlation between features extracted from frames having different frame numbers in the query video sequence and in the original video sequence;
generating a correlation score between the original video sequence and the query video sequence by using a combination of the similarity measure and the time correlation to identify the likelihood score of the query video sequence matching the original video sequence; and
transmitting the query video sequence matching the original video sequence to a database of traversal indexes based on correlation scores that exceed a threshold.
2. The method of claim 1 , wherein signature pairs selected from the original video and the query video that match determine a starting frame for the original video sequence.
3. The method of claim 1 , wherein the video database is stored on a portable device.
4. The method of claim 1 further comprising:
determining the threshold based on a length of a matching original video sequence.
5. The method of claim 1 further comprising:
determining the threshold from information extracted from the original video sequence that matches the query video sequence.
6. The method of claim 1 further comprising:
determining the threshold experimentally for a sequence window length of a matching original video sequence.
7. The method of claim 1 , wherein frames of the query video sequence are processed to generate signatures and for each of the signatures generate a traversal index to provide a direct hash address to a leaf node storing information regarding a region of interest in a frame of the query video sequence.
8. The method of claim 1 , wherein the query video sequence comprises a sequence of signatures and traversal indexes used to search for likely matching frames of the original video sequence.
9. The method of claim 1 , wherein the query video sequence comprises a sequence of signatures with an associated traversal index, the traversal index is generated from a combination of two subsequent signatures selected from two subsequent frames.
10. A method of generating a likelihood score for matching frames of a query video and an original video, the method comprising:
generating a similarity measure between a query video sequence and an original video selected from a video database, the generated similarity measure based on individual frame similarity scores that exceed a first threshold, wherein the original video is an entry in a video database;
generating a time correlation using different starting frame numbers for the query video sequence and the original video sequence to find a starting frame and an original video sequence with highest correlation to the query video sequence;
generating a correlation score between the original video sequence and the query video sequence by using a combination of the similarity measure and the time correlation to identify the likelihood score of the query video sequence matching the original video sequence; and
transmitting the query video sequence matching the original video sequence to a database of traversal indexes based on correlation scores that exceed a threshold.
11. The method of claim 10 further comprising:
performing, for a selected query video signature, a search in a video database to identity a candidate list of nearest video frames from video sequences stored in the video database; and selecting from the candidate list the starting frame and the original video sequence that exceeds a second threshold.
12. The method of claim 10 , wherein frames of the query video sequence are processed to generate signatures and for each of the signatures generate a traversal index to provide a direct hash address to a leaf node storing information regarding a region of interest in a frame of the query video sequence.
13. The method of claim 10 further comprising:
determining the threshold from information extracted from the original video sequence that matches the query video sequence.
14. The method of claim 10 , wherein the video database is stored on a portable device.
15. The method of claim 10 , wherein the correlation score is generated by a portable device.
16. A processing system for generating a likelihood score of a query video sequence matching an original video sequence, the processing system comprising:
a video database located on a portable device that stores a plurality of original videos; and
a processor configured to:
generate a similarity measure between frames of a query video sequence and an original video sequence, selected from the video database, based on frame similarity scores that exceed a threshold;
generate a time correlation between features extracted from frames having different frame numbers in the query video sequence and in the original video sequence;
generate a correlation score between the original video sequence and the query video sequence by using a combination of the similarity measure and the time correlation to identity the likelihood score of the query video sequence matching the original video sequence; and
transmit the query video sequence matching the original video sequence to a database of traversal indexes based on correlation scores that exceed a threshold.
17. The processing system of claim 16 , wherein the processor is further configured to determine the threshold from information extracted from the original video sequence that matches the query video sequence.
18. The processing system of claim 16 , wherein the processor is further configured to process frames of the query video sequence to generate signatures, and to generate a traversal index from each of the generated signatures to provide a direct hash address to a leaf node storing information regarding a region of interest in a frame of the query video sequence.
19. The processing system of claim 16 , wherein the processor is further configured to search for likely matching frames of a video sequence stored in the video database using a sequence of signatures and traversal indexes generated from frames of the query video sequence, and wherein the query video sequence comprises a sequence of signatures and traversal indexes used to search for likely matching frames of the original video sequence.
20. The processing system of claim 16 , wherein the processor is further configured to provide video fingerprinting and search operations on video sequences selected from the video database.