Method and apparatus for multi-dimensional content search and video identification
A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.
1. A computer-implemented method comprising:
obtaining, by a processor, a first query index and a second query index that are derived from different respective features of a frame of a query video;
determining, by the processor, that a distance measure between the first query index and a candidate database index of a reference database satisfies a threshold condition, wherein the candidate database index corresponds to a frame of an original video;
determining, by the processor, a correlation score for the frame of the query video and the frame of the original video based on a comparison of the second query index and an additional candidate database index corresponding to the frame of the original video;
based at least on the correlation score, determining, by the processor, a video sequence likelihood indicative of a confidence of match between the query video and the original video; and
based on the video sequence likelihood, providing, by the processor, a results list that includes a name of the original video.
2. The computer-implemented method of claim 1 , wherein the second query index corresponds to a texture signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a texture signature of a region of the frame of the original video.
3. The computer-implemented method of claim 1 , wherein the second query index corresponds to a motion signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a motion signature of a region of the frame of the original video.
4. The computer-implemented method of claim 1 , wherein the second query index corresponds to an object signature that is indicative of a location of an object within the frame of the query video, and wherein the additional candidate database index corresponds to an object signature that is indicative of a location of an object within the frame of the original video.
5. The computer-implemented method of claim 1 , wherein the additional candidate database index and the name of the original video are stored at a leaf node corresponding to the candidate database index, and wherein the leaf node is addressable by the candidate database index.
6. The computer-implemented method of claim 1 , wherein the first query index corresponds to a global signature of a region of interest of the query video, wherein the region of interest comprises multiple sectors, and wherein the global signature represents features of respective sectors of the multiple sectors.
7. The computer-implemented method of claim 6 , wherein the region of interest is a rectangular grid.
8. The computer-implemented method of claim 1 , wherein the first query index corresponds to local features of an area around a keypoint of the frame of the query video.
9. The computer-implemented method of claim 1 , further comprising using the first query index as a traversal index so as to identify candidate database indexes of the reference database that are within a range of the first query index, wherein the candidate database index is one of the candidate database indexes.
10. A multi-dimensional content search system comprising:
one or more processors; and
a computer-readable medium having stored therein instructions that are executable by the one or more computers to cause the multi-dimensional content search system to perform functions comprising:
obtaining a first query index and a second query index that are derived from different respective features of a frame of a query video,
determining that a distance measure between the first query index and a candidate database index of a reference database satisfies a threshold condition, wherein the candidate database index corresponds to a frame of an original video,
determining a correlation score for the frame of the query video and the frame of the original video based on a comparison of the second query index and an additional candidate database index corresponding to the frame of the original video,
based at least on the correlation score, determining a video sequence likelihood indicative of a confidence of match between the query video and the original video, and
based on the video sequence likelihood, providing a results list that includes a name of the original video.
11. The multi-dimensional content search system of claim 10 , wherein the second query index corresponds to a texture signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a texture signature of a region of the frame of the original video.
12. The multi-dimensional content search system of claim 10 , wherein the second query index corresponds to a motion signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a motion signature of a region of the frame of the original video.
13. The multi-dimensional content search system of claim 10 , wherein the second query index corresponds to an object signature that is indicative of a location of an object within the frame of the query video, and wherein the additional candidate database index corresponds to an object signature that is indicative of a location of an object within the frame of the original video.
14. The multi-dimensional content search system of claim 10 , wherein the additional candidate database index and the name of the original video are stored at a leaf node corresponding to the candidate database index, and wherein the leaf node is addressable by the candidate database index.
15. The multi-dimensional content search system of claim 10 , wherein the first query index corresponds to a global signature of a region of interest of the query video, wherein the region of interest comprises multiple sectors, and wherein the global signature represents features of respective sectors of the multiple sectors.
16. The multi-dimensional content search system of claim 10 , wherein the first query index corresponds to local features of an area around a keypoint of the frame of the query video.
17. A non-transitory computer-readable medium having stored therein instructions that are executable by one or more processors to cause a computing device to perform functions comprising:
obtaining a first query index and a second query index that are derived from different respective features of a frame of a query video;
determining that a distance measure between the first query index and a candidate database index of a reference database satisfies a threshold condition, wherein the candidate database index corresponds to a frame of an original video;
determining a correlation score for the frame of the query video and the frame of the original video based on a comparison of the second query index and an additional candidate database index corresponding to the frame of the original video;
based at least on the correlation score, determining a video sequence likelihood indicative of a confidence of match between the query video and the original video; and
based on the video sequence likelihood, providing a results list that includes a name of the original video.
18. The non-transitory computer-readable medium of claim 17 , wherein the second query index corresponds to a texture signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a texture signature of a region of the frame of the original video.
19. The non-transitory computer-readable medium of claim 17 , wherein the second query index corresponds to a motion signature of a region of the frame of the query video, and wherein the additional candidate database index corresponds to a motion signature of a region of the frame of the original video.
20. The non-transitory computer-readable medium of claim 17 , wherein the second query index corresponds to an object signature that is indicative of a location of an object within the frame of the query video, and wherein the additional candidate database index corresponds to an object signature that is indicative of a location of an object within the frame of the original video.