Automatic extraction method of the structure of a video sequence
View Patent ↗The invention relates to a method intended to automatically create a description of a video sequence—i.e. its table of contents—by analyzing the video sequence. First is a temporal segmentation of the video shots of the sequence, using camera motion parameters. This segmentation uses a similarity criterion involving, for sub-entities of each shot, the level of homogeneity of these sub-entities on the motion parameters of the camera used to acquire the original images and generate the bitstream constituting the processed sequence.
1. A method for an automatic extraction of the structure of a video sequence that corresponds to successive frames, comprising the following steps:
(1) a shot detection step, provided for detecting the boundaries between consecutive shots, a shot being a set of contiguous frames without editing effects;
(2) a partitioning step, provided for splitting each shot into sub-entities, called micro-segments comprising a plurality of temporal sub-entities;
(3) a clustering step, provided for creating a final hierarchical structure of the processed video sequence,
wherein the a motion histogram on which a homogeneity of a micro-segment is computed includes a plurality of bins, each bin of which shows the percentage of frames with a specific type of motion.
2. A method according to claim 1 , wherein said partitioning step uses a criterion involving the level of homogeneity on the motion parameters of the camera used to generate the processed video sequence.
3. A method according to claim 1 , wherein, if the bins of the histogram are not equal to either 1 or 0, i.e. present intermediate values indicating that a micro-segment is not perfectly homogeneous, a distance between two micro-segments is computed, based on the homogeneity of the micro-segments union, said homogeneity being itself deduced from the histogram of a micro-segment and the different motion types, the homogeneity of a shot being equal to the homogeneity of its micro-segments weighted by the length of each of them, a fusion between any pair of micro-segments being decided or not according to the value of the homogeneity of the shot with respect to a predefined threshold T(H) and assuming that the selected micro-segments have already been merged, and such a possible merging process between micro-segments ending when there is no further pair of neighboring micro-segments that can be merged.
4. A method for an automatic extraction of the structure of a video sequence that corresponds to successive frames, comprising the following steps:
(1) a shot detection step, provided for detecting the boundaries between consecutive shots, a shot being a set of contiguous frames without editing effects;
(2) a partitioning step, provided for splitting each shot into sub-entities, called micro-segments;
(3) a clustering step, provided for creating a final hierarchical structure of the processed video sequence;
wherein said shot detection step uses a similarity criterion based on a computation of the mean displaced frame difference curve and the detection of the highest peaks of said curve.
5. A method for indexing data available in the form of a video sequence that corresponds to successive frames, comprising the following segmentation steps:
(1) a structuring step, provided for sub-dividing said sequence into consecutive shots and splitting each of said shots into sub-entities called micro-segments comprising a plurality of temporal sub-entities;
(2) a clustering step, provided for creating on the basis of said segmentation a final hierarchical structure of the processed video sequence;
(3) an indexing step, provided for adding a label to each elements of said hierarchical structure;
wherein the a motion histogram on which the homogeneity of a micro-segment is computed includes a plurality of bins, each bin of which shows the percentage of frames with a specific type of motion.
6. A video indexing device including means for carrying out a method according to claim 5 .
7. An image retrieval system including:
(1) means for carrying out an indexing method according to claim 5 , for defining in a hierarchical fashion the structure of a video sequence that corresponds to successive frames, giving an indexing label to each element of the hierarchical structure thus defined, and storing said labels;
(2) means for performing on the basis of the stored labels any image retrieval using one or several frames of said image to be retrieved.
8. A method for an automatic extraction of the structure of a video sequence that corresponds to successive frames, comprising the following steps:
(1) a shot detection step, provided for detecting the boundaries between consecutive shots, a shot being a set of contiguous frames without editing effects;
(2) a partitioning step, provided for splitting each shot into sub-entities, called micro-segments comprising a plurality of temporal sub-entities, each of the micro-segments having a level of homogeneity computed on a motion histogram;
(3) a clustering step, provided for creating a final hierarchical structure of the processed video sequence;
wherein the motion histogram on which the homogeneity of a micro-segment is computed includes a plurality of bins, each bin of which shows the percentage of frames with a specific type of motion.