Methods of and units for motion or depth estimation and image processing apparatus provided with such motion estimation unit
View Patent ↗In the method of motion estimation a tree ( 120 ) of segments ( 102–114 ) of an image ( 100 ) is generated by performing a hierarchical segmentation. The tree ( 120 ) of segments is analyzed to control the generation of a set ( 118 ) of candidate motion vectors of a segment ( 104 ). For the motion vectors of the set ( 118 ) match penalties are calculated. Finally, a particular motion vector ( 116 ) is selected from the set ( 118 ) of candidate motion vectors on the basis of match penalties. In the method of depth estimation depth data is calculated on the basis of a motion vector and the rules of parallax.
1. A method of motion estimation of segments ( 102 – 114 ) of an image ( 100 ), wherein the segments satisfy a homogeneity criterion, comprising the steps of:
generating a set ( 118 ) of candidate motion vectors of a particular segment ( 104 );
computing for each candidate motion vector a segment-based match penalty; and
selecting a particular motion vector ( 116 ) from the set ( 118 ) of candidate motion vectors on the basis of a respective match penalty,
characterized in further comprising the steps of:
generating a tree ( 120 ) of the segments ( 102 – 114 ) of the image ( 100 ) by performing a hierarchical segmentation, wherein the level of a segment within the tree is based on the segment's homogeneity criterion satisfying a known threshold; and
analyzing the tree ( 120 ) of the segments ( 102 – 114 ) to control the generation of the set ( 118 ) of candidate motion vectors of the particular segment ( 104 ).
2. A method of motion estimation as claimed in claim 1 , characterized in that motion vectors of the segments ( 102 – 114 ) of the tree ( 120 ) are estimated by processing the tree ( 120 ) of the segments ( 102 – 114 ) recursively in a top-down direction.
3. A method of motion estimation as claimed in claim 2 , wherein if a match penalty of the particular motion vector ( 116 ) is unsatisfactory, then motion vectors are estimated of segments ( 102 – 114 ) that correspond to children of a node of the tree ( 120 ) that corresponds to the particular segment ( 104 ).
4. A method of motion estimation as claimed in claim 1 , characterized in that the motion vectors of the segments ( 102 – 114 ) of the tree ( 120 ) are estimated by processing the tree ( 120 ) of the segments ( 102 – 114 ) in a bottom-up direction.
5. A method of motion estimation as claimed in claim 4 , characterized in that the step of generating the set ( 118 ) of candidate motion vectors of the particular segment ( 104 ) comprises the following sub-steps of:
generating an initial list of candidate motion vectors comprising motion vectors of neighboring segments ( 102 – 114 );
ordering the initial list of candidate motion vectors according to positions of the corresponding nodes ( 122 – 146 ) within the tree ( 120 ); and
restricting the initial list of candidate motion vectors to candidate motion vectors with a relatively high order, resulting in the set ( 118 ) of candidate motion vectors.
6. A motion estimator unit ( 300 , 301 ) for motion estimation of segments ( 102 – 114 ) of an image, wherein the segments satisfy a homogeneity criterion, comprising:
a first generating means ( 310 ) for generating a set ( 118 ) of candidate motion vectors of a particular segment ( 104 );
a computing means ( 306 ) for computing for each candidate motion vector a segment-based match penalty; and
a selecting means ( 308 ) for selecting a particular motion vector ( 116 ) from the set ( 118 ) of candidate motion vectors on the basis of respective match penalties,
characterized in further comprising:
a segmentation means ( 302 ) for generating a tree ( 120 ) of the segments ( 102 – 114 ) of the image, designed to perform a hierarchical segmentation, wherein the level of a segment within the tree is based on the segment's homogeneity criterion satisfying a known threshold; and
an analyzing means ( 304 ) for analyzing the tree ( 120 ) of the segments ( 102 – 114 ) to control the first generating means ( 310 ).
7. A motion estimator unit ( 300 ) as claimed in claim 6 , characterized in being designed to estimate motion vectors of the segments ( 102 – 114 ) of the tree ( 120 ) by processing the tree ( 120 ) of the segments ( 102 – 114 ) recursively in a top-down direction.
8. A motion estimator unit ( 300 ) as claimed in claim 7 , characterized in being designed to estimate motion vectors of segments ( 102 – 114 ), corresponding to children of a node of the tree ( 120 ) that corresponds to the particular segment ( 104 ), if a match penalty of the particular motion vector ( 116 ) is unsatisfactory.
9. A motion estimator unit ( 301 ) as claimed in claim 6 , characterized in being designed to estimate motion vectors of the segments ( 102 – 114 ) of the tree ( 120 ) by processing the tree ( 120 ) of the segments ( 102 – 114 ) in a bottom-up direction.
10. A motion estimator unit ( 301 ) as claimed in claim 9 , characterized in that the first generating means ( 310 ) for generating the set ( 118 ) of candidate motion vectors of the particular segment ( 104 ) comprises:
a second generating means ( 303 ) for generating an initial list of candidate motion vectors comprising motion vectors of neighboring segments ( 102 – 114 );
an ordering means ( 305 ) for ordering the initial list of candidate motion vectors according to positions of the corresponding nodes ( 122 – 146 ) within the tree ( 120 ); and
a filter ( 307 ) to restrict the initial list of candidate motion vectors to candidate motion vectors with a relatively high order.
11. An image processing apparatus ( 500 ) comprising:
a motion estimator unit ( 300 , 301 ) for motion estimation of segments ( 102 – 114 ) of an image ( 100 ), wherein the segments satisfy a homogeneity criterion, comprising:
a generator ( 310 ) for generating a set ( 118 ) of candidate motion vectors of a particular segment ( 104 );
a computing means ( 306 ) for computing for each candidate motion vector a segment-based match penalty; and
a selecting means ( 308 ) for selecting a particular motion vector ( 116 ) from the set ( 118 ) of candidate motion vectors on the basis of respective match penalties; and
a motion compensated image processing unit ( 502 ), characterized in that the motion estimator unit ( 300 , 301 ) comprises:
a segmentation means ( 302 ) for generating a tree ( 120 ) of the segments ( 102 – 114 ) of the image ( 100 ), designed to perform a hierarchical segmentation method, wherein the level of a segment within the tree is based on the segment's homogeneity criterion satisfying a known threshold; and
an analyzing means ( 304 ) for analyzing the tree ( 120 ) of the segments ( 102 – 114 ) to control the generator ( 310 ).