Method and system for creating an interpolated image using up-conversion vector with uncovering-covering detection
View Patent ↗The invention relates to a method for creating an interpolated image between a previous image and a current image in a video stream. The invention also relates to an image processing system for creating an interpolated image between a previous image and a current image in a video stream. The invention is based on the following principle: a motion estimator generates at least two vector fields, one at a temporal position smaller then 1A, and another at a temporal position greater than 1A. Then occlusion detection is done to detect covering and uncovering areas. A first vector is fetched from the first vector field and a second vector is fetched from the second vector field. If there is covering, the up-conversion vector is the second vector; if there is uncovering, the up-conversion vector is the first vector; if there is no uncovering-covering, the up-conversion may be either the first vector or the second vector. In this way, the up-conversion vector points at the correct original segment values in the previous image and the current image.
1. A method for creating an interpolated image between a previous image and a current image in a video stream, characterized by:
(a) performing motion estimations between the previous and the current image, resulting in a first vector field and a second vector field, wherein a first one of the motion estimations is performed at a temporal position smaller than ½, and a second one of the motion estimations is performed at a temporal position greater than ½;
wherein the contents of a segment of the interpolated image are determined by:
(b) performing an uncovering-covering detection by analyzing at least one of the first vector field and the second vector field;
(c) fetching a first vector from the first vector field and a second vector from the second vector field;
(d) generating an up-conversion vector, wherein:
(d1) if there is covering, the up-conversion vector is the second vector;
(d2) if there is uncovering, the up-conversion vector is the first vector;
(d3) if there is no uncovering-covering, the up-conversion vector is arbitrarily selected from the first vector and the second vector;
(e) using the up-conversion vector to fetch original segment values in the previous image and the current image, and computing the interpolated value of the segment from the original segment values;
(f) repeating steps (b) to (e) for each segment of the interpolated image.
2. A method according to claim 1 , wherein the uncovering-covering detection in step (b) is performed by:
(b1) calculating the horizontal difference between the vector in N segment positions to the left of the segment and the vector in N segment positions to the right of the segment,
(b2) comparing the horizontal difference to an occlusion threshold using the formula
mv x ( x−N, y )− mv x ( x+N, y )> T occlusion uncovering (1)
mv x ( x−N, y )− mv x ( x+N, y )<− T occlusion covering (2)
wherein N≧1, mv x (x, y) is the horizontal component of a vector at position (x,y) and T occlusion is the occlusion threshold.
3. A method according to claim 1 ,
wherein the motion estimations are performed by a 3Dimensional Recursive Search motion estimator.
4. A method according to claim 1 ,
wherein the fetching in step (c) is performed:
(c1) along a vector from the first vector field if covering is detected,
(c2) along a vector from the second vector field if uncovering is detected, and
(c3) along a zero vector is no uncovering-covering is detected.
5. A method according to claim 2 , wherein between step (c) and step (d) the following step is performed:
comparing the horizontal components of the first vector and the second vector, and if the difference between the horizontal components is smaller than the occlusion threshold, performing steps (d) and (e) as if there is no uncovering-covering.
6. A method according to claim 1 ,
wherein uncovering detection is disabled for up-conversion at temporal positions of ¼ and lower and covering detection is disabled for temporal positions of ¾ and higher.
7. Image processing system for creating an interpolated image between a previous image and a current image in a video stream, characterized in that the system comprises means for:
(a) performing motion estimations between the previous and the current image, resulting in a first vector field and a second vector field, wherein a first one of the motion estimations is performed at a temporal position smaller than ½, and a second one of the motion estimations is performed at a temporal position greater than ½;
determining the contents of a segment of the interpolated image by:
(b) performing an uncovering-covering detection by analyzing at least one of the first vector field and the second vector field;
(c) fetching a first vector from the first vector field and a second vector from the second vector field;
(d) generating an up-conversion vector, wherein:
(d1) if there is covering, the up-conversion vector is the second vector;
(d2) if there is uncovering, the up-conversion vector is the first vector;
(d3) if there is no uncovering-covering, the up-conversion vector is arbitrarily selected from the first vector and the second vector;
(e) using the up-conversion vector to fetch original segment values in the previous image and the current image, and computing the interpolated value of the segment from the original segment values;
(f) repeating steps (b) to (e) for each segment of the interpolated image.
8. Computer program product on a non-transitory computer readable medium for carrying out the method according to claim 1 .