Video processing method and device for depth extraction
View Patent ↗The present invention provides an improved method and device for generating a depth map ( 112 ) by extracting three-dimensional depth information from the movement vectors of an encoded video bit stream ( 102 ) to display two-dimensional video sequences onto three-dimensional displays. In particular, the invention performs depth extraction ( 110 ) by means of a post-processing of the movement vectors of the inter-coded macroblocks, which have been already encoded in the video bit stream, thereby significantly reducing the heavy processing requirements associated with conventional motion estimation techniques.
1. A method of processing video images, wherein the method comprises:
receiving an encoded video bit stream of a two-dimensional video application having video images;
decoding movement associated with a plurality of frames of the encoded video bit stream to extract a plurality of movement vectors by block units of the plurality of frames;
generating a movement map based on the extracted plurality of movement vectors by block units of the plurality of frames;
processing the movement map generated using the plurality of movement vectors by grouping block units having similar types of movement vectors into respective areas such that block units having the similar types of movement vectors are grouped together into said respective areas, wherein said block units in said respective areas comprise a respective depth information;
applying a depth extraction step by integrating the depth information of the respective areas of the movement map based on the types of movement vectors associated with the block units of the video images; and
generating a depth map to render two-dimensional video applications onto a three-dimensional display.
2. The method of claim 1 , further comprising performing a gap filling step, wherein said plurality of movement vectors of the plurality of frames are:
extrapolated from said plurality of movement vectors decoded from a previous frame;
duplicated from said plurality of movement vectors decoded from the previous frame; or
interpolated using said plurality of movement vectors from a plurality of neighboring frames.
3. The method of claim 1 , wherein the decoding movement step comprises decoding said plurality of movement vectors included in compressed video streams according to a video compression standard.
4. The method of claim 3 , wherein the video compression standard comprises the Moving Picture Experts Group standard.
5. The method of claim 1 , wherein said plurality of frames of the encoded video bit stream comprises frames of “I”, “P”, or “B” types.
6. The method of claim 1 , wherein the depth extraction step comprises converting information from said plurality of movement vectors into a depth value according to the relation D=1/M, wherein D represents said depth value and M represents said movement associated with the plurality of frames.
7. The method of claim 1 , further comprising a stabilization step comprising correcting translation and rotational movement of said plurality of movement vectors resulting from parasitic movements of video applications.
8. The method of claim 1 , further comprising a depth filtering step that generates a rough depth map and filters the rough depth map using a deblocking filter to counter perceivable image degradation effects of the video images from the encoded video bit stream.
9. The method of claim 1 , wherein generating the depth map for a three-dimensional display comprises displaying the depth map onto a lenticular liquid crystal display.
10. A video device configured to process video images for three-dimensional display, wherein the device comprises:
a receiving module configured to receive an encoded video bit stream of a two-dimensional video application having video images;
a video decoder configured to decode movement associated with a plurality of frames of the encoded video bit stream to extract a plurality of movement vectors by block units of the plurality of frames;
a module configured to process a movement map generated using the plurality of movement vectors by grouping block units having similar types of movement vectors into respective areas such that block units having the similar types of movement vectors are grouped together into said respective areas, wherein said block units in said respective areas comprise a respective depth information;
a depth extraction module configured to integrate the depth information of the respective areas of the movement map based on the types of movement vectors associated with the block units of the video images; and
a module configured to generate a depth map to render two-dimensional video applications onto a three-dimensional display.
11. The device of claim 10 , where in the device is a digital mobile telephone.
12. The device of claim 10 , where in the device is a movie player.
13. The device of claim 10 , wherein the device further comprises a gap filling module configured to extrapolate said plurality of movement vectors from said plurality of movement vectors decoded from a previous frame.
14. The device of claim 10 , wherein the video decoder decodes said plurality of movement vectors included in compressed video streams according to a video compression standard.
15. The device of claim 14 , wherein the video compression standard is the Moving Picture Experts Group standard.
16. The device of claim 10 , wherein said plurality of frames of the encoded video bit stream comprises frames of “I”, “P”, or “B” types.
17. The device of claim 10 , wherein the depth extraction module is configured to convert information from said plurality of movement vectors into a depth value according to the relation D=1/M, wherein D is said depth value and M is said movement associated with the plurality of frames.
18. The device of claim 10 , wherein the device further comprises a stabilization module configured to correct translation and rotational movement of said plurality of movement vectors resulting from parasitic movements of video applications.
19. The device of claim 10 , wherein the device further comprises a depth filtering module configured to generate a rough depth map adapted to be filtered using a deblocking filter to counter perceivable image degradation effects of the video images from the encoded video bit stream.
20. The device of any claim 10 , wherein the three-dimensional display is a lenticular liquid crystal display.
21. A non-transitory computer-readable medium stored program comprising a sequence of instructions which, when executed by a microprocessor of the device, causes the processor to:
receive an encoded video bit stream of a two-dimensional video application having video Images;
decode movement associated with a plurality of frames of the encoded video bit stream to extract a plurality of movement vectors by block units of the plurality of frames;
generate a movement map based on the extracted plurality of movement vectors by block units of the plurality of frames;
process the movement map generated using the plurality of movement vectors by grouping block units having similar types of movement vectors into respective areas such that block units having the similar types of movement vectors are grouped together into said respective areas, wherein said block units in said respective areas comprise a respective depth information;
apply a depth extraction step by integrating the depth information of the respective areas of the movement map based on the types of movement vectors associated with the block units of the video images; and
generate a depth map to render two-dimensional video applications onto a three-dimensional display.