Stereoscopic video encoding and decoding methods and apparatus
In some embodiments left and right eye images are encoded following an entropy reduction operation being applied to one of the eye images when there is a difference between the left and right images of an image pair. Information about regions of negative parallax within the entropy reduced image of an image pair is encoded along with the images. Upon decoding a sharpening filter is applied to the image in an image pair which was subjected to the entropy reduction operation. In addition edge enhancement filtering is performed on the regions of the recovered entropy reduced image which are identified in the encoded image data as regions of negative parallax. Interleaving of left and right eye images at the input of the encoder combined with entropy reduction allows for efficient encoding, storage, and transmission of 3D images.
1. A method of operating an encoder apparatus to encode stereoscopic image data representing pairs of first and second eye images, each first eye image in an image pair corresponding to one of a left eye image and a right eye image, the second eye image of each image pair corresponding to the other one of the left and right eye images, the method comprising:
determining an amount of difference between a first eye image and a second eye image of a first image pair;
performing a negative parallax detection operation to detect image regions in said second eye image of said first image pair which are in negative parallax relative to a corresponding portion of said first eye image of said first image pair; and
when it is determined that there is a non-zero amount of difference between the first eye image and the second eye image of said first image pair, performing the steps of:
i) performing an entropy reduction operation on second eye image data representing the second eye image of the first image pair to produce entropy reduced second eye image data, wherein said entropy reduction operation includes performing at least one of an FFT based filtering operation or a neighborhood filtering operation on said second eye image data to smooth the second eye image of said first image pair;
ii) encoding first eye image data representing the first one of the eye images of said first image pair to generate compressed first eye image data; and
iii) encoding said entropy reduced second eye image data to generate compressed second eye image data; and
storing or transmitting the compressed first eye image data and compressed second eye image data; and
storing information identifying at least one detected image region of negative parallax in said second eye image of said first image pair in a file with said compressed first eye image data and compressed second eye image data.
2. The method of claim 1 , wherein said encoding said entropy reduced second eye image data includes supplying said entropy reduced second eye image data to an encoder which performs inter-frame motion compensated encoding, data corresponding to said first and second eye images being processed as data corresponding to sequential frames.
3. The method of claim 1 , wherein said entropy reduction operation is a function of the determined amount of difference between said first eye image and said second eye image, the amount of entropy reduction being greater when the amount of difference is a first amount than when it is a second amount, said second amount being greater than said first amount.
4. The method of claim 1 , further comprising:
storing information indicating said amount of difference in said file with said compressed first eye image data and compressed second eye image data.
5. The method of claim 1 , wherein said detected image regions are polygonal image regions; and wherein said information identifying at least one detected image region of negative parallax includes information defining the perimeter of at least one polygonal image region.
6. The method of claim 5 , wherein said information defining the perimeter of the at least one polygonal image region of negative parallax includes information identifying points in said second eye image corresponding to the perimeter and a formula for determining a portion of said perimeter from said identified locations of points corresponding to the perimeter.
7. The method of claim 1 , further comprising:
when it is determined that there is a zero amount of difference between the first eye image and the second eye image of said first image pair performing, prior to performing said storing or transmitting step, the steps of:
ii) encoding said first eye image data to generate compressed first eye image data; and
iii) encoding said second eye image data to generate compressed second eye image data.
8. The method of claim 7 , wherein said encoding said second eye image data generates compressed second eye image data indicating that the second eye image is the same as the first eye image.
9. The method of claim 1 , further comprising:
processing a second image pair including a left eye image and a right eye image.
10. An apparatus to encode stereoscopic image data representing pairs of first and second eye images, each first eye image in an image pair corresponding to one of a left eye image and a right eye image, the second eye image of each image pair corresponding to the other one of the left and right eye images, the apparatus comprising:
a processor configured to:
determine an amount of difference between a first eye image and a second eye image of a first image pair;
perform a negative parallax detection operation to detect image regions in said second eye image of said first image pair which are in negative parallax relative to a corresponding portion of said first eye image of said first image pair;
perform an entropy reduction operation on second eye image data representing the second eye image of the first image pair to produce entropy reduced second eye image data, when it is determined that there is a non-zero amount of difference between the first eye image and the second eye image of said first image pair, wherein said entropy reduction operation includes at least one of an FFT based filtering operation or a neighborhood filtering operation on said second eye image data to smooth the second eye image of said first image pair;
encode first eye image data representing the first one of the eye images of said first image pair to generate compressed first eye image data, and encode said entropy reduced second eye image data to generate compressed second eye image data; and
memory coupled to said processor for storing the compressed first eye image data and compressed second eye image data, and wherein said memory includes information identifying at least one detected image region of negative parallax in said second eye image of said first image pair in a file with said compressed first eye image data and compressed second eye image data.
11. The apparatus of claim 10 , further comprising:
an interface configured to transmit the compressed first eye image data and compressed second eye image data.
12. The apparatus of claim 10 , wherein said processor is configured to treat data corresponding to said first and second eye images being processed as data corresponding to sequential frames with performs inter-frame motion compensated encoding being used to by said process to encode at least some data corresponding to said first and second eye images.
13. The apparatus of claim 10 , wherein said processor is configured to perform said entropy reduction as a function of the determined amount of difference between said first eye image and said second eye image, the amount of entropy reduction being greater when the amount of difference is a first amount than when it is a second amount, said second amount being greater than said first amount.
14. The apparatus of claim 10 , wherein said memory further includes stored information indicating said amount of difference in said file with said compressed first eye image data and compressed second eye image data.
15. The apparatus of claim 10 , wherein said detected image regions are polygonal image regions; and wherein said information identifying at least one detected image region of negative parallax includes information defining the perimeter of at least one polygonal image region.
16. The apparatus of claim 15 , wherein said information defining the perimeter of the at least one polygonal image region of negative parallax includes information identifying points in said second eye image corresponding to the perimeter and a formula for determining a portion of said perimeter from said identified locations of points corresponding to the perimeter.
17. The method of claim 1 , further comprising:
when it is determined that there is a zero amount of difference between the first eye image and the second eye image of said first image pair:
i) encoding first eye image data representing the first one of the eye images of said first image pair to generate said compressed first eye image data; and
iii) encoding said second eye image data to generate said compressed second eye image data.
18. The method of claim 17 wherein an entropy reduction operation is not performed on said second eye image data when there is a zero amount of difference between the first eye image and the second eye image of said first image pair.