Stereoscopic image processing methods and apparatus
Stereoscopic image processing methods and apparatus are described. Left and right eye images of a stereoscopic frame are examined to determine if one or more difference reduction operations designed to reduce the luminance and/or chrominance differences between the left and right frames is within a range used to trigger a difference reduction operation. A difference reduction operation may involve assigning portions of the left and right frames to different depth regions and/or other region categories. A decision on whether or not to perform a difference reduction operation is then performed on a per regions basis with at the difference between the left and right eye portions of at least one region being reduced when a difference reduction operation is to be performed. The difference reduction process may, and in some embodiments is, performed in a precoder which processes left and right eye images of stereoscopic frames prior to stereoscopic encoding.
1. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:
generating a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image by generating a sum of luminance difference values, the generated sum being the first overall disparity value;
determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image;
performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range, and
wherein performing said first difference reduction operation to reduce the first overall level of difference between values of said first type includes treating pixel values corresponding to a foreground image portion differently from pixel values corresponding to a background image portion.
2. The method of claim 1 , wherein the first overall disparity value is an indication of the overall luminance disparity between the left and right eye images being processed.
3. The method of claim 2 ,
wherein said first overall disparity value is a difference between a sum of luminance values in said left eye image and a sum of luminance values in said right eye image.
4. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:
generating, by summing a plurality of difference values, a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image, said values of a first type are chrominance values;
determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image;
performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range; and
wherein performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left and right eye images includes changing at least some chrominance values in at least one of said left and right eye images.
5. The method of claim 4 ,
wherein the first difference reduction operation is not performed on a foreground image region of said left and right eye images; and
wherein the first difference reduction operation is performed on a background region of said left and right eye images.
6. The method of claim 4 , wherein said step of changing at least some of the chrominance values includes limiting the change in chrominance values to a maximum amount of change which is less than a maximum amount of change used for values of another type.
7. The method of claim 1 , wherein values of said first type are luminance values;
wherein performing said first difference reduction operation to reduce the first overall level of difference between values of said first type includes changing at least some luminance values.
8. The method of claim 6 ,
wherein determining if said first overall disparity value is in the first difference range used to control performing of the first difference reduction operation further includes:
comparing said first overall disparity value to a first threshold corresponding to a lower boundary of said first difference range; and
comparing said first overall disparity value to a second threshold corresponding to an upper boundary of said first difference range.
9. The method of claim 8 , wherein determining if said first overall disparity value is in the first difference range used to control performing of the first difference reduction operation further includes:
determining that said overall disparity value is in the first difference range when said comparing steps indicate that said first overall disparity value is above said first threshold and below said second threshold.
10. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:
generating, by summing multiple difference values, a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image, said values of a first type are chrominance values;
determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image;
performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range; and
wherein performing said first difference reduction operation includes:
modifying values of the first type in a first one of said left and right eye images; and
using a depth map in determining what pixel values to modify as part of a disparity reduction operation.
11. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:
generating, by summing a plurality of difference values, a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image, said values of a first type are chrominance values;
determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image;
performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range, said step of performing said first difference reduction operation including:
generating a depth map for each of said left and right eye images; and
identifying different regions in said left and right eye images based on depths to which the regions are determined to correspond, said depths including at least a first image depth and a second image depth.
12. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:
generating a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image, said values of a first type are chrominance values, said step of generating a first overall disparity value including:
i) generating a left eye histogram of pixel values;
ii) generating a right eye histogram of pixel values; and
iii) generating said overall disparity value from said left eye histogram of pixel values and the right eye histogram of pixel values;
determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image; and
performing the first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range.
13. An apparatus for processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, comprising:
a disparity value generation module configured to generate, by summing a plurality of difference values, a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image;
a determination module configured to determine if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image; and
a difference reduction operation module configured to perform said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range, said first difference reduction operation to reduce the first overall level of difference between values of said first type treating pixel values corresponding to a foreground image portion differently from pixel values corresponding to a background image portion.
14. The apparatus of claim 13 ,
wherein said values of said first type are luminance values; and
wherein the first overall disparity value is an indication of the overall luminance disparity between the left and right eye images being processed.
15. The apparatus of claim 13 ,
wherein the difference reduction operation module is configured to modify at least some luminance values.
16. The apparatus of claim 13 ,
wherein said determination module includes a comparison module configured to compare said first overall disparity value to a first threshold corresponding to a lower boundary of said first difference range and to compare said first overall disparity value to a second threshold corresponding to an upper boundary of said first difference range.
17. The apparatus of claim 16 , wherein said determination module is further configured to determine that said overall disparity value is in the first difference range, when a result of a comparison performed by said comparison module indicates that said first overall disparity value is above said first threshold and below said second threshold.
18. The method of claim 4 , wherein the first overall disparity value is an indication of the overall chrominance disparity between the left and right eye images being processed.
19. The method of claim 4 ,
wherein said difference values are chrominance difference values.