Image processing apparatus and image processing method
The present disclosure relates to an image processing apparatus and an image processing method that make it possible to generate an image using texture images and depth images of two viewpoints that represent a three-dimensional structure of a greater number of regions. A selection section selects a quantization method of a depth image of each viewpoint from among a first quantization method that quantizes a reciprocal 1/r of a distance r between a viewpoint corresponding to the depth image and the polygon for each pixel, a second quantization method that quantizes the distance r, and a third quantization method that quantizes a reciprocal 1/r′ of a distance r′ between the other viewpoint that configures the two viewpoints together with the viewpoint corresponding to the depth image and the polygon in each pixel. The present disclosure can be applied, for example, to a display apparatus and the like.
1. An image processing apparatus, comprising:
a central processing unit (CPU) configured to:
generate a first texture image of a specific viewpoint, based on each of a second texture image of a first viewpoint, a third texture image of a second viewpoint, a first depth image corresponding to the second texture image, and a second depth image corresponding to the third texture image, wherein
the second texture image is obtained based on projection of a first rear face of a polygon to a first projection face with respect to the first viewpoint,
the third texture image is obtained based on projection of a second rear face of the polygon to a second projection face with respect to the second viewpoint,
the first projection face is perpendicular to a first sight line direction,
the second projection face is perpendicular to a second sight line direction,
the first sight line direction is from the first viewpoint to a center of the polygon,
the second sight line direction is from the second viewpoint to the center of the polygon,
the first viewpoint and the second viewpoint are opposed to each other across the center of the polygon,
the second texture image corresponds to a first reconstructed portion of the polygon,
the third texture image corresponds to a second reconstructed portion of the polygon, and
each of the first reconstructed portion of the polygon and the second reconstructed portion of the polygon is greater than one-half of the polygon; and
quantize a reciprocal of a first distance between the first viewpoint and the polygon at each pixel of the second texture image.
2. The image processing apparatus according to claim 1 , wherein the CPU is further configured to:
quantize, by a first quantization method, the reciprocal of the first distance between the first viewpoint and the polygon at each pixel; and
change the first quantization method to a second quantization method based on a specific condition.
3. The image processing apparatus according to claim 1 , wherein the CPU is further configured to:
quantize, by a first quantization method, a reciprocal of a second distance between the second viewpoint corresponding to the second depth image and the polygon at each pixel;
quantize, by a second quantization method, the second distance; and
quantize, by a third quantization method, the reciprocal of the first distance between the first viewpoint and the polygon at each pixel.
4. The image processing apparatus according to claim 3 , wherein
the second distance corresponding to the first quantization method and the second quantization method is a distance of a straight line that interconnects the second viewpoint corresponding to the second depth image and the polygon at each pixel, and
the first distance corresponding to the third quantization method is a distance of a straight line that interconnects the first viewpoint and the polygon at each pixel.
5. The image processing apparatus according to claim 3 , wherein
the CPU is further configured to select, for each of the first viewpoint and the second viewpoint, a specific quantization method for the first depth image and the second depth image, based on an angle defined by a depth direction of the second texture image of the first viewpoint and a depth direction of the first texture image of the specific viewpoint.
6. The image processing apparatus according to claim 5 , wherein
the CPU is further configured to select the first quantization method as the specific quantization method for the first depth image and the second depth image, based on the angle that is:
equal to or greater than 0 degrees,
one of smaller than a first threshold value, or equal to or greater than a second threshold value, and
smaller than 360 degrees,
the first threshold value is smaller than 90 degrees, and
the second threshold value is greater than 270 degrees.
7. The image processing apparatus according to claim 6 , wherein
the CPU is further configured to select the second quantization method as the specific quantization method for the first depth image and the second depth image, based on the angle that is:
equal to or greater than the first threshold value,
one of smaller than a third threshold value, or equal to or greater than a fourth threshold value, and
smaller than the second threshold value,
the third threshold value is greater than 90 degrees and smaller than 180 degrees, and
the fourth threshold value is greater than 180 degrees and smaller than 270 degrees.
8. The image processing apparatus according to claim 7 , wherein
the CPU is further configured to select the third quantization method as the specific quantization method for the first depth image and the second depth image, based on the angle that is equal to or greater than the third threshold value and smaller than the fourth threshold value.
9. The image processing apparatus according to claim 1 , wherein
the polygon is generated based on a plurality of picked up images acquired by a plurality of cameras and a plurality of depth images corresponding to the plurality of picked up images,
the plurality of cameras is around an imaging object corresponding to the polygon, and
at least a part of the imaging object is in an imaging range of each of the plurality of cameras.
10. The image processing apparatus according to claim 9 , wherein
the CPU is further configured to generate the first texture image of the specific viewpoint based on a fourth texture image of an omnidirectional image and a third depth image of the omnidirectional image,
generation of the third depth image of the omnidirectional image is based on the plurality of depth images, and
generation of the fourth texture image of the omnidirectional image is based on the plurality of picked up images.
11. An image processing method, comprising:
in an image processing apparatus:
generating a first texture image of a specific viewpoint, based on each of a second texture image of a first viewpoint, a third texture image of a second viewpoint, a first depth image corresponding to the second texture image, and a second depth image corresponding to the third texture image, wherein
the second texture image is obtained based on projection of a first rear face of a polygon to a first projection face with respect to the first viewpoint,
the third texture image is obtained based on projection of a second rear face of the polygon to a second projection face with respect to the second viewpoint,
the first projection face is perpendicular to a first sight line direction,
the second projection face is perpendicular to a second sight line direction,
the first sight line direction is from the first viewpoint to a center of the polygon,
the second sight line direction is from the second viewpoint to the center of the polygon,
the first viewpoint and the second viewpoint are opposed to each other across the center of the polygon,
the second texture image corresponds to a first reconstructed portion of the polygon,
the third texture image corresponds to a second reconstructed portion of the polygon, and
each of the first reconstructed portion of the polygon and the second reconstructed portion of the polygon is greater than one-half of the polygon; and
quantizing a reciprocal of a distance between the first viewpoint and the polygon at each pixel of the second texture image.
12. An image processing apparatus, comprising:
a central processing unit (CPU) configured to:
project a first rear face of a polygon to a first projection face perpendicular to a first sight line direction;
project a second rear face of the polygon to a second projection face perpendicular to a second sight line direction, wherein
the projection of the first rear face of the polygon is with respect to a first viewpoint,
the projection of the second rear face of the polygon is with respect to a second viewpoint,
the first sight line direction is from the first viewpoint to a center of the polygon,
the second sight line direction is from the second viewpoint to the center of the polygon, and
the first viewpoint and the second viewpoint are opposed to each other across the center of the polygon;
generate a first texture image of the first viewpoint based on the projection of the first rear face of the polygon to the first projection face;
generate a second texture image of the second viewpoint based on the projection of the second rear face of the polygon to the second projection face, wherein
the first texture image corresponds to a first reconstructed portion of the polygon,
the second texture image corresponds to a second reconstructed portion of the polygon, and
each of the first reconstructed portion of the polygon and the second reconstructed portion of the polygon is greater than one-half of the polygon;
quantize a reciprocal of a first distance between the first viewpoint and the polygon at each pixel of the first texture image;
obtain a quantization value based on the quantization of the reciprocal of the first distance;
set the quantization value as a pixel value; and
generate, based on the pixel value, a first depth image and a second depth image, wherein
the first depth image corresponds to the first texture image, and
the second depth image corresponds to the second texture image.
13. The image processing apparatus according to claim 12 , wherein CPU is further configured to:
quantize, by a first quantization method, the reciprocal of the first distance between the first viewpoint and the polygon at each pixel; and
change the first quantization method to a second quantization method based on a specific condition.
14. The image processing apparatus according to claim 12 , wherein CPU is further configured to:
quantize, by a first quantization method, a reciprocal of a second distance between the second viewpoint corresponding to the second depth image and the polygon at each pixel;
quantize, by a second quantization method, the second distance; and
quantize, by a third quantization method, the reciprocal of the first distance between the first viewpoint and the polygon at each pixel.
15. The image processing apparatus according to claim 14 , wherein
the second distance corresponding to the first quantization method and the second quantization method is a distance of a straight line that interconnects the second viewpoint corresponding to the second depth image and the polygon at each pixel, and
the first distance corresponding to the third quantization method is a distance of a straight line that interconnects the first viewpoint and the polygon at each pixel.
16. The image processing apparatus according to claim 14 , wherein the CPU is further configured to transmit each of the first texture image of the first viewpoint, the second texture image of the second viewpoint, the first depth image, and the second depth image.
17. The image processing apparatus according to claim 16 , wherein the CPU is further configured to:
generate first depth images of the first viewpoint and the second viewpoint based on the first quantization method;
generate second depth images of the first viewpoint and the second viewpoint based on the second quantization method;
generate third depth images of the first viewpoint and the second viewpoint based on the third quantization method; and
transmit one of the first depth images, the second depth images, or the third depth images.
18. The image processing apparatus according to claim 12 , wherein
the CPU is further configured to generate the polygon based on a plurality of picked up images acquired by a plurality of cameras and a plurality of depth images corresponding to the plurality of picked up images,
the plurality of cameras is around an imaging object corresponding to the polygon, and
at least a part of the imaging object is in an imaging range of each of the plurality of cameras.
19. The image processing apparatus according to claim 18 , wherein the CPU is further configured to:
generate a third texture image of an omnidirectional image based on the plurality of picked up images; and
generate a third depth image of the omnidirectional image based on the plurality of depth images.
20. An image processing method, comprising:
in an image processing apparatus:
projecting a first rear face of a polygon to a first projection face perpendicular to a first sight line direction,
projecting a second rear face of the polygon to a second projection face perpendicular to a second sight line direction, wherein
the projection of the first rear face of the polygon is with respect to a first viewpoint,
the projection of the second rear face of the polygon is with respect to a second viewpoint,
the first sight line direction is from the first viewpoint to a center of the polygon,
the second sight line direction is from the second viewpoint to the center of the polygon, and
the first viewpoint and the second viewpoint are opposed to each other across the center of the polygon;
generating a first texture image of the first viewpoint based on the projection of the first rear face of the polygon to the first projection face;
generating a second texture image of the second viewpoint based on the projection of the second rear face of the polygon to the second projection face, wherein
the first texture image corresponds to a first reconstructed portion of the polygon,
the second texture image corresponds to a second reconstructed portion of the polygon, and
each of the first reconstructed portion of the polygon and the second reconstructed portion of the polygon is greater than one-half of the polygon;
quantizing a reciprocal of a distance between the first viewpoint and the polygon at each pixel of the first texture image;
obtaining a quantization value based on the quantization of the reciprocal of the distance;
setting the quantization value as a pixel value; and
generating, based on the pixel value, a first depth image and a second depth image, wherein
the first depth image corresponds to the first texture image, and
the second depth image corresponds to the second texture image.
21. An image processing apparatus, comprising:
a central processing unit (CPU) configured to:
generate a first texture image of a specific viewpoint, based on each of a second texture image of a first viewpoint, a third texture image of a second viewpoint, a first depth image corresponding to the second texture image, and a second depth image corresponding to the third texture image, wherein
the second texture image is obtained based on projection of a first rear face of a polygon to a first projection face with respect to the first viewpoint,
the third texture image is obtained based on projection of a second rear face of the polygon to a second projection face with respect to the second viewpoint,
the first projection face is perpendicular to a first sight line direction,
the second projection face is perpendicular to a second sight line direction,
the first sight line direction is from the first viewpoint to a center of the polygon,
the second sight line direction is from the second viewpoint to the center of the polygon, and
the first viewpoint and the second viewpoint are opposed to each other across the center of the polygon;
quantize, by a first quantization method, a reciprocal of a first distance between the first viewpoint and the polygon at each pixel of the second texture image;
quantize, by a second quantization method, a reciprocal of a second distance between the second viewpoint corresponding to the second depth image and the polygon at each pixel;
quantize, by a third quantization method, the second distance; and
select, for each of the first viewpoint and the second viewpoint, a specific quantization method for the first depth image and the second depth image, based on an angle defined by a depth direction of the second texture image of the first viewpoint and a depth direction of the first texture image of the specific viewpoint.