Image processing device, image processing method, and program
An image processing device with a configuration enabling generating of images from a new viewpoint by simple signal processing is provided. Input image signals for the right eye, and input image signals for the left eye to be applied to image display are input, and a right image converting unit generates a converted image where the input image signals for the right eye have been subjected to phase change in the right direction or the left direction, in accordance with features of the image. In the same way, a left image converting unit generates a converted image where the input image signals for the left eye have been subjected to phase change in the left direction or the right direction, in accordance with features of the image. The generated image of the right image converting unit and the generated image of the left image converting unit are output as multiple new viewpoint images. A converted image is generated by processing where a luminance differential signal of an input image, or a nonlinear conversion signal of the luminance differential signal is added or subtracted, to or from the input image signal, for example.
1. An image processing device comprising:
processing circuitry; and
a storage medium storing instructions executed by the processing circuitry so that the processing circuitry is configured to:
generate a second image signal of a second viewpoint based on a first image signal of a first viewpoint, the second viewpoint being different from the first viewpoint;
generate a third image signal of a third viewpoint, the third viewpoint different from the first viewpoint and the second viewpoint, by matching one or more pixels in the first image signal to a corresponding one or more pixels in the second image signal and by generating depth information that indicates a difference in position between the one or more pixels in the first image signal and the corresponding one or more pixels in the second image signal;
form a three dimensional (3D) presentation by selecting a combination of images that includes two of the first image signal, the second image signal and the third image signal based on an observation position of a user with a naked-eye compatible 3D display device, and by positioning the images in the combination of images adjacent to each other;
adjust a gain of each image of the combination of images in the 3D presentation according to the depth information to generate a corrected 3D presentation; and
output the corrected 3D presentation to the user.
2. The image processing device according to claim 1 , wherein the processing circuitry is further configured to:
extract a first luminance differential signal of the first image signal;
set the first luminance differential signal as a first feature amount; and
generate the second image signal by:
adding, to the first image signal, the first luminance differential signal or a nonlinear conversion signal of the first luminance differential signal, or
subtracting, from the first image signal the first luminance differential signal or the nonlinear conversion signal of the first luminance differential signal.
3. The image processing device according to claim 1 , wherein the processing circuitry is further configured to:
generate a fifth image signal of a fifth viewpoint based on a fourth image signal of a fourth viewpoint, the fifth viewpoint being different from the fourth viewpoint;
generate a sixth image signal of a sixth viewpoint based on the fifth image signal, the sixth viewpoint being different from the fourth viewpoint and the fifth viewpoint; and
the 3D presentation by selecting the combination of images.
4. The image processing device according to claim 3 , wherein the processing circuitry is further configured to:
extract a second luminance differential signal of the fourth image signal;
set the second luminance differential signal as a second feature amount; and
generate the fifth image signal by:
adding, to the fourth image signal, the second luminance differential signal or a nonlinear conversion signal of the second luminance differential signal, or
subtracting, from the fourth image signal, the second luminance differential signal or the nonlinear conversion signal of the second luminance differential signal.
5. The image processing device according to claim 3 , wherein the processing circuitry is further configured to:
generate image depth information based on the first image signal and the fourth image signal, wherein
the second image signal, the third image signal, the fifth image signal, and the sixth image signal are generated with the image depth information as auxiliary information.
6. The image processing device according to claim 3 , wherein the computing device executes the instructions further to:
generate first image depth estimating information based on the first image signal, wherein
the first image signal is generated with the first image depth estimating information as auxiliary information.
7. The image processing device according to claim 3 , wherein the processing circuitry is further configured to:
calculate an inter-image distance difference between a maximum distance subject and a minimum distance subject from the depth information; and
generate a new viewpoint image with the inter-image distance difference between the viewpoint images distributed approximately and uniformly, in accordance with a number of viewpoint images to be output.
8. An image processing method to be executed at an image processing device, the image processing method comprising:
receiving an input of a first image signal of a first viewpoint;
generating a second image signal of a second viewpoint based on the first image signal, the second viewpoint being different from the first viewpoint;
generating a third image signal of a third viewpoint, the third viewpoint being different from the first viewpoint and the second viewpoint, by matching one or more pixels in the first image signal to a corresponding one or more pixels in the second image signal and by generating depth information that indicates a difference in position between the one or more pixels in the first image signal and the corresponding one or more pixels in the second image signal;
forming, by processing circuitry of the image processing device, a three dimensional (3D) presentation by selecting a combination of images that includes two of the first image signal, the second image signal and the third image signal based on an observation position of a user with a naked-eye compatible 3D display device, and by positioning the images in the combination of images adjacent to each other;
adjusting a gain, by the processing circuitry, of each image of the combination of images in the 3D presentation according to the depth information to generate a corrected 3D presentation; and
outputting the corrected 3D presentation to the user.
9. A non-transitory computer-readable storage medium storing executable instructions, which, when executed by an image processing device, causes the image processing device to execute image processing comprising:
receiving an input of a first image signal of a first viewpoint;
generating a second image signal of a second viewpoint based on the first image signal, the second viewpoint being different from the first viewpoint;
generating a third image signal of a third viewpoint, the third viewpoint being different from the first viewpoint and the second viewpoint, by matching one or more pixels in the first image signal to a corresponding one or more pixels in the second image signal and by generating depth information that indicates a difference in position between the one or more pixels in the first image signal and the corresponding one or more pixels in the second image signal;
forming a three dimensional (3D) presentation by selecting a combination of images that includes two of the first image signal, the second image signal and the third image signal based on an observation position of a user with a naked-eye compatible 3D display device, and by positioning the images in the combination of images adjacent to each other;
adjusting a gain of each image of the combination of images in the 3D presentation according to the depth information to generate a corrected 3D presentation; and
outputting the corrected 3D presentation to the user.