Image processing apparatus and method and image pickup apparatus
View Patent ↗A system to generate a three-dimensional image from a two-dimensional image in accordance with the mixture state of images. A foreground/background component image generator generates, from image data including a foreground area consisting of only foreground object components, a background area consisting of only background object components, and a mixed area in which the foreground object components and the background object components are mixed, a foreground component image consisting of only the foreground object components and a background component image consisting of only the background object components. A foreground-parallax-image generator generates a right-eye foreground parallax image by adding the parallax to the foreground component image, and also generates a left-eye foreground parallax image by adding the parallax to the foreground component image. A synthesizer generates a right-eye parallax image by combining the right-eye foreground parallax image with a background image, and a synthesizer generates a left-eye parallax image by combining the left-eye foreground parallax image with the background image.
1. An image processing apparatus for processing image data which is formed of a predetermined number of pixel data obtained-by an image-capturing device including a predetermined number of pixels, the pixels having a time integrating function, comprising:
a foreground/background component image generating unit configured to generate, from the image data including a foreground area consisting of only foreground object components forming a foreground object, a background area consisting of only background object components forming a background object, and a mixed area in which the foreground object components and the background object components are mixed, a foreground component image consisting of only the foreground object components and a background component image consisting of only the background object components;
a foreground-parallax-image generating unit configured to generate a right-eye foreground parallax image by adding a right-eye parallax to the foreground component image, and to generate a left-eye foreground parallax image by adding a left-eye parallax to the foreground component image,
wherein the right-eye foreground parallax image and left-eye foreground parallax image are stored; and
a parallax-image generating unit configured to generate a right-eye parallax image by combining the right-eye foreground parallax image that has been temporarily stored in the foreground-parallax-image generating unit with a certain background image, and the parallax-image generating unit configured to generate a left-eye parallax image by combining the left-eye foreground parallax image which has been temporarily stored in the foreground-parallax-image generating unit with the background image.
2. The image processing apparatus according to claim 1 , further comprising a motion-blur adjusting unit configured to adjust an amount of motion blur of the foreground component image generated by said foreground/background component image generating unit.
3. The image processing apparatus according to claim 1 , wherein said foreground/background component image generating unit further comprises:
an area specifying unit configured to specify the foreground area, the background area, and the mixed area of the image data; and
a mixture-ratio detection unit configured to detect a mixture ratio indicating a ratio of the mixture of the foreground object components and the background object components in the mixed area based on a result obtained by specifying the foreground area, the background area, and the mixed area.
4. The image processing apparatus according to claim 3 , wherein said foreground/background component image generating unit comprises:
a foreground/background separating unit configured to separate the pixels belonging to the mixed area into the foreground object components and the background object components based on the mixture ratio so as to generate the foreground component image and the background component image based on the separated foreground object components and the separated background object components.
5. An image processing method for processing image data which is formed of a predetermined number of pixel data obtained by an image-capturing device including a predetermined number of pixels, the pixels having a time integrating function, comprising, performed by a processor:
a foreground/background component image generating step, utilizing a foreground/background component image generating unit, of generating, from the image data including a foreground area consisting of only foreground object components forming a foreground object, a background area consisting of only background object components forming a background object, and a mixed area in which the foreground object components and the background object components are mixed, a foreground component image consisting of only the foreground object components and a background component image consisting of only the background object components;
a foreground-parallax-image generating step, utilizing a foreground/background component image generating unit, of generating, a right-eye foreground parallax image by adding a right-eye parallax to the foreground component image, and generating a left-eye foreground parallax image by adding a left-eye parallax to the foreground component image;
a storing step of storing, utilizing a storage unit, the right eye foreground parallax image and left-eye foreground parallax image; and
a parallax-image generating step, utilizing a parallax-image generating unit, of generating, a right-eye parallax image by combining the right-eye foreground parallax image that has been temporarily stored in the storing step with a certain background image, and generating a left-eye parallax image that has been temporarily stored in the storing step by combining the left-eye foreground parallax image with the background image.
6. The image processing method according to claim 5 , further comprising a motion-blur adjusting step, utilizing an adjustment unit, of adjusting an amount of motion blur of the foreground component image generated by processing of said foreground/background component image generating step.
7. The image processing method according to claim 5 , wherein said foreground/background component image generating step further comprises:
an area specifying step of specifying the foreground area, the background area, and the mixed area of the image data; and
a mixture-ratio detection step of detecting a mixture ratio indicating a ratio of the mixture of the foreground object components and the background object components in the mixed area based on a result obtained by specifying the foreground area, the background area, and the mixed area.
8. The image processing method according to claim 7 , wherein said foreground/background component image generating step comprises:
a foreground/background separating step of separating the pixels belonging to the mixed area into the foreground object components and the background object components based on the mixture ratio; and
generating the foreground component image and the background component image based on the separated foreground object components and the separated background object components.
9. An image-capturing apparatus comprising:
an image-capturing unit configured to output a subject image captured by an image-capturing device including a predetermined number of pixels, the pixels having a time integrating function, as image data formed of a predetermined number of pixel data;
a foreground/background component image generating unit configured to generate, from the image data including a foreground area consisting of only foreground object components forming a foreground object, a background area consisting of only background object components forming a background object, and a mixed area in which the foreground object components and the background object components are mixed, a foreground component image consisting of only the foreground object components and a background component image consisting of only the background object components;
a foreground-parallax-image generating unit configured to generate a right-eye foreground parallax image by adding a right-eye parallax to the foreground component image, and for generating a left-eye foreground parallax image by adding a left-eye parallax to the foreground component image,
wherein the right-eye foreground parallax image and left-eye foreground parallax image are stored; and
a parallax-image generating unit configured to generate a right-eye parallax image by combining the right-eye foreground parallax image that has been temporarily stored in the foreground-parallax-image generating unit with a certain background image, and for generating a left-eye parallax image by combining the left-eye foreground parallax image that has been temporarily stored in the foreground-parallax-image generating unit with the background image.
10. The image-capturing apparatus according to claim 9 , further comprising a motion-blur adjusting unit configured to adjust an amount of motion blur of the foreground component image generated by said foreground/background component image generating unit.
11. The image-capturing apparatus according to claim 9 , wherein said foreground/background component image generating unit further comprises:
an area specifying unit configured to specify the foreground area, the background area, and the mixed area of the image data; and
a mixture-ratio detection-unit configured to detect a mixture ratio indicating a ratio of the mixture of the foreground object components and the background object components in the mixed area based on a result obtained by specifying the foreground area, the background area, and the mixed area.
12. The image-capturing apparatus according to claim 11 , wherein said foreground/background component image generating unit comprises a foreground/background separating unit configured to separate the pixels belonging to the mixed area into the foreground object components and the background object components based on the mixture ratio so as to generate the foreground component image and the background component image based on the separated foreground object components and the separated background object components.