Image processing apparatus, image pickup apparatus, image processing method, and non-transitory computer-readable storage medium for generating an image based on plurality of parallax images

1. An image processing apparatus comprising:

at least one processor operatively coupled to a memory, and serving as:

(a) a determiner configured to determine a weight coefficient that varies depending on a position in each of a plurality of parallax images; and

(b) an image generator configured to synthesize the plurality of parallax images based on the weight coefficient to generate an image,

wherein a sum of the weight coefficients of the plurality of parallax images is constant with respect to all positions in the plurality of parallax images.

2. The image processing apparatus according to claim 1 , wherein the image generator is configured to multiply each of the plurality of parallax images by the weight coefficient to create weighted parallax images, and to add the weighted parallax images to generate the image.

3. The image processing apparatus according to claim 1 , wherein the image generator is configured to multiply each of the plurality of parallax images by the weight coefficient to create weighted parallax images, and to shift and to add the weighted parallax images to generate the image.

4. The image processing apparatus according to claim 1 , wherein the weight coefficient varies continuously depending on the position in each of the plurality of parallax images.

5. The image processing apparatus according to claim 1 , wherein the plurality of parallax images are acquired by an image pickup element including a plurality of pixels, each pixel including a plurality of photoelectric converters configured to receive light beams passing through partial pupil regions of an imaging optical system that are different from each other.

6. The image processing apparatus according to claim 1 , wherein the image generator is configured to synthesize the plurality of parallax images based on the weight coefficient in a first region of the image.

7. The image processing apparatus according to claim 6 , wherein the first region is a region set to reduce a blur of a second object that is at a closer range side than a first object.

8. The image processing apparatus according to claim 6 , wherein the image generator is configured to multiply the plurality of parallax images by the weight coefficients that are equal to each other to create weighted parallax images, and to add the weighted parallax images in a second region different from the first region of the image to generate the image.

9. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image in which a second object that is at a closer range side than a first object is captured in a widest range to be minimized in a plurality of weight coefficients for the plurality of parallax images.

10. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image in which a second object that is at a closer range side than a first object is captured in a narrowest range to be maximized in a plurality of weight coefficients for the plurality of parallax images.

11. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image that has a smallest contrast evaluation value to be minimized in a plurality of weight coefficients for the plurality of parallax images.

12. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image that has a largest contrast evaluation value to be maximized in a plurality of weight coefficients for the plurality of parallax images.

13. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image that has a smallest effective aperture value to be minimized in a plurality of weight coefficients for the plurality of parallax images.

14. The image processing apparatus according to claim 6 , wherein the determiner is configured to set, in the first region of the image, the weight coefficient for a parallax image that has a largest effective aperture value to be maximized in a plurality of weight coefficients for the plurality of parallax images.

15. An image pickup apparatus comprising:

(A) an image pickup element including a plurality of pixels, each pixel including a plurality of photoelectric converters configured to receive light beams passing through partial pupil regions of an imaging optical system that are different from each other; and

(B) at least one processor operatively coupled to a memory, and serving as:

(a) a determiner configured to determine a weight coefficient that varies depending on a position in each of a plurality of parallax images obtained from the plurality of photoelectric converters; and

(b) an image generator configured to synthesize the plurality of parallax images based on the weight coefficient to generate an image,

wherein a sum of the weight coefficients of the plurality of parallax images is constant with respect to all positions in the plurality of parallax images.

16. The image pickup apparatus according to claim 15 , wherein the image pickup element includes the plurality of photoelectric converters for each of microlenses, and the microlenses are arrayed in two dimensions.

17. An image processing method comprising:

determining a weight coefficient that varies depending on a position in each of a plurality of parallax images; and

synthesizing the plurality of parallax images based on the weight coefficient to generate an image,

wherein a sum of the weight coefficients of the plurality of parallax images is constant with respect to all positions in the plurality of parallax images.

18. A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process comprising:

determining a weight coefficient that varies depending on a position in each of a plurality of parallax images; and

synthesizing the plurality of parallax images based on the weight coefficient to generate an image,

wherein a sum of the weight coefficients of the plurality of parallax images is constant with respect to all positions in the plurality of parallax images.