Unified image processing for combined images based on spatially co-located zones
A unified image processing algorithm results in better post-processing quality for combined images that are made up of multiple single-capture images. To ensure that each single-capture image is processed in the context of the entire combined image, the combined image is analyzed to determine portions of the image (referred to as “zones”) that should be processed with the same parameters for various image processing algorithms. These zones may be determined based on the content of the combined image. Alternatively, these zones may be determined based on the position of each single-capture image with respect to the entire combined image or the other single-capture images. Once zones and their corresponding image processing parameters are determined for the combined image, they are translated to corresponding zones each of the single-capture images. Finally, the image processing algorithms are applied to each of the single-capture images using the zone-specified parameters.
1. An apparatus comprising:
a plurality of image sensors;
a processor; and
a non-transitory computer-readable medium comprising instructions that when executed by the processor cause the processor to:
access a plurality of images respectively captured by the plurality of image sensors;
combine the plurality of images to form a combined image;
determine a plurality of zones for the combined image, each zone of the plurality of zones encompassing a portion of the combined image;
project one of the plurality of zones onto at least two of the plurality of images to determine two or more projected zones in respective images of the plurality of images;
perform an image processing operation with a first set of one or more parameters in the two or more projected zones of the plurality of images; and
perform the image processing operation with a second set of one or more parameters in a different zone of the plurality of images, wherein the second set of one or more parameters is different from the first set of one or more parameters.
2. The apparatus of claim 1 , wherein the plurality of image sensors are arranged such that the plurality of images are representative of a shared field of view from similar angles and the combined image is flat.
3. The apparatus of claim 1 , wherein the plurality of image sensors are arranged such that the plurality of images are representative of a shared field of view from significantly different angles and the combined image is curved.
4. The apparatus of claim 1 , wherein the plurality of image sensors are arranged such that the combined image is a spherical image.
5. The apparatus of claim 1 , comprising:
a first hemispherical lens covering a first one of the plurality of image sensors that captures a first hemispherical image that is one of the plurality of images; and
a second hemispherical lens covering a second one of the plurality of image sensors that captures a second hemispherical image that is one of the plurality of images, wherein the first hemispherical image and the second hemispherical image are combined to form the combined image as a contiguous spherical image.
6. The apparatus of claim 5 , wherein the first hemispherical image and the second hemispherical image are combined based on, a configuration from a group consisting of a back-to-back configuration, a side-by-side configuration, a folded symmetrical configuration, and a folded asymmetrical configuration.
7. The apparatus of claim 5 , wherein the first hemispherical lens is a fish-eye lens and the second hemispherical lens is a fish-eye lens.
8. The apparatus of claim 1 , wherein the instructions further cause the processor to:
identify one or more image features in the combined image, and wherein each zone in the plurality of zones corresponds to an identified image feature.
9. The apparatus of claim 8 , wherein at least one of the plurality of zones fully encompasses one of the one or more image features.
10. The apparatus of claim 1 , wherein the plurality of zones is determined based on depth information corresponding to the combined image.
11. The apparatus of claim 1 , wherein the instructions further cause the processor to: map each of the plurality of zones in the combined image to one or more zones in the plurality of images.
12. The apparatus of claim 11 , wherein a same image processing operation is performed on overlapping zones in the plurality of images.
13. The apparatus of claim 1 , wherein the instructions further cause the processor to:
determine the first set of one or more parameters based on a property of the one of the plurality of zones.
14. The apparatus of claim 1 , wherein the instructions further cause the processor to:
after performing the image processing operation, re-combine the plurality of images based on overlapping regions of the plurality of images to produce a processed combined image.
15. A method comprising:
accessing a plurality of images respectively captured by a plurality of image sensors;
combining the plurality of images to form a combined image;
determining a plurality of zones for the combined image, each zone of the plurality of zones encompassing a portion of the combined image;
projecting one of the plurality of zones onto at least two of the plurality of images to determine two or more projected zones in respective images of the plurality of images;
performing an image processing operation with a first set of one or more parameters in the two or more projected zones of the plurality of images; and
performing the image processing operation with a second set of one or more parameters in a different zone of the plurality of images, wherein the second set of one or more parameters is different from the first set of one or more parameters.
16. The method of claim 15 , wherein the plurality of image sensors are arranged such that the plurality of images are representative of a shared field of view from significantly different angles and the combined image is curved.
17. The method of claim 15 , wherein the plurality of image sensors are arranged such that the combined image is a spherical image.
18. The method of claim 15 , comprising:
after performing the image processing operation, re-combining the plurality of images based on overlapping regions of the plurality of images to produce a processed combined image.
19. A system for image capture comprising:
an image store configured to store a plurality of images that are representative of a shared field of view;
an image combination module configured to combine the plurality of images to form a combined image;
a zone determination module configured to determine a plurality of zones for the combined image, each zone of the plurality of zones encompassing a portion of the combined image; and
an image processing module configured to project one of the plurality of zones onto at least two of the plurality of images to determine two or more projected zones in respective images of the plurality of images, perform an image processing operation with a first set of one or more parameters in the two or more projected zones of the plurality of images, and perform the image processing operation with a second set of one or more parameters in a different zone of the plurality of images, wherein the second set of one or more parameters is different from the first set of one or more parameters.
20. The system of claim 19 , comprising a plurality of image sensors configured to respectively capture the plurality of images.