Image stitching in a multi-camera array
Images captured by multi-camera arrays with overlap regions can be stitched together using image stitching operations. An image stitching operation can be selected for use in stitching images based on a number of factors. An image stitching operation can be selected based on a view window location of a user viewing the images to be stitched together. An image stitching operation can also be selected based on a type, priority, or depth of image features located within an overlap region. Finally, an image stitching operation can be selected based on a likelihood that a particular image stitching operation will produce visible artifacts. Once a stitching operation is selected, the images corresponding to the overlap region can be stitched using the stitching operation, and the stitched image can be stored for subsequent access.
1. A method of stitching images, the method comprising:
capturing a first image of a first field of view via a first camera of a camera apparatus;
capturing a second image of a second field of view via a second camera of the camera apparatus, the first image and the second image including an overlap region corresponding to a common field of view between the first field of view and the second field of view, the first image and the second image including one or more image features of one or more objects within the overlap region, the one or more image features including a first image feature of a first object within the overlap region;
identifying depth of the one or more image features within the overlap region;
selecting an image stitching algorithm based on the depth of the one or more image features within the overlap region such that different image stitching algorithms of different qualities are selected for different depths of the one or more image features within the overlap region, wherein a first image stitching algorithm of a first quality is selected based on first depth of the first image feature within the overlap region and a second image stitching algorithm of a second quality different from the first quality is selected based on second depth of the first image feature within the overlap region; and
generating a stitched image based on stitching of at least a portion of the first image and at least a portion of the second image via the selected image stitching algorithm such that different amounts of processing power are used to generate the stitched image for the different depths of the one or more image features within the overlap region, wherein usage of the first image stitching algorithm of the first quality based on the first depth of the first image feature results in usage of a first amount of processing power and usage of the second image stitching algorithm of the second quality based on the second depth of the first image feature results in usage of a second amount of processing power different from the first amount of processing power.
2. A method of processing image data obtained from a first camera and a second camera of a camera apparatus, the first camera having a first field of view, the second camera having a second field of view, the first and second fields of view having a common field of view, the method comprising:
obtaining first image data of the first field of view via the first camera;
obtaining second image data of the second field of view via the second camera, the first image data and the second image data including an overlap region corresponding to the common field of view between the first field of view and the second field of view, the first image data and the second image data including one or more image features of one or more objects within the overlap region, the one or more image features including a first image feature of a first object within the overlap region;
identifying depth of the one or more image features within the overlap region;
selecting an image stitching algorithm based on the depth of the one or more image features within the overlap region such that different image stitching algorithms of different qualities are selected for different depths of the one or more image features within the overlap region, wherein a first image stitching algorithm of a first quality is selected based on first depth of the first image feature within the overlap region and a second image stitching algorithm of a second quality different from the first quality is selected based on second depth of the first image feature within the overlap region; and
generating a stitched image based on stitching of at least a portion of the first image data and at least a portion of the second image data via the selected image stitching algorithm such that different amounts of processing power are used to generate the stitched image for the different depths of the one or more image features within the overlap region, wherein usage of the first image stitching algorithm of the first quality based on the first depth of the first image feature results in usage of a first amount of processing power and usage of the second image stitching algorithm of the second quality based on the second depth of the first image feature results in usage of a second amount of processing power different from the first amount of processing Power.
3. The method of claim 1 , wherein selection of the image stitching algorithm is performed based on closest depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first depth or the second depth of the first image feature being the closest depth of the one or more image features within the overlap region.
4. The method of claim 1 , wherein selection of the image stitching algorithm is performed based on furthest depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first depth or the second depth of the first image feature being the furthest depth of the one or more image features within the overlap region.
5. The method of claim 1 , wherein selection of the image stitching algorithm is performed based depth of a facial feature within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first image feature being the facial feature of a person within the overlap region.
6. The method of claim 1 , wherein selection of the image stitching algorithm is performed based on average depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on average of multiple depth of multiple image features within the overlap region.
7. The method of claim 1 , wherein the quality of the selected image stitching algorithm:
increases with decrease in the depth of the one or more image features within the overlap region; and
decreases with increase in the depth of the one or more image features within the overlap region.
8. The method of claim 7 , wherein the quality of the selected image stitching algorithm decreases responsive to the depth of the one or more image features within the overlap region being below a threshold depth.
9. The method of claim 1 , wherein:
the overlap region is divided into multiple sub-blocks;
separate selection of the image stitching algorithm is performed for individual ones of the multiple sub-blocks based on depth of individual image features within the individual ones of the multiple sub-blocks.
10. The method of claim 2 , wherein selection of the image stitching algorithm is performed based on closest depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first depth or the second depth of the first image feature being the closest depth of the one or more image features within the overlap region.
11. The method of claim 2 , wherein selection of the image stitching algorithm is performed based on furthest depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first depth or the second depth of the first image feature being the furthest depth of the one or more image features within the overlap region.
12. The method of claim 2 , wherein selection of the image stitching algorithm is performed based depth of a facial feature within the overlap region such that the image stitching algorithm is selected based on the first depth or the second depth of the first image feature responsive to the first image feature being the facial feature of a person within the overlap region.
13. The method of claim 2 , wherein selection of the image stitching algorithm is performed based on average depth of the one or more image features within the overlap region such that the image stitching algorithm is selected based on average of multiple depth of multiple image features within the overlap region.
14. The method of claim 2 , wherein the quality of the selected image stitching algorithm:
increases with decrease in the depth of the one or more image features within the overlap region; and
decreases with increase in the depth of the one or more image features within the overlap region.
15. The method of claim 14 , wherein the quality of the selected image stitching algorithm decreases responsive to the depth of the one or more image features within the overlap region being below a threshold depth.
16. The method of claim 2 , wherein:
the overlap region is divided into multiple sub-blocks;
separate selection of the image stitching algorithm is performed for individual ones of the multiple sub-blocks based on depth of individual image features within the individual ones of the multiple sub-blocks.