Systems and methods for fusing images
A method performed by an electronic device is described. The method includes obtaining a first image from a first camera, the first camera having a first focal length and a first field of view. The method also includes obtaining a second image from a second camera, the second camera having a second focal length and a second field of view disposed within the first field of view. The method further includes aligning at least a portion of the first image and at least a portion of the second image to produce aligned images. The method additionally includes fusing the aligned images based on a diffusion kernel to produce a fused image. The diffusion kernel indicates a threshold level over a gray level range. The method also includes outputting the fused image. The method may be performed for each of a plurality of frames of a video feed.
1. A method performed by an electronic device, the method comprising:
obtaining a first image of a scene from a first camera, the first camera having a first focal length and a first field of view;
obtaining a second image of the scene from a second camera, the second camera having a second focal length and a second field of view disposed within the first field of view;
determining depth information associated with the scene;
determining to fuse at least a first portion of the first image with at least a second portion of the second image based on a focus position of the first image;
aligning, based on one or more keypoints in each of the first image and the second image the depth information, and a depth-based spatial transform, the at least a portion of the first image with the at least a portion of the second image to produce aligned images;
fusing the aligned images based on an anisotropic diffusion kernel to produce a fused image, wherein the anisotropic diffusion kernel is guided by at least one reference structure in the first image; and
outputting the fused image.
2. The method of claim 1 , wherein the focus position is associated with a region of interest corresponding with an autofocus center.
3. The method of claim 1 , wherein the anisotropic diffusion kernel controls fusion of the aligned images based on at least one of noise characteristics, degree of object motion, or edge information.
4. The method of claim 1 , wherein the anisotropic diffusion kernel is determined based on scene content.
5. The method of claim 1 , wherein fusing the aligned images comprises:
determining a photometric similarity measure;
determining the anisotropic diffusion kernel; and
blending the aligned images based on the photometric similarity measure and the anisotropic diffusion kernel.
6. The method of claim 1 , wherein the anisotropic diffusion kernel controls fusion of the aligned images based on a light level.
7. The method of claim 1 , wherein the anisotropic diffusion kernel is based on contrast.
8. The method of claim 7 , wherein a bandwidth of the anisotropic diffusion kernel is adaptive based on contrast.
9. The method of claim 8 , wherein the adaptive bandwidth provides increasing fusing relative to decreasing contrast.
10. The method of claim 1 , wherein the anisotropic diffusion kernel performs guided noise reduction.
11. The method of claim 10 , wherein the guided noise reduction is guided based on the first image or the second image.
12. The method of claim 1 , wherein fusing the aligned images is further based on a similarity measure indicative of a degree of similarity between the at least a portion of the first image and the at least a portion of the second image.
13. The method of claim 1 , wherein the method is performed for one or more frames of a video feed.
14. The method of claim 1 , wherein fusing the aligned images is further based on a difference in lens characteristics between a lens of the first camera and a lens of the second camera.
15. The method of claim 1 , wherein fusing the aligned images comprises compositing the aligned images within a region of interest.
16. The method of claim 15 , wherein compositing the aligned images comprises:
determining a first composite region from the first image and a second composite region from the second image; and
performing seam blending between the first composite region and the second composite region.
17. The method of claim 15 , wherein compositing the aligned images is performed in order to recover the region of interest based on replacing a portion of the region of interest that does not exist in the second image with at least a part of the first image.
18. The method of claim 1 , wherein the first image and the second image are captured concurrently.
19. The method of claim 1 , wherein the first image and the second image are captured at different times.
20. An electronic device, comprising:
a memory; and
a processor coupled to the memory, wherein the processor is configured to:
obtain a first image of a scene from a first camera, the first camera having a first focal length and a first field of view;
obtain a second image of the scene from a second camera, the second camera having a second focal length and a second field of view disposed within the first field of view;
determine depth information associated with the scene;
determine to fuse at least a first portion of the first image with at least a second portion of the second image based on a focus position of the first image;
align, based on one or more keypoints in each of the first image and the second image the depth information, and a depth-based spatial transform, the at least a portion of the first image with the at least a portion of the second image to produce aligned images;
fuse the aligned images based on an anisotropic diffusion kernel to produce a fused image, wherein the anisotropic diffusion kernel is guided by at least one reference structure in the first image; and
output the fused image.
21. The electronic device of claim 20 , wherein the anisotropic diffusion kernel controls fusion of the aligned images based on at least one of noise characteristics, degree of object motion, or edge information.
22. The electronic device of claim 20 , wherein the processor is configured to determine the anisotropic diffusion kernel based on scene content.
23. The electronic device of claim 20 , wherein the processor is configured to fuse the aligned images by:
determining a photometric similarity measure;
determining the anisotropic diffusion kernel; and
blending the aligned images based on the photometric similarity measure and the anisotropic diffusion kernel.
24. The electronic device of claim 20 , wherein the anisotropic diffusion kernel is based on contrast.
25. The electronic device of claim 24 , wherein a bandwidth of the anisotropic diffusion kernel is adaptive based on contrast.
26. The electronic device of claim 25 , wherein the adaptive bandwidth provides increasing fusing relative to decreasing contrast.
27. The electronic device of claim 20 , wherein the anisotropic diffusion kernel performs guided noise reduction.
28. The electronic device of claim 27 , wherein the guided noise reduction is guided based on the first image or the second image.
29. The electronic device of claim 20 , wherein the processor is configured to fuse the aligned images further based on a similarity measure indicative of a degree of similarity between the at least a portion of the first image and the at least a portion of the second image.
30. The electronic device of claim 20 , wherein the method is performed for one or more frames of a video feed.
31. The electronic device of claim 20 , wherein the processor is configured to fuse the aligned images further based on a difference in lens characteristics between a lens of the first camera and a lens of the second camera.
32. The electronic device of claim 20 , wherein the processor is configured to fuse the aligned images by compositing the aligned images within a region of interest.
33. The electronic device of claim 32 , wherein the processor is configured to composite the aligned images by:
determining a first composite region from the first image and a second composite region from the second image; and
performing seam blending between the first composite region and the second composite region.
34. The electronic device of claim 32 , wherein the processor is configured to composite the aligned images in order to recover the region of interest based on replacing a portion of the region of interest that does not exist in the second image with at least a part of the first image.
35. The electronic device of claim 20 , wherein the first image and the second image are captured concurrently.
36. The electronic device of claim 20 , wherein the first image and the second image are captured at different times.
37. The electronic device of claim 20 , further comprising the first camera, the second camera, and a display.
38. The electronic device of claim 20 , wherein the focus position is associated with a region of interest corresponding with an autofocus center.
39. The electronic device of claim 20 , wherein the anisotropic diffusion kernel controls fusion of the aligned images based on a light level.