IP Library › Granted Patent US 12,437,432

Granted Patent B2

US 12,437,432 · App. 18/434,684 · Granted Oct 7, 2025

Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints

Inventors: Florian Ciurea (Campbell, CA); Kartik Venkataraman (San Jose, CA); Gabriel Molina (Grass Valley, CA); Dan Lelescu (Morgan Hill, CA)

Assignee: Adeia Imaging LLC

G06T7/593 G02B27/0075 G06T7/557 G06T7/85 G06T15/20 H04N13/128 H04N13/232 H04N13/243 H04N23/16 G06T2200/21 G06T2207/10012 G06T2207/10024 G06T2207/10052 H04N2013/0081 H04N2013/0088

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,437,432
App. No.: 18/434,684
Granted: Oct 7, 2025
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Claims (46)

1. An imaging system, comprising:

a plurality of imagers;

a processor; and

a memory containing an image processing application that configures the processing system to:

obtain a plurality of images of a scene comprising a right image and a left image from the plurality of imagers;

rectify the plurality of images;

compute disparities between pixels in the right image and pixels in the left image;

select a reference image from among the right image and the left image;

identify occluded pixels in the reference image; and

generate a confidence map for the computed disparities associated with the reference image, where the confidence map comprises a confidence metric for pixels in the reference image based on the computed disparities, and where the confidence metric for occluded pixels encodes a confidence factor indicating detection of an occlusion.

2. The imaging system of claim 1 , wherein the confidence metric encodes an error.

3. The imaging system of claim 1 , wherein:

the image processing application further configures the processor to:

generate reduced resolution images from the plurality of images; and

computing the disparity between pixels in the right image and pixels in the left image comprises utilizing the reduced resolution images to compute disparity between pixels in the right image and pixels in the left image.

4. The imaging system of claim 1 , wherein the image processing application further configures the processor to obtain calibration information for a left imager and a right imager from the plurality of imagers, where the right image is generated by the right imager, and the left image is generated by the left imager, and the rectification is based on the calibration information.

5. The imaging system of claim 4 , wherein the calibration information comprises a focal length of the right imager and a focal length of the left imager.

6. The imaging system of claim 1 , wherein the image processing application further configures the processor to refine, using the confidence map, the computed disparities to account for occluded pixels.

7. The imaging system of claim 1 , wherein the image processing application further configures the processor to estimate depths of objects within the scene based on the computed disparities and the confidence map.

8. The imaging system of claim 1 , wherein obtaining the plurality of images of the scene comprises capturing a plurality of images using the plurality of imagers; and obtaining a plurality of lower resolution images.

9. The imaging system of claim 1 , wherein to identify occluded pixels, the image processing application further configures the processor to identify likely occlusion zones in the reference image.

10. The imaging system of claim 1 , wherein to identify occluded pixels, the image processing application further configures the processor to:

calculate a photometric distance of corresponding pixels in a non-reference image along an epipolar line with a reference pixel in the reference image; and

determine that a corresponding pixel is occluded when its photometric distance exceeds a threshold.

11. The imaging system of claim 1 , wherein the image processing application further configures the processor to generate a depth map based on the disparities.

12. An imaging method of generating confidence maps for images, comprising:

obtaining a plurality of images of a scene comprising a right image and a left image from a plurality of imagers;

rectifying the plurality of images;

computing disparities between pixels in the right image and pixels in the left image;

selecting a reference image from among the right image and the left image;

identifying occluded pixels in the reference image; and

generating a confidence map for the computed disparities associated with the reference image, where the confidence map comprises a confidence metric for pixels in the reference image based on the computed disparities, and where the confidence metric for occluded pixels encodes a confidence factor indicating detection of an occlusion.

13. The method of claim 12 , wherein the confidence metric encodes an error.

14. The method of claim 12 , further comprising:

generating reduced resolution images from the plurality of images; and

15. The method of claim 12 , further comprising obtaining calibration information for a left imager and a right imager from the plurality of imagers, where the right image is generated by the right imager, and the left image is generated by the left imager, and the rectification is based on the calibration information.

16. The method of claim 15 , wherein the calibration information comprises a focal length of the right imager and a focal length of the left imager.

17. The method of claim 12 , further comprising refining, using the confidence map, the computed disparities to account for occluded pixels.

18. The method of claim 12 , further comprising estimating depths of objects within the scene based on the computed disparities and the confidence map.

19. The method of claim 12 , wherein obtaining the plurality of images of the scene comprises capturing a plurality of images using the plurality of imagers; and obtaining a plurality of lower resolution images.

20. The method of claim 12 , wherein identifying occluded pixels comprises identifying likely occlusion zones in the reference image.

21. The method of claim 12 , wherein identifying occluded pixels comprises:

calculating a photometric distance of corresponding pixels in a non-reference image along an epipolar line with a reference pixel in the reference image; and

determining that a corresponding pixel is occluded when its photometric distance exceeds a threshold.

22. The method of claim 12 , further comprising generating a depth map based on the disparities.

Assignments (8)