Granted Patent B2

US 12,254,644 · App. 17/705,674 · Granted Mar 18, 2025

Inventors: Ziv Attar (Los Altos, CA); George Douglas MacEwen (Mountain View, CA)

Assignee: LEICA CAMERA AG

G06T7/55G06T3/4053G06T5/50G06T2207/20221G06T2207/20228

Abstract

Claims (53)

1. A method of generating a digital image, the method comprising:

obtaining, via a first electronic sensor, a plurality of images of a target within a time period;

selecting one or more pixels in a first image of the plurality of images;

identifying corresponding pixels, that correspond to the one or more selected pixels, in one or more other images of the plurality of images, the one or more selected pixels and the corresponding pixels defining sets of reference pixels;

identifying two or more images of the plurality of images with optimal disparity according to the sets of reference pixels;

generating modified depth information via the identified two or more images;

generating a final digital image via the modified depth information;

wherein generating the modified depth information includes:

determining disparity information for the identified two or more images; and

combining the determined disparity information with depth information from a second electronic sensor;

wherein generating the modified depth information includes downscaling the two or more identified images; and

determining the disparity information for the identified two or more images includes determining disparities via the downscaled versions of the two or more identified images;

wherein generating the modified depth information includes upscaling the combined disparity and depth information;

wherein generating the modified depth information includes:

shifting pixels of an image of the identified two or more images according to the upscaled combined disparity and depth information; and

determining remaining disparity information after the shifting;

wherein generating the modified depth information includes:

combining the remaining disparity information with the depth information; and

upscaling the remaining combined disparity and depth information; and

wherein the modified depth information includes the upscaled remaining combined disparity and depth information.

2. The method of claim 1 , wherein the one or more selected pixels each include associated location information and color information.

3. The method of claim 2 , wherein the associated location information includes two-dimensional location information.

4. The method of claim 2 , wherein the associated location information includes three-dimensional location information.

5. The method of claim 1 , wherein identifying the two or more images includes determining two-dimensional disparities between respective pairs of the reference pixels.

6. The method of claim 1 , wherein identifying the two or more images includes determining three-dimensional disparities between respective pairs of the reference pixels.

7. The method of claim 1 , wherein the optimal disparity is a minimum disparity that can be utilized for distinguishing between depth levels.

8. The method of claim 7 , wherein the optimal disparity is less than a maximum threshold to filter out images with occlusion that exceeds an occlusion threshold and/or outlier images caused by dropping a mobile electronic device or a sensor malfunction.

9. The method of claim 1 , including obtaining depth information via a second electronic sensor;

wherein the depth information includes depth data and a confidence map; and

the modified depth information is different than the depth information obtained via the second electronic sensor.

10. The method of claim 1 , wherein the modified depth information includes a modified depth map and/or a modified disparity map.

11. The method of claim 1 , wherein generating the modified depth information includes filtering the combined disparity and depth information.

12. The method of claim 1 , wherein the remaining disparity information is determined via pixel kernels.

13. A mobile electronic device, comprising:

a first electronic sensor;

a second electronic sensor; and

an electronic control unit;

wherein the electronic control unit is configured to:

obtain, via the first electronic sensor, a plurality of images of a target within a time period;

select one or more pixels in a first image of the plurality of images;

identify corresponding pixels, that correspond to the one or more selected pixels, in one or more other images of the plurality of images, the one or more selected pixels and the corresponding pixels defining sets of reference pixels;

identify two or more images of the plurality of images with optimal disparity according to the sets of reference pixels;

generate modified depth information; and

generate a final digital image via the modified depth information;

wherein generating modified depth information includes:

determining disparity information for the identified two or more images;

combining the determined disparity information with depth information from the second electronic sensor;

upscaling the combined disparity and depth information;

shifting pixels of an image of the identified two or more images according to the upscaled combined disparity and depth information;

determining remaining disparity information after the shifting;

combining the remaining disparity information with the depth information; and

upscaling the remaining combined disparity and depth information; and

wherein the modified depth information includes the upscaled remaining combined disparity and depth information.

Assignments (1)

Continuity (2)

References Cited (331)

Cited By (1)

US 12,511,768