Granted Patent B2

US 12,631,941 · App. 18/625,802 · Granted May 19, 2026

Inventors: Micha Galor Gluskin (San Diego, CA); Ying Noyes (San Diego, CA); Wenbin Wang (San Diego, CA); Lee-Kang Liu (San Diego, CA); Balakrishna Mandadi (Hyderabad, IN); Yiqian Wang (San Diego, CA); Nan Cui (San Diego, CA)

Assignee: QUALCOMM Incorporated

G03B13/36G03B3/10H04N23/55H04N23/632H04N23/73

Abstract

Claims (64)

1 . A device for image capture, the device comprising:

a memory; and

one or more processors coupled to the memory and configured to:

receive, during a preview mode or a recording, a first image;

generate a first saliency map indicative of relative saliency of different regions within the first image, wherein the relative saliency of the different regions is indicative of a likelihood of attracting viewer gaze;

generate one or more additional images based on manipulating pixels in the first image;

generate one or more additional saliency maps indicative of relative saliency of different regions within the one or more additional images; and

determine, during the preview mode or the recording, a camera setting based on a comparison of at least the first saliency map to at least one of the additional saliency maps.

2 . The device of claim 1 , wherein the first image is generated with a lens of a camera at a first lens position, and wherein to determine the camera setting, the one or more processors are configured to determine an autofocus setting that defines a second lens position for the lens.

3 . The device of claim 2 , wherein the one or more processors are configured to:

determine that the second lens position and the first lens position is the same; and

avoid adjustment of a lens position of the lens.

4 . The device of claim 2 , wherein the one or more processors are configured to:

determine that the second lens position and the first lens position is different; and

adjust a lens position of the lens to the second lens position.

5 . The device of claim 1 , wherein to generate the one or more additional images based on manipulating pixels in the first image, the one or more processors are configured to generate the one or more additional images based on depth of image content in the first image.

6 . The device of claim 1 , wherein to generate the one or more additional images, the one or more processors are configured to generate the one or more additional images based on manipulating pixels of objects in a foreground of the first image.

7 . The device of claim 1 , wherein to determine the camera setting, the one or more processors are configured to determine the camera setting based on one or more of performing a cross correlation, a sum of absolute difference process, or a mean square error process between the first saliency map and the at least one of the additional saliency maps.

8 . The device of claim 1 , wherein to determine the camera setting based on the comparison of at least the first saliency map to at least one of the additional saliency maps, the one or more processors are configured to:

determine that the first saliency map and the one or more additional saliency maps are substantially the same; and

determine an autofocus setting based on regions having relative saliency in the first saliency map and the one or more additional saliency maps.

9 . The device of claim 1 , wherein to determine the camera setting based on the comparison of at least the first saliency map to at least one of the additional saliency maps, the one or more processors are configured to:

determine that the first saliency map and the one or more additional saliency maps are not substantially the same;

determine foreground areas in the first image; and

determine an autofocus setting based on the foreground areas.

10 . The device of claim 1 , wherein the one or more additional images comprises a first additional image and a second additional image, and wherein to generate the one or more additional images based on manipulating pixels in the first image, the one or more processors are configured to:

manipulate pixels of the first image to generate the first additional image; and

manipulate pixels of the first additional image to generate the second additional image.

11 . The device of claim 1 , wherein to generate the one or more additional images, the one or more processors are configured to inpaint the first image to generate the one or more additional images.

12 . The device of claim 1 , wherein to generate the first saliency map, the one or more processors are configured to:

downscale the image to generate a N×M sized downscaled image; and

generate the first saliency map based on the downscaled image,

wherein a size of the first saliency map is X×Y, and

wherein at least one of X is less than N or Y is less than M.

13 . The device of claim 1 ,

wherein to generate the one or more additional images, the one or more processors are configured to simulate different exposures on the first image by changing tone of the first image to generate the one or more additional images,

wherein to generate the one or more additional saliency maps, the one or more processors are configured to generate the one or more additional saliency maps within the one or more additional images that are generated by simulating different exposures on the first image,

wherein the one or more processors are configured to generate a plurality of metering maps based on the first saliency map and the one or more additional saliency maps, and determine an updated metering map based on the plurality of metering maps, and

wherein to determine the camera setting, the one or more processors are configured to determine an autoexposure setting based on the updated metering map.

14 . The device of claim 1 , wherein to determine the camera setting, the one or more processors are configured to:

determine a most salient depth based on the comparison of at least the first saliency map to at least one of the additional saliency maps; and

determine the camera setting based on the determined most salient depth.

15 . The device of claim 1 , wherein the device is one or more of a digital camera, a digital video camcorder, or a camera-equipped wireless communication device handset.

16 . The device of claim 1 , wherein to generate the first saliency map, the one or more processors are configured to generate the first saliency map based on the first image, and wherein to generate the one or more additional saliency maps, the one or more processors are configured to generate the one or more additional saliency maps based on the one or more additional images.

17 . A method for image capture, the method comprising:

receiving, during a preview mode or a recording, a first image;

generating a first saliency map indicative of relative saliency of different regions within the first image, wherein the relative saliency of the different regions is indicative of a likelihood of attracting viewer gaze;

generating one or more additional images based on manipulating pixels in the first image;

generating one or more additional saliency maps indicative of relative saliency of different regions within the one or more additional images; and

determining, during the preview mode or the recording, a camera setting based on a comparison of at least the first saliency map to at least one of the additional saliency maps.

18 . The method of claim 17 , wherein the first image is generated with a lens of a camera at a first lens position, and wherein determining the camera setting comprises determining an autofocus setting that defines a second lens position for the lens.

19 . The method of claim 18 , further comprising:

determining that the second lens position and the first lens position is the same; and

avoiding adjustment of a lens position of the lens.

20 . The method of claim 18 , further comprising:

determining that the second lens position and the first lens position is different; and

adjusting a lens position of the lens to the second lens position.

21 . The method of claim 17 , wherein generating the first saliency map comprises generating the first saliency map based on the first image, and wherein generating the one or more additional saliency maps comprises generating the one or more additional saliency maps based on the one or more additional images.

22 . A computer-readable storage medium storing instructions thereon that when executed cause one or more processors to:

receive, during a preview mode or a recording, a first image;

generate a first saliency map indicative of relative saliency of different regions within the first image, wherein the relative saliency of the different regions is indicative of a likelihood of attracting viewer gaze;

generate one or more additional images based on manipulating pixels in the first image;

generate one or more additional saliency maps indicative of relative saliency of different regions within the one or more additional images; and

determine, during the preview mode or the recording, a camera setting based on a comparison of at least the first saliency map to at least one of the additional saliency maps.

Assignments (1)

Continuity (3)

References Cited (24)