Granted Patent E1

US 50,836 · App. 18/139,154 · Granted Mar 17, 2026

Inventors: Shizhong Liu (San Diego, CA); Hailong Wang (San Diego, CA); Wan Shun Vincent Ma (San Diego, CA)

Assignee: QUALCOMM Incorporated

H04N23/69H04N23/45H04N23/58H04N23/667

Abstract

Claims (141)

1 . An apparatus configured to capture image data, the apparatus comprising:

a memory configured to store image data; and

one or more processors in communication with the memory, the one or more processors configured to:

cause a first image sensor to capture a first one or more frames of the image data using a first image capture mode in the case that a digital zoom level is less smaller than a first predefined binning transition threshold and greater than or equal to a first camera transition threshold, wherein the first image capture mode includes a first binning level that uses two or more four pixels of the first image sensor for each output pixel of the image data, and wherein the first one or more frames of the image data have a first resolution, and wherein the first image sensor captures the first one or more frames of the image data using a first color array pattern;

determine that a first requested digital zoom level satisfies is greater than or equal to the first predefined binning transition threshold; and

cause the first image sensor to capture a second one or more frames of the image data using a second image capture mode, wherein the second image capture mode includes a second binning level that uses fewer pixels one pixel of the first image sensor relative to the first binning level for each output pixel of the image data, wherein the second one or more frames of the image data have the first resolutionand, wherein the first image sensor captures the second one or more frames of the image data using the first color array patternof the first one or more frames of the image data, and wherein there are no intermediate capture modes between the first image capture mode and the second image capture mode;

determine that a second requested zoom level is smaller than the first camera transition threshold;

cause a transition from a first camera comprising the first image sensor to a second camera comprising a second image sensor, wherein the second camera has a different optical zoom level or a different effective focal length than the first camera; and

cause the second image sensor to capture third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses one pixel of the second image sensor for each output pixel of the image data,

wherein the second image sensor comprises a same number of pixels as a number of pixels output by the first image sensor in the first image capture mode.

2 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

determine that the a third requested digital zoom level satisfies a second camera transition threshold, the second camera transition threshold being greater than the first binning transition threshold;

cause a transition from a the first camera comprising the first image sensor to a second third camera comprising a third image sensor, wherein the third camera has a different optical zoom level or a different effective focal length than the first camera and the second camera; and

cause the second camera third image sensor to capture a third fourth one or more frames of the image data using the second a fourth image capture mode, wherein the second fourth image capture mode further includes using a fourth binning level that uses one or more pixels pixel of an the third image sensor that corresponds to the second camera for each output pixel of the image data.

3 . The apparatus of claim 2 , wherein the second image sensor and the third image sensor that corresponds to the second camera comprises the first image sensor comprise the same number of pixels.

4 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

determine that the requested digital zoom level satisfies a camera transition threshold;

cause a transition from a first camera comprising the first image sensor to a second camera;

cause the second camera to capture a third one or more frames of the image data using the first image capture mode in the case that the requested digital zoom level is less than a second predefined threshold, wherein the first image capture mode further includes using two or more pixels of an image sensor that corresponds to the second camera for each output pixel of the image data;

determine that a second requested digital zoom level satisfies the second predefined threshold; and

cause the second camera to capture a fourth one or more frames of the image data using the second image capture mode, wherein the second image capture mode further includes using fewer pixels of the image sensor that corresponds to the second camera relative to the first image capture mode for each output pixel of the image data second image sensor and the third image sensor are 12MP image sensors.

5 . The apparatus of claim 1 , wherein to cause the first image sensor to capture the first one or more frames of the image data using the first image capture mode, the one or more processors are configured to cause the first image sensor to combine four pixels of the first image sensor for each output pixel of the image data, and

wherein to cause the first image sensor to capture the second one or more frames of the image data using the second image capture mode, the one or more processors are configured to cause the first image sensor to use one pixel of the first image sensor for each output pixel of the image data.

6 . The apparatus of claim 5 1 , wherein the one or more processors are further configured to:

cause the first image sensor to output a reduced portion of output pixels for each of the second one or more frames captured using the second image capture mode, wherein the reduced portion corresponds to a center twenty-five percent of the output pixels for each of the second one or more frames captured using the second image capture mode.

7 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

cause the first image sensor to output a reduced portion of output pixels for each of the second one or more frames captured using the second image capture mode; and

receive, from the first image sensor, the reduced portion of output pixels for the second one or more frames captured using the second image capture mode.

8 . The apparatus of claim 7 , wherein the reduced portion comprises a predefined percentage of a full field of view of the first image sensor, wherein the full field of view comprises all pixels of the first image sensor.

9 . The apparatus of claim 1 , wherein the first image capture mode provides a lower pixel density compared to that of the second image capture mode.

10 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

determine that the requested digital zoom level satisfies a third predefined threshold; and

cause the first image sensor to capture a third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses fewer pixels of the first image sensor relative to the second binning level for each output pixel of the image data.

11 . The apparatus of claim 10 , wherein to cause the first image sensor to capture the first one or more frames of the image data using the first image capture mode, the one or more processors are configured to cause the first image sensor to combine at least five pixels of the first image sensor for each output pixel of the image data, wherein to cause the first image sensor to capture the second one or more frames of the image data using the second image capture mode, the one or more processors are configured to cause the first image sensor to use four pixels of the first image sensor for each output pixel of the image data, and

wherein to cause the first image sensor to capture the third one or more frames of the image data using the third image capture mode, the one or more processors are configured to cause the first image sensor to use one pixel of the first image sensor for each output pixel of the image data.

12 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

receive the output pixels from the first image sensor; and

perform digital cropping to achieve the first requested digital zoom level using the output pixels.

13 . The apparatus of claim 1 , wherein the one or more processors are further configured to:

determine that the first requested digital zoom level satisfies is greater than or equal to an upsampling threshold, the upsampling threshold corresponding to a digital zoom level that is less being smaller than the first predefined binning transition threshold, and wherein the upsampling threshold is based at least in part on a desired resolution parameter; and

perform upsampling to achieve the first requested digital zoom level using the output pixels.

14 . The apparatus of claim 1 , further comprising:

the first image sensor; and

a display.

15 . A method of capturing image data comprising:

causing a first image sensor to capture a first one or more frames of image data using a first image capture mode in the case that a digital zoom level is less smaller than a first predefined binning transition threshold and greater than or equal to a first camera transition threshold, wherein the first image capture mode includes a first binning level that uses two or more four pixels of the first image sensor for each output pixel of the image data, and wherein the first one or more frames of the image data have a first resolution, and wherein the first image sensor captures the first one or more frames of the image data using a first color array pattern;

determining that a first requested digital zoom level satisfies is greater than or equal to the first predefined binning transition threshold; and

causing the first image sensor to capture a second one or more frames of the image data using a second image capture mode, wherein the second image capture mode includes a second binning level that uses fewer pixels one pixel of the first image sensor relative to the first binning level for each output pixel of the image data, wherein the second one or more frames of the image data have the first resolutionand, wherein the first image sensor captures the second one or more frames of the image data using the first color array patternof the first one or more frames of the image data, and wherein there are no intermediate capture modes between the first image capture mode and the second image capture mode;

determining that a second requested zoom level is smaller than the first camera transition threshold;

causing a transition from a first camera comprising the first image sensor to a second camera comprising a second image sensor, wherein the second camera has a different optical zoom level or a different effective focal length than the first camera; and

causing the second image sensor to capture third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses one pixel of the second image sensor for each output pixel of the image data,

wherein the second image sensor comprises a same number of pixels as a number of pixels output by the first image sensor in the first image capture mode.

16 . The method of claim 15 , further comprising:

determining that the a third requested digital zoom level satisfies a second camera transition threshold, the second camera transition threshold being greater than the first binning transition threshold;

causing a transition from a the first camera to a second third camera comprising a third image sensor, wherein the third camera has a different optical zoom level or a different effective focal length than the first camera and the second camera; and

causing the second camera third image sensor to capture a third fourth one or more frames of the image data using the second a fourth image capture mode, wherein the second fourth image capture mode further includes using a fourth binning level that uses one or more pixels pixel of an the third image sensor corresponding to the second camera for each output pixel of the image data.

17 . The method of claim 16 , wherein the number of pixels of the second image sensor and the third image sensor corresponding to the second camera that are used for each output pixel is based at least in part on a difference between a total number of pixels of the first image sensor and a total number of pixels of the image sensor corresponding to the second camera comprise the same number of pixels.

18 . The method of claim 15 , further comprising:

causing the first image sensor to output a reduced portion of output pixels for each of the second one or more frames captured using the second image capture mode; and

receiving, from the first image sensor, the reduced portion of output pixels for the second one or more frames captured using the second image capture mode.

19 . The method of claim 15 , further comprising:

determining that the requested digital zoom level satisfies a third predefined threshold;

and causing the first image sensor to capture a third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses fewer pixels of the first image sensor relative to the second binning level for each output pixel of the image data.

20 . The method of claim 19 , wherein causing the first image sensor to capture the first one or more frames of the image data using the first image capture mode comprises causing the first image sensor to combine at least five pixels of the first image sensor for each output pixel of the image data, wherein causing the first image sensor to capture the second one or more frames of the image data using the second image capture mode comprises causing the first image sensor to use four pixels of the first image sensor for each output pixel of the image data, and

wherein causing the first image sensor to capture the third one or more frames of the image data using the third image capture mode comprises causing the first image sensor to use one pixel of the first image sensor for each output pixel of the image data.

21 . The method of claim 15 , further comprising:

receiving the output pixels from the first image sensor; and

performing digital cropping to achieve the first requested digital zoom level using the output pixels.

22 . The method of claim 15 , further comprising:

determining that the first requested digital zoom level satisfies is greater than or equal to an upsampling threshold, the upsampling threshold corresponding to a requested digital zoom level that is less being smaller than the first predefined binning transition threshold, and wherein the upsampling threshold is based at least in part on a desired resolution parameter; and

performing upsampling to achieve the first requested digital zoom level using the output pixels.

23 . An apparatus configured to capture image data, the apparatus comprising:

means for causing an a first image sensor to capture a first one or more frames of image data using a first image capture mode in the case that a digital zoom level is less smaller than a first predefined binning transition threshold and greater than or equal to a first camera transition threshold, wherein the first image capture mode includes a first binning level that uses two or more four pixels of an the first image sensor for each output pixel of the image data, and wherein the first one or more frames of the image data have a first resolution, and wherein the first image sensor captures the first one or more frames of the image data using a first color array pattern;

means for determining that a first requested digital zoom level satisfies is greater than or equal to the first predefined binning transition threshold; and

means for causing the first image sensor to capture a second one or more frames of the image data using a second image capture mode, wherein the second image capture mode includes a second binning level that uses fewer pixels one pixel of the first image sensor relative to the first binning level for each output pixel of the image data, wherein the second one or more frames of the image data have the first resolutionand, wherein the first image sensor captures the second one or more frames of the image data using the first color array patternof the first one or more frames of the image data, and wherein there are no intermediate capture modes between the first image capture mode and the second image capture mode;

means for determining that a second requested zoom level is smaller than the first camera transition threshold;

means for causing a transition from a first camera comprising the first image sensor to a second camera comprising a second image sensor, wherein the second camera has a different optical zoom level or a different effective focal length than the first camera; and

means for causing the second image sensor to capture third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses one pixel of the second image sensor for each output pixel of the image data,

wherein the second image sensor comprises a same number of pixels as a number of pixels output by the first image sensor in the first image capture mode.

24 . The apparatus of claim 23 , the apparatus further comprising:

means for receiving the output pixels from the first image sensor; and

means for performing digital cropping to achieve the first requested digital zoom level using the output pixels.

25 . The apparatus of claim 23 , wherein the means for causing the image sensor to capture the first one or more frames of the image data using the first image capture mode comprises means for causing the image sensor to combine four pixels of the image sensor for each output pixel of the image data, and

wherein the means for causing the image sensor to capture the second one or more frames of the image data using the second image capture mode comprises means for causing the image sensor to use one pixel of the image sensor for each output pixel of the image data.

26 . The apparatus of claim 23 , further comprising:

means for causing the image sensor to output a reduced portion of output pixels for the second one or more frames of the image data; and

means for receiving, from the image sensor, the reduced portion of output pixels for the second one or more frames of the image data.

27 . A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to:

cause a first image sensor to capture a first one or more frames of image data using a first image capture mode in the case that a digital zoom level is less smaller than a first predefined binning transition threshold and greater than or equal to a first camera transition threshold, wherein the first image capture mode includes a first binning level that uses two or more four pixels of the first image sensor for each output pixel of the image data, and wherein the first one or more frames of the image data have a first resolution, and wherein the first image sensor captures the first one or more frames of the image data using a first color array pattern;

determine that a first requested digital zoom level satisfies is greater than or equal to the first predefined binning transition threshold; and

cause the first image sensor to capture a second one or more frames of the image data using a second image capture mode, wherein the second image capture mode includes a second binning level that uses fewer pixels one pixel of the first image sensor relative to the first binning level for each output pixel of the image data, wherein the second one or more frames of the image data have the first resolutionand, wherein the first image sensor captures the second one or more frames of the image data using the first color array patternof the first one or more frames of the image data, and wherein there are no intermediate capture modes between the first image capture mode and the second image capture mode;

determine that a second requested zoom level is smaller than the first camera transition threshold;

cause a transition from a first camera comprising the first image sensor to a second camera comprising a second image sensor, wherein the second camera has a different optical zoom level or a different effective focal length than the first camera; and

cause the second image sensor to capture third one or more frames of the image data using a third image capture mode, wherein the third image capture mode includes a third binning level that uses one pixel of the second image sensor for each output pixel of the image data,

wherein the second image sensor comprises a same number of pixels as a number of pixels output by the first image sensor in the first image capture mode.

28 . The non-transitory computer-readable storage medium of claim 27 , wherein the one or more processors are further configured to:

determine that the a third requested digital zoom level satisfies a second camera transition threshold, the second camera transition threshold being greater than the first binning transition threshold;

cause a transition from a the first camera to a second third camera comprising a third image sensor, wherein the third camera has a different optical zoom level or a different effective focal length than the first camera and the second camera; and

cause the second camera third image sensor to capture the second fourth one or more frames of the image data using the second a fourth image capture mode, wherein the second fourth image capture mode further includes using a fourth binning level that uses one or more pixels pixel of an the third image sensor that corresponds to the second camera for each output pixel of the image data.

29 . The non-transitory computer-readable storage medium of claim 27 , wherein the instructions, when executed, further cause one or more processors to at least:

receive the output pixels from the first image sensor; and

perform digital cropping to achieve the first requested digital zoom level using the output pixels.

30 . The non-transitory computer-readable storage medium of claim 27 , wherein the instructions, when executed, further cause one or more processors to at least:

cause the first image sensor to capture the first one or more frames of the image data using the first image capture mode by causing the first image sensor to combine four pixels of the first image sensor for each output pixel of the image data, and

cause the first image sensor to capture the second one or more frames of the image data using the second image capture mode by causing the first image sensor to use one pixel of the first image sensor for each output pixel of the image data.

31 . The non-transitory computer-readable storage medium of claim 27 , wherein the instructions, when executed, further cause one or more processors to at least:

cause the first image sensor to output a reduced portion of output pixels for each of the second one or more frames captured using the second image capture mode; and

receive, from the first image sensor, the reduced portion of output pixels for the second one or more frames captured using the second image capture mode.

32. The method of claim 15 , wherein the second image sensor and the third image sensor are 12MP image sensors.

33. The method of claim 15 , further comprising:

causing the first image sensor to output a reduced portion of pixels for each of the second one or more frames captured using the second image capture mode, wherein the reduced portion corresponds to a center twenty-five percent of the pixels for each of the second one or more frames captured using the second image capture mode.

34. The method of claim 18 , wherein the reduced portion comprises a predefined percentage of a full field of view of the first image sensor, wherein the full field of view comprises all pixels of the first image sensor.

35. The method of claim 15 , wherein the first image capture mode provides a lower pixel density compared to that of the second image capture mode.

36. The apparatus of claim 23 , further comprising:

means for determining that a third requested zoom level satisfies a second camera transition threshold, the second camera transition threshold being greater than the first binning transition threshold;

means for causing a transition from the first camera to a third camera comprising a third image sensor, wherein the third camera has a different optical zoom level or a different effective focal length than the first camera and the second camera;

means for causing the third image sensor to capture fourth one or more frames of the image data using a fourth image capture mode, wherein the fourth image capture mode includes a fourth binning level that uses one pixel of the third image sensor for each output pixel of the image data.

37. The apparatus of claim 36 , wherein the second image sensor and the third image sensor comprise the same number of pixels.

38. The apparatus of claim 23 , further comprising:

means for causing the first image sensor to output a reduced portion of pixels for each of the second one or more frames captured using the second image capture mode; and

means for receiving, from the first image sensor, the reduced portion of pixels for the second one or more frames captured using the second image capture mode.

39. The method of claim 38 , wherein the reduced portion comprises a predefined percentage of a full field of view of the first image sensor, wherein the full field of view comprises all pixels of the first image sensor.

40. The apparatus of claim 23 , further comprising:

means for determining that the first requested zoom level is greater than or equal to an upsampling threshold, the upsampling threshold being smaller than the first binning transition threshold, and wherein the upsampling threshold is based at least in part on a desired resolution parameter; and

means for performing upsampling to achieve the first requested zoom level using the output pixels.

41. The apparatus of claim 23 , wherein the second image sensor and the third image sensor are 12MP image sensors.

42. The apparatus of claim 23 , further comprising:

means for causing the first image sensor to output a reduced portion of pixels for each of the second one or more frames captured using the second image capture mode, wherein the reduced portion corresponds to a center twenty-five percent of the pixels for each of the second one or more frames captured using the second image capture mode.

43. The apparatus of claim 23 , wherein the first image capture mode provides a lower pixel density compared to that of the second image capture mode.

44. The non-transitory computer-readable storage medium of claim 28 , wherein the second image sensor and the third image sensor comprise the same number of pixels.

45. The non-transitory computer-readable storage medium of claim 27 , wherein the second image sensor and the third image sensor are 12MP image sensors.

46. The non-transitory computer-readable storage medium of claim 27 , wherein the instructions further cause the one or more processors to:

cause the first image sensor to output a reduced portion of pixels for each of the second one or more frames captured using the second image capture mode, wherein the reduced portion corresponds to a center twenty-five percent of the pixels for each of the second one or more frames captured using the second image capture mode.

47. The non-transitory computer-readable storage medium of claim 27 , wherein the instructions further cause the one or more processors to:

cause the first image sensor to output a reduced portion of pixels for each of the second one or more frames captured using the second image capture mode; and

receive, from the first image sensor, the reduced portion of pixels for the second one or more frames captured using the second image capture mode.

48. The non-transitory computer-readable storage medium of claim 47 , wherein the reduced portion comprises a predefined percentage of a full field of view of the first image sensor, wherein the full field of view comprises all pixels of the first image sensor.

49. The non-transitory computer-readable storage medium of claim 27 , wherein the first image capture mode provides a lower pixel density compared to that of the second image capture mode.

50. The non-transitory computer-readable storage medium of claim 27 , wherein instructions further cause the one or more processors to:

determine that the first requested zoom level is greater than or equal to an upsampling threshold, the upsampling threshold being smaller than the first binning transition threshold, and wherein the upsampling threshold is based at least in part on a desired resolution parameter; and

perform upsampling to achieve the first requested zoom level using the output pixels.

Assignments (1)

Continuity (2)

References Cited (20)