IP Library Granted Patent US 11,611,734
Granted Patent B2
US 11,611,734 · App. 16/830,573 · Granted Mar 21, 2023

Devices and methods for an imaging system with a dual camera architecture

Inventors: Sam D'Amico (Stanford, CA); Xiaoyu Miao (Sunnyvale, CA); Babak Parviz (Mountain View, CA)
Assignee: Google LLC
H04N13/239H04N5/265H04N13/243H04N23/45
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,611,734
App. No.
16/830,573
Granted
Mar 21, 2023
Kind
B2
Abstract

An electronic device can include a first image sensor configured to capture a first image of a field of view and a second image sensor configured to capture a second image of the field of view. The electronic device can include a color filter adjacent to the second image sensor such that the field of view is viewable by the second image sensor through the color filter. The first image can have a first pixel resolution. The second image can have a second pixel resolution. The electronic device can include a controller configured to determine a third image based on luminance content of the first image and color content of the second image. The third image can have a third pixel resolution indicative of a spatial resolution of the first image and a spectral resolution of the second image.

Claims (43)

1. A device comprising:

a first camera that captures, via a first aperture at a first location of the device, a first image of a field of view, wherein the first image is a grayscale image;

a second camera that captures, via a second aperture at a second location of the device, a second image of the field of view, wherein the second camera comprises a color filter configured to selectively allow different wavelengths of light received via the second aperture to illuminate respective different sets of pixels of the second camera, wherein the second image is a color image including a representation of a color content of the field of view, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;

a display; and

a controller, coupled to the first camera and the second camera, wherein the controller is configured to:

determine a luminance content of the second image;

based on the luminance content of the second image being greater than a threshold, incorporate the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and

cause the display to display the third image.

2. The device of claim 1 , further comprising:

a first lens that focuses light from the field of view into the first camera; and

a second lens that focuses light from the field of view into the second camera.

3. The device of claim 2 , further comprising:

a first image sensor included in the first camera, wherein the first image sensor receives at least a portion of the light focused by the first lens; and

a second image sensor included in the second camera, wherein the second image sensor receives at least a portion of the light focused by the second lens.

4. The device of claim 1 , wherein the color filter includes at least one of a Bayer filter, an RGBE filter, a CYYM filter, a CYGM filter, or an RGBW filter.

5. The device of claim 1 , wherein the greyscale image includes a representation of luminance content in the field of view, and wherein the third image includes luminance content of the first image and color content of the second image.

6. The device of claim 1 , wherein the first image has a first pixel resolution indicative of a spatial resolution of the first image, and wherein the second image has a second pixel resolution indicative of a spectral resolution of the second image.

7. The device of claim 6 , wherein the third image has a third pixel resolution indicative of the spatial resolution of the first image and the spectral resolution of the second image.

8. The device of claim 6 , wherein the first pixel resolution of the first image is greater than the second pixel resolution of the second image.

9. The device of claim 8 , wherein the third image has a third pixel resolution that is substantially same as the first pixel resolution of the first image.

10. The device of claim 1 , wherein the controller aligns features in the first image with corresponding features in the second image to obtain the third image.

11. A method comprising:

operating, by a computing device, a first camera to capture, via a first aperture at a first location, a first image of a field of view, wherein the first image is a grayscale image;

operating, by the computing device, a second camera, comprising a color filter, that captures, via a second aperture at a second location, a second image of the field of view, wherein the second image is a color image including a representation of a color content of the field of view at multiple different colors or wavelengths, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;

determining, by the computing device, a luminance content of the second image;

based on the luminance content of the second image being greater than a threshold, incorporating, by the computing device, the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and

causing, by the computing device, a display to display the third image.

12. The method of claim 11 , wherein the color filter includes at least one of a Bayer filter, an RGBE filter, a CYYM filter, a CYGM filter, or an RGBW filter.

13. The method of claim 11 , wherein the grayscale image includes a representation of luminance content in the field of view, and wherein the third image includes luminance content of the first image and color content of the second image.

14. The method of claim 11 , wherein the first image has a first pixel resolution indicative of a spatial resolution of the first image, and wherein the second image has a second pixel resolution indicative of a spectral resolution of the second image.

15. The method of claim 14 , wherein the third image has a third pixel resolution indicative of the spatial resolution of the first image and the spectral resolution of the second image.

16. The method of claim 14 , wherein the first pixel resolution of the first image is greater than the second pixel resolution of the second image.

17. The method of claim 16 , wherein the third image has a third pixel resolution that is substantially same as the first pixel resolution of the first image.

18. The method of claim 11 , further comprising:

aligning, by the computing device, features in the first image with corresponding features in the second image to obtain the third image.

19. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:

causing a first camera to capture, via a first aperture at a first location, a first image of a field of view, wherein the first image is a grayscale image;

causing a second camera that comprises a color filter to capture, via a second aperture at a second location, a second image of the field of view, wherein the second image is a color image including a representation of a color content of the field of view at multiple different colors or wavelengths, and wherein the first location and second location are separated by a distance such that the contents of the first image and second image differ due to the parallax effect;

determine a luminance content of the second image;

based on the luminance content of the second image being greater than a threshold, incorporating the color content obtained from the color image into the grayscale image obtained from the first camera to obtain a third image; and

causing a display to display the third image.

20. The non-transitory computer readable medium of claim 19 , wherein the operations further comprise:

aligning features in the first image with corresponding features in the second image to obtain the third image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2023
From: D'AMICO, SAM; MIAO, XIAOYU; PARVIZ, BABAK
To: GOOGLE INC.
Reel/Frame 062572/0851 →
CHANGE OF NAME Recorded Feb 2, 2023
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 062634/0845 →
Continuity (4)
Continuation 15970887 · May 4, 2018
Continuation 15265867 · Sep 15, 2016
Continuation 13960920 · Aug 7, 2013
Related Publication 20200228781A1 · Jul 16, 2020