IP Library Granted Patent US 9,917,998
Granted Patent B2
US 9,917,998 · App. 14/773,748 · Granted Mar 13, 2018

Systems and methods for measuring scene information while capturing images using array cameras

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,917,998
App. No.
14/773,748
Granted
Mar 13, 2018
Kind
B2
Abstract

Systems and methods for measuring scene information while capturing images using array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of measuring scene information while capturing an image using an array camera includes defining at least two subsets of active cameras, configuring the active cameras using image capture settings, capturing image data using the active cameras, synthesizing at least one image using image data captured by a first subset of active cameras, measuring scene information using image data captured by a second subset of active cameras, and determining whether the image capture settings satisfy at least one predetermined criterion for at least one image capture parameter using the measured scene information, where new image capture settings are determined and utilized to configure the active cameras upon a determination that the image capture settings do not satisfy the at least one predetermined criterion.

Claims (64)

1. A method of measuring scene information while capturing an image using an array camera, the method comprising:

defining a first subset and a second subset of active cameras in an array camera using a processor configured by software, wherein the first subset includes a first active camera, the second subset includes a second active camera different than the first active camera, and the first subset includes more than one active camera;

configuring the first subset and the second subset of active cameras in the array camera with image capture settings for each subset using the processor configured by software;

capturing image data using the active cameras in the array camera;

synthesizing at least one image from image data captured by the first subset of active cameras using the processor configured by software;

measuring scene information using the processor configured by software based upon image data captured by the second subset of active cameras, wherein measuring the scene information is performed independently of image data captured by the first subset of active cameras;

determining whether the image capture settings satisfy at least one predetermined criterion for at least one image capture parameter based upon the measured scene information obtained using the image data captured by the second subset of active cameras using the processor configured by software; and

when the image capture settings do not satisfy the at least one predetermined criterion for at least one image capture parameter, determine new image capture settings and configure the first subset of active cameras in the array camera, independently of the second subset of active cameras, with the new image capture settings using the processor configured by software.

2. The method of claim 1 , wherein a subset of active cameras is configured using image capture settings specific to the entire subset of active cameras.

3. The method of claim 1 , wherein a subset of active cameras is configured using image capture settings specific to each individual active camera within the subset.

4. The method of claim 1 , wherein synthesizing at least one image using image data captured by the first subset of active cameras comprises fusing image data captured by the cameras in the first subset of active cameras to create a high resolution image.

5. The method of claim 4 , wherein synthesizing at least one image using image data captured by the first subset of active cameras further comprises performing super resolution processing using the image data captured by the cameras in the first subset of active cameras to create a high resolution image.

6. The method of claim 1 , wherein the at least one predetermined criterion is at least one predetermined criterion for a parameter selected from the group consisting of exposure, focus settings, shutter speed, aperture, and light sensitivity.

7. The method of claim 1 , wherein the cameras in the second subset of active cameras are configured using different image capture settings and measure different scene information in parallel.

8. The method of claim 1 , wherein the new image capture settings are used to configure the first subset of active cameras to capture an additional set of image data used to synthesize an image and the new image capture settings for the second subset of active cameras configure the second set of active cameras to capture additional image data in order to measure additional information concerning the scene.

9. The method of claim 8 , wherein the new image capture settings for the second subset of active cameras are determined so that the first subset of cameras can smoothly transition to the image capture settings of any camera in the second subset of active cameras.

10. The method of claim 1 , wherein the new image capture settings are used to configure a set of active cameras to capture an additional set of image data used to synthesize an image and the set of active cameras is larger than the first subset of active cameras and includes at least one camera from the second subset of cameras.

11. The method of claim 1 , wherein:

the measured scene information comprises exposure measurements at each of the different image capture settings; and

the at least one predetermined criterion for at least one image capture parameter is at least one predetermined criterion with respect to auto-exposure.

12. The method of claim 1 , wherein:

the measured scene information comprises measurements of the dynamic range of the scene;

the at least one predetermined criterion for at least one image capture parameter comprises determination of image capture settings for high dynamic range image capture based upon measurements of the dynamic range of the scene; and

when the image capture settings satisfy the at least one predetermined criterion for at least one image capture parameter:

configure a plurality of subsets of active cameras using the processor configured by software, where each subset of active cameras has different image capture settings;

capture image data using the plurality of active cameras; and

synthesize at least one high dynamic range image from image data captured by the plurality of active cameras using the processor configured by software.

13. The method of claim 1 , wherein:

cameras in the array camera include autofocus mechanisms;

the measured scene information comprises focus measurements at each of the different image capture settings; and

the at least one predetermined criterion for at least one image capture parameter is at least one predetermined criterion with respect to autofocus.

14. The method of claim 1 , wherein the first subset of active cameras comprises a 3×3 array of active cameras patterned using a π filter group, where the π filter group includes:

a green camera at each corner,

a green reference camera in the center,

blue cameras on opposite sides of the reference camera, and

red cameras on opposite sides of the reference camera.

15. The method of claim 1 , wherein the array camera includes a camera module comprising an array of individual cameras.

16. The method of claim 15 , wherein the camera module comprises:

an imager array including an array of focal planes, where each focal plane comprises an array of light sensitive pixels;

an optic array including an array of lens stacks, where each lens stack creates an optical channel that forms an image on the array of light sensitive pixels within a corresponding focal plane;

wherein pairings of lens stacks and focal planes form multiple cameras including the plurality of active cameras.

17. The method of claim 16 , wherein the lens stacks within the optical channels sample the same object space with sub-pixel offsets to provide sampling diversity.

18. A method of measuring scene information while capturing an image using an array camera, the method comprising:

configuring a plurality of active cameras within an array camera with different image capture settings using a processor configured by software, the plurality of active cameras including a first subset and a second subset of active cameras, wherein the first subset includes a first active camera, and the second subset includes a second active camera different than the first active camera;

capturing image data using the active cameras in the array camera;

measuring scene information using the processor configured by software based upon image data captured by the second subset of active cameras in parallel with the first subset of active cameras using different image capture settings, wherein measuring the scene information is performed independently of image data captured by the first subset of active cameras;

determining whether image capture settings satisfy at least one predetermined criterion for at least one image capture parameter based upon the measured scene information obtained using the image data captured by the second subset of active cameras using the processor configured by software;

when the image capture settings do not satisfy the at least one predetermined criterion for at least one image capture parameter, determine a first set of new image capture settings and a second set of new image capture settings, configure the first subset of active cameras with the first set of new image capture settings, and configure the second subset of active cameras in the array camera with the second set of new image capture settings to make additional measurements of scene information using the processor configured by software; and

when the image capture settings satisfy the at least one predetermined criterion for at least one image capture parameter:

configure a plurality of active cameras in the array camera using the image capture settings using the processor configured by software;

capture image data using the plurality of active cameras; and

synthesize at least one image from image data captured by the plurality of active cameras using the processor configured by software.

19. The method of claim 18 , wherein:

the measured scene information comprises exposure measurements at each of the different image capture settings; and

the at least one predetermined criterion for at least one image capture parameter is at least one predetermined criterion with respect to auto-exposure.

20. The method of claim 18 , wherein:

the measured scene information comprises measurements of the dynamic range of the scene;

the at least one predetermined criterion for at least one image capture parameter comprises determination of image capture settings for high dynamic range image capture based upon measurements of the dynamic range of the scene;

configuring a plurality of active cameras using the image capture settings further comprises configuring a plurality of subsets of active cameras using the processor configured by software, where each subset of active cameras has different image capture settings; and

synthesizing at least one image from image data captured by the plurality of active cameras further comprises synthesizing at least one high dynamic range image from image data captured by the plurality of active cameras using the processor configured by software.

21. The method of claim 18 , wherein:

cameras in the array camera include autofocus mechanisms;

the measured scene information comprises focus measurements at each of the different image capture settings; and

the at least one predetermined criterion for at least one image capture parameter is at least one predetermined criterion with respect to autofocus.

Assignments (13)
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2018
From: FOTONATION CAYMAN LIMITED
To: FOTONATION LIMITED
Reel/Frame 046539/0815 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2016
From: PELICAN IMAGING CORPORATION
To: FOTONATION CAYMAN LIMITED
Reel/Frame 040675/0025 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2016
From: KIP PELI P1 LP
To: PELICAN IMAGING CORPORATION
Reel/Frame 040674/0677 →
CHANGE OF NAME Recorded Oct 19, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE SPECIAL OPPORTUNITIES FUND LP
Reel/Frame 040494/0930 →
CHANGE OF NAME Recorded Oct 19, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE SPECIAL OPPORTUNITIES FUND LP
Reel/Frame 040423/0725 →
SECURITY INTEREST Recorded Jun 13, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE OPPORTUNITIES FUND LP
Reel/Frame 039117/0345 →
SECURITY INTEREST Recorded Jun 13, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE OPPORTUNITIES FUND LP
Reel/Frame 038982/0151 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR AND ASSIGNEE PREVIOUSLY RECORDED AT REEL: 037565 FRAME: 0439. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Jan 25, 2016
From: KIP PELI P1 LP
To: DBD CREDIT FUNDING LLC
Reel/Frame 037591/0377 →
SECURITY INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: KIP PELI P1 LP
Reel/Frame 037565/0439 →
SECURITY INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: DBD CREDIT FUNDING LLC
Reel/Frame 037565/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: KIP PELI P1 LP
Reel/Frame 037565/0385 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 8, 2015
From: VENKATARAMAN, KARTIK; CIUREA, FLORIAN
To: PELICAN IMAGING CORPORATION
Reel/Frame 036514/0291 →