IP Library Granted Patent US 9,124,831
Granted Patent B2
US 9,124,831 · App. 14/203,308 · Granted Sep 1, 2015

System and methods for calibration of an array camera

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,124,831
App. No.
14/203,308
Granted
Sep 1, 2015
Kind
B2
Abstract

Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the detecting of defects in the imaging components of the array camera and determining whether the detected defects may be tolerated by image processing algorithms. The calibration process also determines translation information between imaging components in the array camera for use in merging the image data from the various imaging components during image processing. Furthermore, the calibration process may include a process to improve photometric uniformity in the imaging components.

Claims (57)

1. A method for calibrating an array camera using a processing system wherein the array camera includes a plurality of imaging components where the plurality of imaging components includes a reference imaging component and a plurality of associate imaging components associated with the reference imaging component, the method comprising:

performing a defect detection process that includes:

identifying pixel defects in each of the plurality imaging components, and

determining whether pixel defects in a particular imaging component may be corrected in image processing using image data from at least one other one of the plurality of imaging components including:

capturing a first set of image data of a field target under high intensity lighting with each of the plurality of imaging components;

capturing a second set of image data of a field target under low intensity lighting;

analyzing the first and second sets of image data to identify each pixel having an unexpected response value in each imaging component;

flagging the identified pixels with a category indicating the category unexpected response received for each imaging component;

selecting a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances;

determining a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera;

determining whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect; and

generating a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other of the plurality of imaging components intersect the parallax translation path of the selected defect; and

performing a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.

2. The method of claim 1 further comprising performing a Modulation Transfer Function (MTF) calibration process on each of the plurality of imaging components in the array camera that includes determining MTF data over the Field of View (FoV) or the Field Height (FH) for each of the imaging components.

3. The method of claim 2 wherein the MTF calibration process further comprises:

capturing image data of a target with each of the plurality imaging components;

analyzing the image data captured by the plurality of imaging components to determine MTF data over one of a FoV and FH for each of the imaging components wherein the MTF data for each of the imaging components is generated from the MTF detected in the image data; and

storing the MTF data for each of the imaging components.

4. The method of claim 1 further comprising performing a photometric calibration process of each of the plurality of imaging components in the array camera that includes determining image data adjustments for each of the imaging components that provide image data from each of the imaging components with a uniform intensity profile.

5. The method of claim 4 wherein the performing the photometric calibration comprises:

capturing image data of a flat field target under controlled lighting conditions with each of the plurality of imaging components in the array camera;

analyzing the image data of each of the imaging components to determine a per pixel adjustment needed to match cos ^4 fall-off response for an image; and

generating a grid of adjustments for each of the plurality of imaging components wherein each adjustment in the grid indicates the adjustment information for data from a set of pixels in the particular imaging component needed to match the cos ^4 fall-off response for an image.

6. The method of claim 1 wherein the identifying of defects in each of the imaging components comprises:

storing the identified pixels with the flagged categories.

7. The method of claim 6 wherein the detecting of defects in each imaging component further comprises:

determining the number of each category of defects identified in each imaging component;

comparing the number of each category of defects to a threshold; and

generating a disposition output for a particular imaging component in response to the number of defects in a category for the particular imaging component being greater than the threshold.

8. A device for calibrating an array camera including a plurality of imaging components where the plurality of imaging components include a reference imaging component and a plurality of associate imaging components associated with the reference imaging component comprising:

a memory; and

a processor configured via one or more applications stored in the memory to:

perform a defect detection process that includes:

identify pixel defects in each of the plurality imaging components, and

determine whether pixel defects in a particular imaging component may be corrected in image processing including:

capture a first set of image data of a field target under high intensity lighting with each of the plurality of imaging components;

capture a second set of image data of a field target under low intensity lighting;

analyze the first and second sets of image data to identify each pixel having an unexpected response value in each imaging component;

flag the identified pixels with a category indicating the category unexpected response received for each imaging component;

select a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances;

determine a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera;

determine whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect; and

generate a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other of the plurality of imaging components intersect the parallax translation path of the selected defect; and

perform a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.

9. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process for calibrating an array camera including a plurality of imaging components where the plurality of imaging components include a reference imaging component and a plurality of associate imaging components associated with the reference imaging component, the process comprising:

performing a defect detection process that includes:

identifying pixel defects in each of the plurality imaging components, and

determining whether pixel defects in a particular imaging component may be corrected in image processing including:

capturing a first set of image data of a field target under high intensity lighting with each of the plurality of imaging components;

capturing a second set of image data of a field target under low intensity lighting;

analyzing the first and second sets of image data to identify each pixel having an unexpected response value in each imaging component;

flagging the identified pixels with a category indicating the category unexpected response received for each imaging component;

selecting a defect in a first one of the plurality of imaging components in the array camera and determining a parallax translation path of the defect over a range of supported distances;

determining a parallax translation path over the range of supported distances from defects in other ones of the plurality of imaging components of the array camera;

determining whether a predetermined number of parallax translation paths of defects in other ones of the plurality of imaging components intersect the parallax translation path of the selected defect; and

generating a disposition output for the first one of the imaging components having the selected defect if a predetermined number of parallax translation paths for defects from other of the plurality of imaging components intersect the parallax translation path of the selected defect; and

performing a geometric calibration process that includes determining translation information that relates image data from particular pixels in a reference imaging component to image data from corresponding pixels in each associate imaging component associated with the reference imaging component.

Assignments (13)
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2018
From: FOTONATION CAYMAN LIMITED
To: FOTONATION LIMITED
Reel/Frame 046539/0815 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2016
From: PELICAN IMAGING CORPORATION
To: FOTONATION CAYMAN LIMITED
Reel/Frame 040675/0025 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2016
From: KIP PELI P1 LP
To: PELICAN IMAGING CORPORATION
Reel/Frame 040674/0677 →
CHANGE OF NAME Recorded Oct 19, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE SPECIAL OPPORTUNITIES FUND LP
Reel/Frame 040494/0930 →
CHANGE OF NAME Recorded Oct 19, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE SPECIAL OPPORTUNITIES FUND LP
Reel/Frame 040423/0725 →
SECURITY INTEREST Recorded Jun 13, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE OPPORTUNITIES FUND LP
Reel/Frame 039117/0345 →
SECURITY INTEREST Recorded Jun 13, 2016
From: DBD CREDIT FUNDING LLC
To: DRAWBRIDGE OPPORTUNITIES FUND LP
Reel/Frame 038982/0151 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR AND ASSIGNEE PREVIOUSLY RECORDED AT REEL: 037565 FRAME: 0439. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Jan 25, 2016
From: KIP PELI P1 LP
To: DBD CREDIT FUNDING LLC
Reel/Frame 037591/0377 →
SECURITY INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: KIP PELI P1 LP
Reel/Frame 037565/0439 →
SECURITY INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: DBD CREDIT FUNDING LLC
Reel/Frame 037565/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2016
From: PELICAN IMAGING CORPORATION
To: KIP PELI P1 LP
Reel/Frame 037565/0385 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2014
From: MULLIS, ROBERT
To: PELICAN IMAGING CORPORATION
Reel/Frame 032397/0561 →