IP Library Granted Patent US 9,369,646
Granted Patent B2
US 9,369,646 · App. 14/754,005 · Granted Jun 14, 2016

Sensor dark pixel offset estimation

Inventors: Chwen-Yuan Ku (San Jose, CA); H. Keith Nishihara (Los Altos, CA); M. Dirk Robinson (Menlo Park, CA)
Assignee: Skybox Imaging, Inc.
H04N5/361
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,369,646
App. No.
14/754,005
Granted
Jun 14, 2016
Kind
B2
Abstract

Examples of systems and methods to provide estimates of dark current for pixels of a photosensor as a function of the temperature of the sensor and the gain applied to the photosensor are described. In various implementations, the dark current estimated for each pixel can depend at least partly on a global scale factor and a global bias that depend on temperature and gain and a temperature-independent and gain-independent offset value for each pixel. The scale, bias, and offsets may be determined from multiple dark field images taken by the sensor over a range of operating temperatures. In some cases, the scale and bias can be determined using a subset of less than all the image pixels. Scale and bias derived for a particular sensor can be used in the calibration of different sensors.

Claims (71)

1. A computer-implemented method for correcting for dark current in a photosensor comprising a plurality of pixels, the method comprising:

under control of one or more physical computing devices configured with specific computer-executable instructions,

accessing, by the one or more physical computing devices, a scale factor and a bias factor for the plurality of pixels of the photosensor, wherein the plurality of pixels includes all the pixels of the photosensor, the scale factor and the bias factor depending at least partly on a temperature of the photosensor and a gain of the photosensor, and the scale factor and the gain factor both contributing to any dark current in the photosensor, the scale factor having the same value for each pixel in the plurality of pixels, and the bias factor having the same value for each pixel in the plurality of pixels;

for each pixel in the plurality of pixels of the photosensor:

accessing, by the one or more physical computing devices, a pixel value that represents a response of the pixel to receipt of light;

accessing, by the one or more physical computing devices, an offset value for the pixel, the offset value not dependent on the temperature or the gain of the photosensor;

determining, by the one or more physical computing devices, a corrected pixel value that depends at least partly on the pixel value, the scale factor, the bias factor, and the offset value, wherein the scale factor and the bias factor are global factors that are the same for each pixel in the plurality of pixels, and wherein the pixel value and the offset value are local factors capable of having values that are different for different pixels in the plurality of pixels; and

providing, by the one or more physical computing devices, the corrected pixel value.

2. The method of claim 1 , wherein the photosensor comprises a total number of pixels, the plurality of pixels comprises a first number of pixels, and the total number is greater than the first number.

3. The method of claim 2 , wherein at least some of the pixels of the photosensor are divided into multiple groups of pixels, with each of the multiple groups including a plurality of pixels, and the method of claim 1 is separately applied to each of the multiple groups of pixels using a respective scale factor and bias factor that applies to the group.

4. The method of claim 1 , wherein determining the corrected pixel value comprises subtracting an estimate of the dark current for the pixel from the pixel value, wherein the estimate of the dark current for the pixel is the sum of (i) the scale factor multiplied by the offset value and (ii) the bias factor.

5. The method of claim 1 , wherein the pixel value is a reset-corrected pixel value from the photosensor.

6. The method of claim 1 , wherein providing the corrected pixel value comprises storing the corrected pixel value in nontransitory computer storage or communicating the corrected pixel value via a transmission medium.

7. The method of claim 1 , wherein accessing a scale factor and a bias factor or accessing an offset value for the pixel comprises one or more of: accessing a lookup table or evaluating a mathematical function.

8. The method of claim 7 , wherein the mathematical function comprises a spline function or a polynomial function.

9. A system for correcting for dark current in a sensor comprising a plurality of pixels, the system comprising:

physical data storage configured to store:

(i) a scale factor applicable to the plurality of pixels, wherein the plurality of pixels includes all the pixels of the photosensor,

(ii) a bias factor applicable to the plurality of pixels, and

(iii) an offset value for each pixel in the plurality of pixels,

wherein the scale factor and the bias factor depend at least partly on a temperature of the sensor and a gain of the sensor, and the offset value does not depend on the temperature or the gain of the sensor, and the scale factor and the gain factor both contribute to any dark current in the photosensor; and

a physical computer processor in communication with the physical data storage, the physical computer processor configured to:

access the scale factor and the bias factor;

access a pixel value representing an output of a pixel in the plurality of pixels;

access the offset value for the pixel in the plurality of pixels; and

determine a corrected pixel value for the pixel, wherein the corrected pixel value depends at least partly on the pixel value, the scale factor, the bias factor, and the offset value, wherein the scale factor and the bias factor are global factors that are the same for each pixel in the plurality of pixels, and wherein the pixel value and the offset value are local factors capable of having values that are different for different pixels in the plurality of pixels.

10. The system of claim 9 , wherein the computer processor is configured to determine the corrected pixel value by subtracting from the pixel value (i) the scale factor multiplied by the offset value and (ii) the bias value.

11. The system of claim 9 , wherein to access the scale factor, the bias factor, or the offset value, the computer process is configured to access a lookup table or evaluate a mathematical function.

12. A sensor system comprising:

the system for correcting for dark current of claim 9 ; and

the sensor.

13. The sensor system of claim 12 , further comprising a temperature sensor configured to measure the temperature of the sensor.

14. An imaging system comprising the sensor system of claim 12 , wherein the sensor comprises a photosensor.

15. The imaging system of claim 14 , further comprising a temperature sensor configured to measure the temperature of the photosensor.

16. A computer-implemented method for estimating dark current in a sensor comprising a plurality of pixels, the method comprising:

under control of one or more computing devices configured with specific computer-executable instructions,

accessing, by the one or more computing devices, a dark current value for each of the plurality of pixels in the sensor;

fitting, by the one or more computing devices, the dark current values for the plurality of pixels to a dark current model, wherein the dark current model is based at least partly on:

a scale factor;

a bias factor; and

an offset,

wherein the scale factor and the bias factor depend at least partly on a temperature of the sensor, the scale factor has a first same value for all the pixels in the plurality of pixels, the bias factor has a second same value for all the pixels in the plurality of pixels, the offset is independent of the temperature of the sensor, and the offset has values that can be different for different pixels in the plurality of pixels; and

wherein the fitting first determines the scale factor and the bias factor, and then determines the offset value for a pixel in the plurality of pixels based at least in part on the scale factor, the bias factor, and the dark current value for the pixel; and

providing, by the one or more computing devices the scale factor, the bias factor, and the offset.

17. The method of claim 16 , wherein at least one of the scale factor or the bias factor further depends at least partly on a gain of the sensor.

18. The method of claim 16 , wherein the fitting is performed by optimizing an objective function based at least partly on a sum of squared differences between the dark current value and the dark current model for each of the pixels in the plurality of pixels.

19. The method of claim 16 , wherein the fitting determines the scale factor and the bias factor from fewer than all of the pixels in the sensor.

20. The method of claim 16 , wherein the dark current value for each of the pixels in the plurality of pixels is an average dark current value determined at least partly from a plurality of dark frame images taken by the sensor.

21. The method of claim 16 , wherein fitting comprises determining a lookup table for at least one of the scale factor, the bias factor, or the offset.

22. The method of claim 16 , wherein fitting comprises determining a mathematical function that represents at least one of the scale factor, the bias factor, or the offset.

23. The method of claim 16 , further comprising measuring the dark current value for each of the plurality of pixels in the sensor.

24. The method of claim 16 , wherein the plurality of pixels used for the fitting does not include pixels flagged as bad pixels.

25. The method of claim 24 , further comprising identifying the bad pixels by determining pixels whose dark current is above a threshold.

26. The method of claim 24 , further comprising:

excluding a first set of bad pixels for fitting the scale factor and the bias factor; and

excluding a second set of bad pixels for fitting the offset,

wherein the first set of bad pixels includes more bad pixels than the second set of bad pixels.

27. A system for estimating dark current in a sensor comprising a plurality of pixels, the system comprising:

physical data storage configured to store a dark current value for each of the plurality of pixels in the sensor; and

a physical computer processor in communication with the physical data storage, the physical computer processor configured to:

fit the dark current values for the plurality of pixels to a dark current model, wherein the dark current model is based at least partly on a scale factor, a bias factor, and an offset, the scale factor and the bias factor depending at least partly on a temperature of the sensor, the scale factor having a first same value for all the pixels in the plurality of pixels, the bias factor having a second same value for all the pixels in the plurality of pixels, the offset independent of the temperature of the sensor, and the offset having values that can be different for different pixels in the plurality of pixels, and wherein the fitting first determines the scale factor and the bias factor, and then determines the offset value for a pixel in the plurality of pixels based at least in part on the scale factor, the bias factor, and the dark current value for the pixel; and

output the dark current model to the physical data storage.

28. The system of claim 27 , wherein to fit the dark current values, the computer processor is configured to determine a lookup table or a functional fit to one or more of the scale factor, the bias factor, or the offset as at least a function of the temperature of the sensor.

29. A method for determining dark current in a second sensor based at least in part on a dark current model for a first sensor that is different from the second sensor, the dark current model for the first sensor comprising a scale factor and a bias factor, the scale factor and the bias factor depending at least partly on temperature, the scale factor and the bias factor each having a respective global value for all the pixels in the first sensor, the method comprising:

accessing the scale factor and the bias factor for the first sensor;

accessing dark current values for pixels of the second sensor;

determining, by execution of instructions by a physical computer processor, offset values for each of the pixels of the second sensor, the offset values based at least in part on the dark current values for the second sensor and the scale factor and the bias factor for the first sensor; and

providing a dark current model for the second sensor based at least partly on the offset values for each of the pixels of the second sensor and the scale factor and the bias factor for the first sensor.

30. The method of claim 29 , wherein at least one of the scale factor and the bias factor further depends at least partly on a gain of the first sensor.

31. The method of claim 29 , wherein providing a dark current model for the second sensor comprises communicating the dark current model to the second sensor.

32. The method of claim 29 , further comprising measuring the dark current values for pixels of the second sensor.

Assignments (9)
MERGER AND CHANGE OF NAME Recorded May 5, 2022
From: PLANET LABS INC.; PLANET LABS PBC
To: PLANET LABS PBC
Reel/Frame 059857/0587 →
RELEASE OF SECURITY INTEREST Recorded Dec 10, 2021
From: SILICON VALLEY BANK
To: PLANET LABS INC.; TERRA BELLA TECHNOLOGIES INC.; PL FOREIGN HOLDCO, INC.
Reel/Frame 058359/0501 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jun 21, 2019
From: PLANET LABS INC.; PL INTERMEDIATE TB, INC.; PLANET LABS TB, INC.; TERRA BELLA TECHNOLOGIES INC.; PLANET LABS LLC; PL FOREIGN HOLDCO, INC.
To: SILICON VALLEY BANK
Reel/Frame 049558/0515 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2017
From: TERRA BELLA TECHNOLOGIES, INC.
To: PLANET LABS, INC.
Reel/Frame 044261/0748 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT SERIAL NO. 15/061,851 PREVIOUSLY RECORDED AT REEL: 043277 FRAME: 0669. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 18, 2017
From: GOOGLE INC.
To: PLANET LABS TB, INC.
Reel/Frame 043661/0060 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF THE ASSIGNEE PREVIOUSLY RECORDED AT REEL: 043277 FRAME: 0669. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 17, 2017
From: GOOGLE INC.
To: PLANET LABS TB, INC.
Reel/Frame 043409/0837 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2017
From: GOOGLE INC.
To: PLANT LABS TB, INC.
Reel/Frame 043277/0669 →
CHANGE OF NAME Recorded Oct 24, 2016
From: SKYBOX IMAGING, INC.
To: TERRA BELLA TECHNOLOGIES INC.
Reel/Frame 040260/0433 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 29, 2015
From: KU, CHWEN-YUAN; NISHIHARA, H. KEITH; ROBINSON, M. DIRK
To: SKYBOX IMAGING, INC.
Reel/Frame 035929/0257 →
Continuity (3)
Continuation 13681712 · Nov 20, 2012
Provisional Application 61625322 · Apr 17, 2012
Related Publication 20150319385A1 · Nov 5, 2015