HETEROGENEOUS MIX OF SENSORS AND CALIBRATION THEREOF
Aspects of calibration of sensors are described. In one embodiment, a characteristic associated with a first or a second sensor is identified. The characteristic may be identified before or after assembly of a device including the first and second sensors. In turn, an operating characteristic of at least one of the first sensor or the second sensor may be adjusted based on the identified characteristic. Further, a first image may be captured with the first sensor and a second image may be captured with the second sensor. An attribute of the second image, for example, may be adjusted to substantially address any difference between the attribute of the second image and a corresponding attribute of the first image using the characteristic for calibration. The adjustments described herein may assist various processing techniques which operate on pairs of images, for example, particularly when a heterogeneous mix of sensors is used.
1 . An image processing method, comprising:
identifying a characteristic for calibration associated with at least one of a first sensor or a second sensor, a pixel density of the second sensor being a fraction of the pixel density of the first sensor;
capturing a first image with the first sensor;
capturing a second image with a second sensor; and
adjusting, with a processing circuit, an attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the first image using the characteristic for calibration.
2 . The method of claim 1 , further comprising, before adjusting the attribute of the second image, adapting an operating characteristic of at least one of the first sensor or the second sensor to accommodate for at least one of noise, defective pixels, dark current, vignetting, demosaicing, or white balancing using the characteristic for calibration.
3 . The method of claim 1 , further comprising:
before adjusting the attribute of the second image, downscaling the first image to a downscaled first image, wherein:
adjusting the attribute of the second image comprises adjusting the attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the downscaled first image using the characteristic for calibration.
4 . The method of claim 1 , wherein adjusting the attribute of the second image comprises aligning the second image with the first image to substantially address a difference in alignment between the first sensor and the second sensor.
5 . The method of claim 1 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a limited-spectral component sensor; and
adjusting the attribute of the second image comprises normalizing values among the first image and the second image to substantially address a difference in sensitivity between the first sensor and the second sensor.
6 . The method of claim 1 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a luminance sensor; and
adjusting the attribute of the second image comprises normalizing luminance values among the first image and the second image to substantially address a difference in luminance sensitivity between the first sensor and the second sensor.
7 . The method of claim 6 , further comprising generating a high dynamic range image by combining luminance values of the second image with the first image.
8 . The method of claim 1 , further comprising:
generating a depth map for focusing the first image, the depth map including a mapping among relative depth values in a field of view based on a difference between pixels of the first image and pixels of the second image; and
generating an edge map by identifying edges in at least one of the first image or the second image.
9 . An image processing device, comprising:
a first sensor and a second sensor;
a memory coupled to the first sensor and the second sensor; and
a processing circuit coupled to the memory and configured to:
identify a characteristic for calibration associated with at least one of the first sensor or the second sensor;
capture a first image with the first sensor and capture a second image with the second sensor; and
adjust an attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the first image using the characteristic for calibration.
10 . The image processing device of claim 9 , wherein the processing circuit is further configured to, before adjusting the attribute of the second image, adapt an operating characteristic of at least one of the first sensor or the second sensor to accommodate for at least one of noise, defective pixels, dark current, vignetting, demosaicing, or white balancing using the characteristic for calibration.
11 . The image processing device of claim 9 , wherein the processing circuit is further configured to:
before adjusting the attribute of the second image, downscale the first image to a downscaled first image; and
adjust the attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the downscaled first image using the characteristic for calibration.
12 . The image processing device of claim 9 , wherein the processing circuit is further configured to align the second image with the first image to substantially address a difference in alignment between the first sensor and the second sensor.
13 . The image processing device of claim 9 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a limited-spectral component sensor; and
the processing circuit is further configured to normalize values among the first image and the second image to substantially address a difference in sensitivity between the first sensor and the second sensor.
14 . The image processing device of claim 9 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a luminance sensor; and
the processing circuit is further configured to normalize luminance values among the first image and the second image to substantially address a difference in luminance sensitivity between the first sensor and the second sensor.
15 . The image processing device of claim 9 , wherein the processing circuit is further configured to:
generate a depth map for focusing the first image, the depth map including a mapping among relative depth values in a field of view based on a difference between pixels of the first image and pixels of the second image; and
generate an edge map by identifying edges in at least one of the first image or the second image.
16 . An image processing method, comprising:
identifying a characteristic for calibration associated with at least one of a first sensor or a second sensor;
adapting an operating characteristic of at least one of the first sensor or the second sensor;
capturing a first image with the first sensor and capturing a second image with the second sensor; and
adjusting, with a processing circuit, an attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the first image using the characteristic for calibration.
17 . The method of claim 16 , further comprising:
before adjusting the attribute of the second image, downscaling the first image to a downscaled first image, wherein:
adjusting the attribute of the second image comprises adjusting the attribute of the second image to substantially address a difference between the attribute of the second image and a corresponding attribute of the downscaled first image using the characteristic for calibration.
18 . The method of claim 16 , wherein adjusting the attribute of the second image comprises aligning the second image with the first image to substantially address a difference in alignment between the first sensor and the second sensor.
19 . The method of claim 16 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a limited-spectral component sensor; and
adjusting the attribute of the second image comprises normalizing values among the first image and the second image to substantially address a difference in sensitivity between the first sensor and the second sensor.
20 . The method of claim 16 , wherein:
the first sensor comprises a multi-spectral component sensor and the second sensor comprises a luminance sensor; and
adjusting the attribute of the second image comprises normalizing luminance values among the first image and the second image to substantially address a difference in luminance sensitivity between the first sensor and the second sensor.