Sequential spectral imaging
One embodiment includes a sequential spectral imaging system with a color filter disposed over imaging sensor. The color filter includes zones of multiple color elements of discrete or continuous spectra. The color filter is configured to have multiple cycles of wavelength bands along diagonal lines of the imaging sensor, each cycle of wavelength bands includes a full spectra from red to blue. Another embodiment combines an imaging sensor of a wide FOV with pixelated color filters and a spectra sensor of smaller FOV. A calibration technique acquires imaging sensor's spectral response. The sequential spectral imaging system acquires a sequence of continuous frames of spatial and spectral data during recording an object moving relatively to the camera. Multiple frames of the moving object are tracked sequentially. Image processing to correct distortion and extract features enables identification and tracking of the object. The object's full spectra is established by connecting different frames.
1. A sequential spectral imaging camera, comprising:
an image sensor amounted on a camera, wherein the image sensor comprises a plurality of pixels sensitive to light signals;
a color filter disposed on the image sensor, wherein the color filter comprises a plurality of areas of continuously varying wavelengths based on locations on the color filter, wherein the image sensor acquires a sequence of image frames each associated with spectral, spatial, and temporal data of an object moving relatively to the camera, wherein one image frame corresponds to one temporal point;
a clock used to add temporal information to the sequence of image frames, for each pixel on the image sensor; and
a computer system capable of recording the sequence of image frames, comparing data of each pixel with spectral calibration data from a calibration module, synchronizing the temporal information from the clock, and reconstructing images of the object;
wherein the calibration module comprises a broadband light source to emit light in a range of wavelengths, a wavelength selective device capable of scanning and dividing the broadband light source into multiple wavelengths, and an integration sphere to project the light into the image sensor, and wherein the calibration module collects and saves the spectral calibration data of each pixel of the image sensor.
2. The sequential spectral imaging camera as in claim 1 , wherein the color filter comprises discrete wavelength regions from 300 nm to 1600 nm.
3. The sequential spectral imaging camera as in claim 1 , wherein the color filter comprises continuous wavelength regions from 300 nm to 1600 nm.
4. The sequential spectral imaging camera as in claim 1 , wherein the plurality of areas of continuously varying wavelengths have the wavelength arranged monotonically increasing along column and rows of the image sensor.
5. The sequential spectral imaging camera as in claim 1 , wherein the color filter comprises N×N zones of wavelength bands, wherein each said zone of the wavelength bands includes a full wavelength spectra, wherein the wavelength bands are monotonically arranged along a diagonal line of each said zone from blue to near IR, wherein N is an integer and N>1.
6. The sequential spectral imaging camera as in claim 5 , wherein the wavelength bands at edge of each zone have a same color as wavelength bands at a border in neighboring zones to smooth image transition of the object.
7. A sequential spectral imaging system as in claim 1 , wherein the image sensor is made of silicon, or germanium, or a combination of silicon and germanium.
8. A sequential spectral imaging system as in claim 1 , wherein the image sensor has a spectral response in the UV, visible and near IR and mid-IR range.
9. A method to operate a sequential spectral imaging camera, comprising:
providing an image sensor amounted on a camera, wherein the image sensor comprises a plurality of pixels sensitive to light signals; calibrating spectral data for each pixel of the image sensor;
disposing a color filter on the image sensor, wherein the color filter comprises a plurality of areas of continuously varying wavelengths based on locations on the color filter, wherein the image sensor acquires a sequence of image frames each associated with spectral, spatial, and temporal data of an object performing a relative motion to the camera;
adding temporal information from a clock to the sequence of image frames for each pixel on the image sensor;
tracking the spectral, spatial, temporal pixel data of the sequence of different image frames from a same object on the image sensor by a computer system;
processing data of each pixel with spectral calibration data from a calibration module, wherein the calibration module comprises a broadband light source to emit light in a range of wavelengths, a wavelength selective device capable of scanning and dividing the broadband light source into multiple wavelengths, and an integration sphere to project the light into the image sensor, and wherein the calibration module collects and saves the spectral calibration data of each pixel of the image sensor;
synchronizing the temporal information from the clock; and
reconstructing spectrum of the object.
10. A method to operate a sequential spectral imaging camera according to claim 9 , further comprising imaging processing comprising correction of distortion with the help of an algorithm such as simultaneous localization and mapping (SLAM) and 3D volumetric spectral extraction.
11. A sequential spectral imaging camera, comprising:
an image sensor amounted on a camera, wherein the image sensor comprises a plurality of pixels sensitive to light signals;
a color filter disposed on the image sensor, wherein the color filter is configured to respond to a plurality of colors based on locations on the color filter, wherein the image sensor acquires a sequence of image frames each associated with spectral, spatial, and temporal data of an object moving relatively to the camera, wherein one image frame corresponds to one temporal point;
a clock used to add temporal information to the sequence of image frames, for each pixel on the image sensor; and
a computer system capable of recording the sequence of image frames, comparing data of each pixel with spectral calibration data from a calibration module, synchronizing the temporal information from the clock, and reconstructing images of the object;
wherein the calibration module comprises a broadband light source to emit light in a range of wavelengths, a wavelength selective device capable of scanning and dividing the broadband light source into multiple wavelengths, and an integration sphere to project the light into the image sensor, wherein the calibration module collects and saves the spectral calibration data of each pixel of the image sensor.
12. The sequential spectral imaging camera as in claim 11 , wherein the color filter comprises wavelength regions arranged monotonically along column and rows of the image sensor.
13. A sequential spectral imaging system as in claim 11 , wherein the image sensor has a spectral response in the UV, visible and near IR and mid-IR range.
14. A calibration apparatus to set up a sequential spectral imaging system as in claim 11 , further comprising:
a data recorder recording a spectral response data for each of the pixels of the image sensor as a calibration data.