Systems and methods for overhead imaging and video
Examples of an imaging sensor include a two-dimensional staring sensor with spectral filter strips for multispectral overhead imaging. The sensor may also include a panchromatic sensor with block or strip filters. The sensor may be used to collect multispectral color image data at a sampling resolution from overhead imaging platforms such as airplanes or satellites. The sensor can be used to provide video images. If a panchromatic sensor is included, the sensor may be used to collect panchromatic image data. Examples of methods for processing the image date include using the panchromatic image data to perform multi-frame enhancement or panchromatic sharpening on spectral images to improve their quality and resolution.
1. A computer-implemented method of enhancing image data, the method comprising:
receiving, by one or more computing devices, a sequence of spectral image strips, each spectral image strip comprising an image of a portion of a region of interest, each spectral image strip comprising image data from a spectral range of wavelengths of light, wherein each spectral image strip contains overlapping measurements of the region of interest relative to one or more neighboring spectral image strips;
registering, by the one or more computing devices, the sequence of spectral image strips to provide a spectral image of the region of interest based at least in part on the overlapping measurements of the region of interest;
receiving, by the one or more computing devices, a plurality of panchromatic images of the region of interest, each panchromatic image comprising image data from a panchromatic range of wavelengths of light that is broader than the spectral range of wavelengths of light;
performing, by the one or more computing devices, multi-frame enhancement on the plurality of panchromatic images to generate an enhanced panchromatic image having a higher resolution than each of the plurality of panchromatic images; and
generating, by the one or more computing devices, a panchromatic-sharpened spectral image using at least the enhanced panchromatic image and the spectral image, the panchromatic-sharpened spectral image having a higher resolution than the spectral image;
wherein each spectral image strip and each panchromatic image is captured by a single sensor device.
2. The method of claim 1 , wherein receiving a sequence of spectral image strips comprises:
receiving, by the one or more computing devices, a first sequence of spectral image strips comprising image data from a first spectral range of wavelengths of light;
receiving, by the one or more computing devices, a second sequence of spectral image strips comprising image data from a second spectral range of wavelengths of light, the second spectral range different from the first spectral range; and
registering the sequence of spectral image strips comprises:
registering, by the one or more computing devices, the first sequence of spectral image strips to provide a first spectral image of the region of interest; and
registering, by the one or more computing devices, the second sequence of spectral image strips to provide a second spectral image of the region of interest.
3. The method of claim 2 , further comprising combining, by the one or more computing devices, the first spectral image and the second spectral image to generate a multispectral image of the region of interest.
4. The method of claim 1 , further comprising upsampling, by the one or more computing devices, the spectral image to the resolution of the enhanced panchromatic image.
5. The method of claim 1 , wherein the plurality of panchromatic images and the spectral image have a substantially similar capture resolution.
6. The method of claim 1 , wherein generating the panchromatic-sharpened spectral image comprises:
converting the spectral image into a color system comprising a luminance component to create a converted spectral image; and
replacing the luminance component of the converted spectral image with the enhanced panchromatic image.
7. The method of claim 6 , further comprising converting, by the one or more computing devices, back into a color system of the spectral image after replacing the luminance component of the converted spectral image with the enhanced panchromatic image.
8. The method of claim 1 , wherein receiving the plurality of panchromatic images comprises receiving at least four panchromatic images.
9. The method of claim 1 , wherein the plurality of panchromatic images are captured by an imaging system having a diffraction limit, and receiving the plurality of panchromatic images comprises receiving sufficient panchromatic images such that the enhanced panchromatic image has a resolution of approximately the diffraction limit of the imaging system.
10. The method of claim 1 , wherein generating the panchromatic-sharpened spectral image comprises injecting details from the enhanced panchromatic image into the spectral image.
11. The method of claim 1 , wherein
performing multi-frame enhancement comprises performing multi-frame enhancement on only a sub-frame area of the panchromatic images to generate an enhanced panchromatic image of the sub-frame area; and
generating the panchromatic-sharpened spectral image comprises generating the panchromatic-sharpened spectral image of only the sub-frame area.