Compact multi-spectrum imaging with fusion
Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, illumination modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.
1. A system comprising:
a housing;
a visible spectrum sensor assembly disposed within the housing and comprising a focal plane array (FPA) of visible spectrum sensors and adapted to capture visible spectrum images of a scene;
an infrared sensor assembly disposed within the housing and comprising a FPA of infrared sensors and adapted to capture infrared images of the scene;
a singular and discrete common circuit board disposed within the housing and physically connected to both the visible spectrum sensor assembly and the infrared sensor assembly, wherein the visible spectrum sensor assembly and the infrared sensor assembly are directly connected and parallel to and securedly fixed in position adjacent to each other by a first planar surface of the common circuit board to provide permanent alignment of the visible spectrum sensor assembly and the infrared sensor assembly to each other, such that the visible spectrum images and the infrared images are substantially rotationally aligned to each other due to the direct connection of the visible spectrum sensor assembly and the infrared sensor assembly to the first planar surface of the common circuit board; and
a logic device disposed within the housing and electrically coupled to the visible spectrum sensor assembly and the infrared sensor assembly through the common circuit board, wherein the logic device is adapted to:
receive at least one visible spectrum image captured by the visible spectrum sensor assembly directly connected to the first planar surface of the common circuit board, and
receive at least one infrared image captured by the infrared sensor assembly directly connected to the first planar surface of the common circuit board.
2. The system of claim 1 , further comprising a memory in communication with the logic device, wherein the logic device is adapted to:
store the at least one visible spectrum image and the at least one infrared image to the memory in a common data file according to a common data format, wherein the common data format comprises at least one component corresponding to a radiometric component of the at least one infrared image.
3. The system of claim 2 , wherein:
the common data format comprises a Red-Green-Blue-Temperature (RGBT) format encoding RGB components of the at least one visible spectrum image and a radiometric temperature component of the at least one infrared image.
4. The system of claim 1 , wherein:
the logic device is adapted to register the images to each other by performing interpolation, scaling, cropping, morphing and/or filtering operations on one or more of the images to substantially match common spatial content within the images;
the visible spectrum sensor assembly comprises a first field of view, the infrared sensor assembly comprises a second field of view, and wherein the first field of view and the second field of view substantially overlap; and
the housing comprises a plurality of electrical connections embedded within the housing and configured to electrically connect the logic device to the common circuit board, wherein heat generated by the visible spectrum sensor assembly and the infrared sensor assembly is dissipated through the common circuit board and the housing via the electrical connections.
5. The system of claim 1 , further comprising:
an illuminator in communication with the logic device and adapted to illuminate at least a portion of the scene in the visible and/or infrared spectrum; and
a user interface in communication with the logic device, wherein the logic device is adapted to:
receive user input from the user interface indicating a portion-of-interest of the scene imaged by the infrared sensor assembly,
control the illuminator to illuminate at least the portion-of-interest in a spectrum sensed by the visible spectrum sensor assembly,
receive at least one illuminated captured image of the portion-of-interest from the visible spectrum sensor assembly, and
generate a combined image comprising illuminated characteristics of the scene derived from the at least one illuminated captured image.
6. The system of claim 1 , further comprising an additional sensor assembly physically connected to the common circuit board and directly connected and parallel to the first planar surface of the common circuit board to provide permanent alignment of the additional sensor assembly to the visible spectrum sensor assembly and the infrared sensor assembly, wherein the logic device is adapted to:
receive at least one additional image of the scene from the additional sensor assembly; and
generate a combined image comprising stereo imaging characteristics of the scene derived from the at least one additional image, the at least one visible spectrum image, and/or the at least one infrared image.
7. The system of claim 1 , wherein:
the common circuit board, the visible spectrum sensor assembly, and the infrared sensor assembly are disposed within a mobile telephone, a tablet computing device, a personal digital assistant, a visible light camera, a music player, or a portable host device.
8. The system of claim 1 , wherein:
the visible spectrum sensor assembly and the infrared sensor assembly are implemented with a common aperture and/or optical elements adapted to provide a common view of the scene for the visible spectrum sensor assembly and the infrared sensor assembly, and wherein the common aperture and/or optical elements are disposed within the housing.
9. The system of claim 1 , wherein:
the logic device is implemented as a common processing module for the visible spectrum sensor assembly and the infrared sensor assembly; and
the logic device, the visible spectrum sensor assembly, and the infrared sensor assembly are arranged in a staggered arrangement on the common circuit board.
10. The system of claim 1 , wherein the logic device is adapted to:
receive control parameters;
derive color characteristics of the scene from at least one of the images;
derive high spatial frequency content from at least one of the images; and
generate a combined image comprising relative contributions of the color characteristics and the high spatial frequency content, wherein:
the relative contributions are determined by the control parameters:
the at least one infrared image comprises one or more unblurred infrared images of the scene corrected by a plurality of non-uniform correction (NUC) terms used to remove noise from the one or more unblurred infrared images; and
the NUC terms are determined based on an intentionally blurred infrared image of the scene.
11. The system of claim 1 , wherein:
the at least one visible spectrum image is captured substantially at a first time;
the at least one infrared image comprises a radiometric component and is captured substantially at a second time; and
the logic device is adapted to process the at least one visible spectrum image and the at least one infrared image to generate a combined image comprising visible spectrum characteristics of the scene derived from the at least one visible spectrum image and infrared characteristics of the scene derived from the radiometric component of the infrared image.
12. The system of claim 1 , wherein:
the FPA of infrared sensors comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts;
a size of the array of microbolometers is approximately 80 by 60; and
the infrared sensor assembly is adapted to be inserted into a socket having a size less than approximately 8.5 mm by 8.5 mm.
13. A method comprising:
receiving visible spectrum radiation from a scene at a focal plane array (FPA) of visible spectrum sensors of a visible spectrum sensor assembly disposed within a housing;
receiving infrared radiation from the scene at a FPA of infrared sensors of an infrared sensor assembly disposed within the housing, the visible spectrum sensor assembly and the infrared sensor assembly being aligned to each other by a singular and discrete common circuit board disposed within the housing and physically connected to both the visible spectrum sensor assembly and the infrared sensor assembly, wherein the visible spectrum sensor assembly and the infrared sensor assembly are directly connected and parallel to and securedly fixed in position adjacent to each other by a first planar surface of the common circuit board to provide permanent alignment of the visible spectrum sensor assembly and the infrared sensor assembly to each other, such that visible spectrum images captured by the visible spectrum sensor assembly and infrared images captured by the infrared sensor assembly are substantially rotationally aligned to each other due to the direct connection of the visible spectrum sensor assembly and the infrared sensor assembly to the first planar surface of the common circuit board;
generating at least one visible spectrum image of the scene in response to receiving the visible spectrum radiation;
generating at least one infrared image of the scene in response to receiving the infrared radiation; and
producing an output signal comprising data corresponding to at least one of the generated images.
14. The method of claim 13 , further comprising:
storing the data corresponding to the at least one visible spectrum image and the at least one infrared image to a memory in a common data file according to a common data format, wherein the common data format comprises at least one component corresponding to a radiometric component of the at least one infrared image.
15. The method of claim 14 , wherein:
the common data format comprises a Red-Green-Blue-Temperature (RGBT) format encoding RGB components of the at least one visible spectrum image and a radiometric temperature component of the at least one infrared image.
16. The method of claim 13 , further comprising registering the generated images to each other by performing interpolation, scaling, cropping, morphing and/or filtering operations on one or more of the images to substantially match common spatial content within the images, wherein:
the visible spectrum sensor assembly comprises a first field of view, the infrared sensor assembly comprises a second field of view, and wherein the first field of view and the second field of view substantially overlap; and
the housing comprises a plurality of electrical connections embedded within the housing and configured to electrically connect a logic device to the common circuit board, wherein heat generated by the visible spectrum sensor assembly and the infrared sensor assembly is dissipated through the common circuit board and the housing via the electrical connections.
17. The method of claim 13 , further comprising:
receiving user input from a user interface indicating a portion-of-interest of the scene imaged by the infrared sensor assembly;
controlling an illuminator to illuminate at least the portion-of-interest in a spectrum sensed by the visible spectrum sensor assembly, wherein the illuminator is adapted to illuminate at least a portion of the scene in the visible and/or infrared spectrum;
generating at least one illuminated captured image of the portion-of-interest by the visible spectrum sensor assembly; and
generating a combined image comprising illuminated characteristics of the scene derived from the at least one illuminated captured image.
18. The method of claim 13 , further comprising:
generating at least one additional image of the scene from an additional sensor assembly physically connected to the common circuit board and directly connected to the first planar surface of the common circuit board to provide permanent alignment of the additional sensor assembly to the visible spectrum sensor assembly and the infrared sensor assembly; and
generating a combined image comprising stereo imaging characteristics of the scene derived from the at least one additional image, the at least one visible spectrum image, and/or the at least one infrared image.
19. The method of claim 13 , wherein:
the common circuit board, the visible spectrum sensor assembly, and the infrared sensor assembly are disposed within a mobile telephone, a tablet computing device, a personal digital assistant, a visible light camera, a music player, or a portable host device.
20. The method of claim 13 , wherein:
the at least one visible spectrum image and the at least one infrared image are generated by receiving the infrared radiation and the visible spectrum radiation through a common aperture and/or optical elements adapted to provide a common view of the scene for the visible spectrum sensor assembly and the infrared sensor assembly, and wherein the common aperture and/or optical elements are disposed within the housing.
21. The method of claim 13 , wherein:
the images are generated and the output signal is produced by a common processing module, the visible spectrum sensor assembly, and the infrared sensor assembly arranged in a staggered arrangement on the common circuit board.
22. The method of claim 13 , further comprising:
receiving control parameters;
deriving color characteristics of the scene from at least one of the images;
deriving high spatial frequency content from at least one of the images; and
generating a combined image comprising relative contributions of the color characteristics and the high spatial frequency content, wherein:
the relative contributions are determined by the control parameters,
the at least one infrared image comprises one or more unblurred infrared images of the scene corrected by a plurality of non-uniform correction (NUC) terms used to remove noise from the one or more unblurred infrared images; and
the NUC terms are determined based on an intentionally blurred infrared image of the scene.
23. The method of claim 13 , wherein:
the at least one visible spectrum image is generated substantially at a first time; and
the at least one infrared image comprises a radiometric component and is generated substantially at a second time, the method further comprising:
processing the at least one visible spectrum image and the at least one infrared image to generate a combined image comprising visible spectrum characteristics of the scene derived from the at least one visible spectrum image and infrared characteristics of the scene derived from the radiometric component of the infrared image.
24. The method of claim 13 , wherein:
the FPA of infrared sensors comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts;
a size of the array of microbolometers is approximately 80 by 60; and
the infrared sensor assembly is adapted to be inserted into a socket having a size less than approximately 8.5 mm by 8.5 mm.
25. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors of a system are adapted to cause the system to perform a method comprising:
receiving at least one visible spectrum image of a scene captured by a visible spectrum sensor assembly disposed within a housing and comprising a focal plane array (FPA) of visible spectrum sensors;
receiving at least one infrared image of the scene captured by an infrared sensor assembly disposed within the housing and comprising a FPA of infrared sensors, the visible spectrum sensor assembly and the infrared sensor assembly being aligned to each other by a singular and discrete common circuit board disposed within the housing and physically connected to both the visible spectrum sensor assembly and the infrared sensor assembly, wherein the visible spectrum sensor assembly and the infrared sensor assembly are directly connected and parallel to and securedly fixed in position adjacent to each other by a first planar surface of the common circuit board to provide permanent alignment of the visible spectrum sensor assembly and the infrared sensor assembly to each other, such that visible spectrum images captured by the visible spectrum sensor assembly and infrared images captured by the infrared sensor assembly are substantially rotationally aligned to each other due to the direct connection of the visible spectrum sensor assembly and the infrared sensor assembly to the first planar surface of the common circuit board; and
generating a combined image comprising visible spectrum characteristics of the scene derived from the at least one visible spectrum image and infrared characteristics of the scene derived from the at least one infrared image.