Radiometric thermal imaging improvements for navigation systems and methods
Thermal imaging and navigation systems and related techniques are provided to improve the operation of manned or unmanned mobile platforms, including passenger vehicles. A system includes a thermal imaging device configured to be mounted on a vehicle. The thermal imaging device is configured to, when mounted on the vehicle, capture a first image of a scene encompassing a portion of the vehicle and capture a second image associated with a reflection of the scene from the portion of the vehicle. The system further includes a logic device configured to communicate with the thermal imaging device and determine a disparity map based on the first image and the second image.
1. A system comprising:
a thermal imaging device configured to be mounted on a vehicle, wherein the thermal imaging device is configured to, when mounted on the vehicle, capture an-a first image of a scene encompassing a portion of the vehicle, wherein the image comprises a first image portion encompassing at least one object in the scene and a second image portion encompassing the at least one object in a reflection of the scene from the portion of the vehicle, and wherein the thermal imaging device and the portion of the vehicle are fixed in position relative to each other; and
a logic device configured to:
communicate with the thermal imaging device;
flip the second image portion along an axis to obtain a mirrored image;
determine a disparity associated with the at least one object based on a shift between the at least one object in a first image plane associated with the first image portion and the at least one object in a second image plane associated with the mirrored image;
determine a depth associated with the at least one object based on the disparity associated with the at least one object and a distance between the thermal imaging device and a virtual imaging device associated with the second image plane;
determine a path loss associated with electromagnetic radiation emitted by the at least one object based on ambient sensor data associated with the scene and the depth associated with the at least one object; and
determine radiometric data associated with the scene based on the path loss and the depth.
2. The system of claim 1 , wherein the logic device is further configured to:
identify the at least one object as being common to the first image portion and the second image portion.
3. The system of claim 2 , wherein the logic device is configured to identify the at least one object as being common to the first image portion and the second image portion using a convolution neural network and/or an edge detector.
4. The system of claim 1 , wherein:
the thermal imaging device is configured to be mounted on a roof of the vehicle,
the portion of the vehicle comprises a portion of a hood of the vehicle,
the logic device is further configured to de-warp the second image portion based on a shape of the hood and a pose of the thermal imaging device to obtain a de-warped image, and
the logic device is configured to determine the disparity based on the first image portion and the de-warped image.
5. The system of claim 1 , wherein:
the thermal imaging device is further configured to provide thermal image data corresponding to a projected course for the vehicle; and
the logic device is further configured to receive the thermal image data corresponding to the projected course.
6. The system of claim 5 , further comprising:
a sensor system coupled to the vehicle and configured to provide sensor data associated with the projected course;
wherein the logic device is configured to receive the sensor data corresponding to the thermal image data, wherein the projected course is based, at least in part, on a combination of the sensor data and the thermal image data.
7. The system of claim 6 , further comprising a communication device configured to establish a wireless communication link with an update server associated with the vehicle, wherein:
the logic device is further configured to receive the thermal image data from the thermal imaging device as the vehicle maneuvers along the projected course and report information corresponding to the projected course over the wireless communication link to the update server.
8. The system of claim 6 , wherein the sensor system comprises an orientation sensor, a position sensor, a visible spectrum imaging system, and/or a ranging sensor system, and wherein the sensor data comprises visible spectrum image data corresponding to the projected course, orientation data associated with motion of the vehicle, position data associated with motion of the vehicle, and/or ranging sensor data corresponding to the projected course.
9. The system of claim 8 , wherein:
the sensor data comprises the visible spectrum image data and the logic device is further configured to generate blended imagery based, at least in part, on the visible spectrum image data and the thermal image data; and/or
the sensor data comprises the ranging sensor data and the ranging sensor system comprises a grille mounted radar system and/or a grille mounted lidar system.
10. A method comprising:
capturing, by a thermal imaging device mounted on a vehicle, an image of a scene encompassing a portion of the vehicle, wherein the image comprises a first image portion encompassing at least one object in the scene and a second image portion encompassing the at least one object in a reflection of the scene from the portion of the vehicle, and wherein the thermal imaging device and the portion of the vehicle are fixed in position relative to each other;
flipping the second image portion along an axis to obtain a mirrored image;
determining a disparity associated with the at least one object based on a shift between the at least one object in a first image plane associated with the first image portion and the at least one object in a second image plane associated with the mirrored image;
determining a depth associated with the at least one object based on the disparity associated with the at least one object and a distance between the thermal imaging device and a virtual imaging device associated with the second image plane;
determining a path loss associated with electromagnetic radiation emitted by the at least one object based on ambient sensor data associated with the scene and the depth associated with the at least one object; and
determining radiometric data associated with the scene based on the path loss and the depth.
11. The method of claim 10 , further comprising:
identifying the at least one object as being common to the first image portion and the second image portion.
12. The method of claim 11 , wherein the identifying is performed using a convolution neural network and/or an edge detector.
13. The method of claim 10 , wherein the thermal imaging device is mounted on a roof of the vehicle, and wherein the portion of the vehicle comprises a portion of a hood of the vehicle.
14. The method of claim 10 , further comprising capturing, by the thermal imaging device, thermal image data corresponding to a projected course for the vehicle.
15. The method of claim 14 , further comprising receiving sensor data corresponding to the thermal image data, wherein the projected course is based, at least in part, on a combination of the sensor data and the thermal image data.
16. The method of claim 15 , wherein the sensor data comprises visible spectrum image data corresponding to the projected course, orientation data associated with motion of the vehicle, position data associated with motion of the vehicle, and/or ranging sensor data corresponding to the projected course.
17. The method of claim 14 , wherein the thermal image data is captured as the vehicle maneuvers along the projected course, wherein the method further comprises:
reporting information corresponding to the projected course over a wireless communication link to an update server associated with the vehicle via a communication device configured to establish the wireless communication link with the update server.
18. The method of claim 10 , wherein the radiometric data comprises a temperature associated with the at least one object.
19. The method of claim 10 , further comprising determining an atmospheric transmission for each pixel of the first image portion that is associated with overlapping fields of view of the thermal imaging device and the virtual imaging device based on a distance between the pixel and the at least one object and the path loss associated with the electromagnetic radiation emitted by the at least one object, wherein the radiometric data associated with the scene is based on the atmospheric transmission for each pixel of the first image portion that is associated with the overlapping fields of view.
20. The system of claim 1 , wherein the radiometric data comprises a temperature associated with the at least one object.