Vehicle based radar upsampling
Various embodiments of the present disclosure may include one or more object detection devices. The object detection devices may include at least one distance sensor such as a radar, lidar, or other distance sensor and at least one thermal sensor such as a thermal imaging device. One or more object detection devices may be mounted to vehicles to provide enhanced representations of an area around the vehicles.
1. An apparatus comprising:
an object detection device comprising:
a first distance sensor configured to detect objects within at least a first part of a scene and determine object distances which are distances to the objects, and output first distance data, and
a first thermal sensor configured to detect objects in at least the first part of the scene and output first thermal data; and
a logic device configured to:
receive the first distance data,
receive the first thermal data,
for each object detected by the first distance sensor, in an increasing order of object distances, match the object to the first thermal data to determine a distance to a matching object in the first thermal data; and
generate, based, at least in part, on the distances to matching objects in the first thermal data, a three-dimensional representation of at least the first part of the scene.
2. The apparatus of claim 1 , wherein the logic device is configured to generate the three-dimensional representation by, at least, interpolating, warping, and/or upsampling, with a function, one of the first distance data or the first thermal data, wherein the function is determined, at least in part, with the other of the first distance data or the first thermal data.
3. The apparatus of claim 1 , wherein the logic device is further configured to:
classify the objects with the first thermal data.
4. The apparatus of claim 1 , wherein:
the first distance sensor is configured to cycle through a plurality of planes defined with respect to the first distance sensor as being at respective different distances from the first distance sensor, wherein cycling through the plurality of planes comprises emitting signals directed towards the planes; and
the three-dimensional representation associates each object detected in the first thermal data with one of the planes to indicate the distance to the object in the first thermal data.
5. The apparatus of claim 3 , wherein:
the object detection device further comprises a second thermal sensor configured to detect thermal signatures within a second part of the scene and output second thermal data, wherein the first and second parts of the scene at least partially overlap; and
the logic device is further configured to:
receive the second thermal data, and
determine, based on the first thermal data and the second thermal data, a first object distance.
6. The apparatus of claim 1 , wherein the object detection device is a first object detection device and further comprising:
a second objection detection device comprising:
a second distance sensor configured to detect objects within a second part of the scene and output second distance data, and
a second thermal sensor configured to detect thermal signatures within the second part of the scene and output second thermal data, wherein the first and second parts of the scene at least partially overlap and the logic device is further configured to:
receive the second distance data;
receive the second thermal data, wherein the three-dimensional representation is generated based, additionally, on the second distance data and the second thermal data, wherein the three-dimensional representation further comprises a representation of a scene portion comprising the overlap of the first and second parts of the scene.
7. The apparatus of claim 1 , wherein the first distance sensor is a radar based sensor and/or a lidar based sensor.
8. The apparatus of claim 1 , where the logic device is further configured to output the three-dimensional representation to a user interface.
9. The apparatus of claim 1 , further comprising a mobile structure, wherein the object detection device is coupled to the mobile structure.
10. The apparatus of claim 9 , wherein the mobile structure comprises a plurality of object detection devices such that each object detection device comprises a distance sensor and a thermal sensor.
11. The apparatus of claim 10 , wherein the plurality of object detection devices are coupled to the mobile structure to provide a 360 degree coverage around the mobile structure and the three-dimensional representation generated by the logic device comprises at least a representation of the 360 degree coverage around the mobile structure.
12. A method comprising:
receiving first distance data from a first distance sensor of an object detection device, wherein the first distance sensor is configured to detect objects within at least a first part of a scene;
receiving first thermal data from a first thermal sensor of the object detection device, wherein the first thermal data is configured to detect objects in at least the first part of the scene;
for each object detected by the first distance sensor, in an increasing order of object distances, matching the object to the first thermal data to determine a distance to a matching object in the first thermal data; and
generating, based, at least in part, on the distances to matching objects in the first thermal data, a three-dimensional representation of at least the first part of the scene.
13. The method of claim 12 , further comprising:
classifying the objects with the first thermal data.
14. The method of claim 13 , further comprising:
cycling, with the first distance sensor, through a plurality of planes defined with respect to the first distance sensor as being at respective different distances from the first distance sensor, wherein cycling through the plurality of planes comprises emitting signals directed towards the planes;
wherein the three-dimensional representation associates each object detected in the first thermal data with one of the planes to indicate the distance to the object in the first thermal data.
15. The method of claim 13 , further comprising:
receiving second thermal data from a second thermal sensor of the object detection device, wherein the second thermal data is configured to detect thermal signatures within a second part of the scene, wherein the first and second parts of the scene at least partially overlap; and
determining, based on the first thermal data and the second thermal data, a first object distance.
16. The method of claim 12 , further comprising:
receiving second distance data from a second distance sensor, wherein the second distance sensor is configured to detect objects within a second part of the scene;
receiving second thermal data from a second thermal sensor, wherein the second thermal data is configured to detect thermal signatures within the second part of the scene, wherein the three-dimensional representation is generated based, additionally, on the second distance data and the second thermal data, and wherein the three-dimensional representation further comprises a representation of a scene portion comprising the overlap of the first and second parts of the scene; and
outputting the three-dimensional representation to a user interface.
17. The method of claim 12 , wherein the first distance sensor is a radar based sensor and/or a lidar based sensor.
18. The method of claim 12 , wherein the object detection device is coupled to a mobile structure.
19. The method of claim 18 , wherein the mobile structure comprises a plurality of object detection devices such that each object detection device comprises a distance sensor and a thermal sensor.
20. The method of claim 19 , wherein the plurality of object detection devices are coupled to the mobile structure to provide a 360 degree coverage around the mobile structure and the three-dimensional representation generated by the logic device comprises at least a representation of the 360 degree coverage around the mobile structure.