Rotating LIDAR with co-aligned imager
Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.
1. A method, comprising:
capturing, using a first sensor, a first pixel corresponding to an object, wherein the first pixel is captured at a first time;
capturing, using the first sensor, a second pixel corresponding to the object, wherein the second pixel is captured at a second time;
determining, using a second sensor, a distance to the object;
mapping the first pixel to the second pixel based at least on:
the determined distance to the object, and
a rate of rotation of the first sensor; and
generating, using the first pixel and the second pixel, a combined pixel.
2. The method of claim 1 , wherein the first sensor comprises a camera sensor, and wherein the second sensor comprises a light detection and ranging (lidar) sensor.
3. The method of claim 1 , wherein the second sensor is co-rotated with the first sensor.
4. The method of claim 1 , wherein mapping the first pixel to the second pixel is further based on a difference between the first time and the second time.
5. The method of claim 1 , wherein generating the combined pixel comprises:
aggregating a first intensity of the first pixel and a second intensity of the second pixel.
6. The method of claim 1 , further comprising:
generating a three-dimensional voxel that comprises the combined pixel and the distance to the object.
7. The method of claim 1 , further comprising:
generating an image of the object, wherein the image of the object comprises a plurality of combined pixels, wherein the plurality of combined pixels comprises the combined pixel generated using the first pixel and the second pixel.
8. The method of claim 1 , further comprising:
using the combined pixel to determine a driving path for a vehicle.
9. A sensing system, comprising:
a first sensor configured to capture:
a first pixel corresponding to an object, wherein the first pixel is captured at a first time, and
a second pixel corresponding to the object, wherein the second pixel is captured at a second time;
a second sensor configured to collect an imaging data representative of a distance to the object; and
one or more processing devices configured to:
determine, using the imaging data, the distance to the object;
map the first pixel to the second pixel based at least on:
the determined distance to the object, and
a rate of rotation of the first sensor; and
generate, using the first pixel and the second pixel, a combined pixel.
10. The sensing system of claim 9 , wherein the first sensor comprises a camera sensor, and wherein the second sensor comprises a light detection and ranging (lidar) sensor.
11. The sensing system of claim 9 , wherein the second sensor is co-rotated with the first sensor.
12. The sensing system of claim 9 , wherein mapping the first pixel to the second pixel is further based on a difference between the first time and the second time.
13. The sensing system of claim 9 , wherein to generate the combined pixel, the one or more processing devices are to:
aggregating a first intensity of the first pixel and a second intensity of the second pixel.
14. The sensing system of claim 9 , wherein the one or more processing devices are further to:
generate a three-dimensional voxel that comprises the combined pixel and the distance to the object.
15. The sensing system of claim 9 , wherein the one or more processing devices are further to:
generate an image of the object, wherein the image of the object comprises a plurality of combined pixels, wherein the plurality of combined pixels comprises the combined pixel generated using the first pixel and the second pixel.
16. The sensing system of claim 9 , wherein the one or more processing devices are further to:
use the combined pixel to determine a driving path for a vehicle.
17. A vehicle comprising:
a light detection and ranging (lidar) sensor configured to determine a distance to an object;
a camera sensor configured to capture:
a first pixel corresponding to an object, wherein the first pixel is captured at a first time, and
a second pixel corresponding to the object, wherein the second pixel is captured at a second time;
a controller configured to:
map the first pixel to the second pixel based at least on:
the determined distance to the object, and
a rate of rotation of the camera sensor; and
generate, using the first pixel and the second pixel, a combined pixel; and
a vehicle control system configured to:
determine a driving path for the vehicle in view of the combined pixel.
18. The vehicle of claim 17 , wherein the lidar sensor is co-rotated with the camera sensor.
19. The vehicle of claim 17 , wherein mapping the first pixel to the second pixel is further based on a difference between the first time and the second time.
20. The vehicle of claim 17 , wherein to generate the combined pixel, the controller is to:
aggregate a first intensity of the first pixel and a second intensity of the second pixel.