Augmenting panoramic LIDAR results with color
Methods and systems can augment 360 degree panoramic LIDAR results (e.g., from a spinning LIDAR system) with color obtained from color cameras. A color-pixel-lookup table can specify the correspondence between LIDAR pixels (depth/ranging pixels) and color pixels, which may be done at different viewing object distances. The operation of the color cameras can be triggered by the angular positions of the LIDAR system. For example, a color image of a particular camera can be captured when the LIDAR system is at a particular angular position, which can be predetermined based on properties of the cameras (e.g., shutter speed). Alternatively or in addition, a common internal clock can be used to assign timestamps to LIDAR and color pixels as they are captured. The corresponding color pixel(s), e.g., as determined using a color-pixel-lookup table, with the closest timestamp can be used for colorization.
1. A panoramic color light detection and ranging (LIDAR) system comprising:
a stationary base;
a LIDAR system rotationally coupled to the stationary base and configured to perform a plurality of ranging measurements by transmitting light and detecting reflected light to acquire a LIDAR image including a plurality of LIDAR pixels distributed around an angular field of view;
a plurality of color cameras fixedly coupled to the stationary base and disposed circumferentially around the LIDAR system, each camera having a respective camera field of view that is a portion of the angular field of view, the plurality of color cameras configured to acquire a plurality of color images comprising a plurality of color pixels;
a memory storing a color-pixel lookup table that defines mappings between the plurality of LIDAR pixels and the plurality of color pixels;
a controller operatively coupled to the memory and configured to:
trigger an image capture of each of the plurality of color images based on a determined angular position of the LIDAR system relative to a respective camera field of view so as synchronize capturing of a portion of the LIDAR pixels with capturing of color pixels of a respective camera of the plurality of color cameras; and
associate at least one of the plurality of color pixels of an identified one of the plurality of color images with a corresponding LIDAR pixel of the plurality of LIDAR pixels using the color-pixel lookup table stored in the memory, thereby generating a colorized LIDAR pixel.
2. The panoramic color LIDAR system of claim 1 , further comprising:
a rotary encoder that is configured to output an angular position signal that corresponds to a pointing direction of the LIDAR system relative to a fixed reference point on the stationary base, wherein the controller is configured to receive the angular position signal and trigger the image capture when the angular position signal indicates that the angular position is equal to a specified value of the angular position signal associated with one of the plurality of color cameras.
3. The panoramic color LIDAR system of claim 1 , wherein the controller is configured to:
store a group of the plurality of LIDAR pixels in the memory until a corresponding color image of the plurality of color images is acquired; and
associate one or more of the plurality of color pixels with each of the group of the plurality of LIDAR pixels in response to the corresponding color image being acquired.
4. The panoramic color LIDAR system of claim 1 , further comprising:
an internal clock configured to record a plurality of time stamps;
the controller further configured to:
identify a first capture time for the at least one of the plurality of color pixels based on a first time stamp of the plurality of time stamps; and
identify a second capture time for the corresponding LIDAR pixel based on a second time stamp of the plurality of time stamps,
wherein associating the at least one of the plurality of color pixels with the corresponding LIDAR pixel is further based on comparing the first time stamp and the second time stamp.
5. The panoramic color LIDAR system of claim 1 , further comprising:
an encoder receiver configured to determine the angular position by detecting a first feature corresponding to the determined angular position.
6. The panoramic color LIDAR system of claim 1 , the controller further configured to:
output the colorized LIDAR pixel to a user interface.
7. A panoramic color light detection and ranging (LIDAR) system comprising:
a stationary base;
a LIDAR system rotationally coupled to the stationary base and configured to perform a plurality of ranging measurements by transmitting light and detecting reflected light to acquire a LIDAR image including a plurality of LIDAR pixels distributed around an angular field of view;
a plurality of color cameras fixedly coupled to the stationary base and disposed circumferentially around the LIDAR system, each camera having a respective camera field of view that is a portion of the angular field of view, the plurality of color cameras configured to acquire a plurality of color images comprising a plurality of color pixels; and
a controller configured for:
determining a spatial correspondence between respective ones of the LIDAR pixels and respective ones of the color pixels, wherein determining the spatial correspondence comprises using a color-pixel lookup table that defines mappings between the plurality of LIDAR pixels and the plurality of color pixels to associate the color pixels with the respective LIDAR pixels;
combining the color pixels with spatially corresponding LIDAR pixels to generate colorized LIDAR pixels; and
providing the colorized LIDAR pixels to a user interface.
8. The panoramic LIDAR system of claim 7 , wherein the color-pixel lookup table is indexed by angular positions around 360 degrees for each revolution, wherein each entry of the color-pixel lookup table identifies one or more color pixels of a respective color image for a particular angular position and a particular sensor in a pixel array.
9. The panoramic LIDAR system of claim 7 , wherein the color-pixel lookup table includes depth values for mapping between LIDAR pixels and the color pixels, the controller further configured for:
determining depth values of the LIDAR pixels, wherein the color pixels are further determined based on the depth values.
10. The panoramic LIDAR system of claim 7 , wherein the colorized LIDAR pixels are provided via a three-dimensional color map of detected objects over time.
11. The panoramic LIDAR system of claim 10 , wherein the color map further comprises a distance value at a particular time for a particular view for a specific object.
12. The panoramic LIDAR system of claim 7 , wherein capturing the plurality of color images comprises:
capturing of each of the plurality of color images based on a determined angular position of the LIDAR system relative to a respective camera field of view so as synchronize capturing of a portion of the LIDAR pixels with capturing of color pixels of a respective camera of the plurality of color cameras.
13. A method comprising: performing, using a LIDAR system rotationally coupled to a stationary base, a plurality of ranging measurements by transmitting light and detecting reflected light to acquire a LIDAR image including a plurality of LIDAR pixels distributed around an angular field of view;
acquiring, using a plurality of cameras, a plurality of color images comprising a plurality of color pixels, each camera of the plurality of cameras having a respective camera field of view that is a portion of the angular field of view;
determining a spatial correspondence between respective ones of the LIDAR pixels and respective ones of the color pixels, wherein determining the spatial correspondence comprises using a color-pixel lookup table that defines mappings between the plurality of LIDAR pixels and the plurality of color pixels to associate the color pixels with the respective LIDAR pixels;
combining the color pixels with spatially corresponding LIDAR pixels to generate colorized LIDAR pixels; and
providing the colorized LIDAR pixels to a user interface.
14. The method of claim 13 wherein the color-pixel lookup table is indexed by angular positions around 360 degrees for each revolution, wherein each entry of the color-pixel lookup table identifies one or more color pixels of a respective color image for a particular angular position and a particular sensor in a pixel array.
15. The method of claim 13 , wherein the color-pixel lookup table includes depth values for mapping between LIDAR pixels and the color pixels, the method further comprising:
determining depth values of the LIDAR pixels, wherein the color pixels are further determined based on the depth values.
16. The method of claim 13 , wherein the colorized LIDAR pixels are provided via a three-dimensional color map of detected objects over time.
17. The method of claim 16 , wherein the color map further comprises a distance value at a particular time for a particular view for a specific object.
18. The method of claim 13 , wherein capturing the plurality of color images comprises:
capturing of each of the plurality of color images based on a determined angular position relative to a respective camera field of view so as synchronize capturing of a portion of the LIDAR pixels with capturing of color pixels.