Systems and methods for convolutional high resolution lidar imaging
Disclosed herein are systems, methods, and computer program products for operating a lidar system. The methods comprise: performing operations by each of a plurality of photodetectors to facilitate measurements of an intensity of a light signal reflected off an object external to the lidar system; receiving, by a processor, result values from the photodetectors that indicate measured reflected intensities of the light signal; performing, by the processor, at least one convolutional algorithm to combine different sets of the result values to produce a plurality of feature values; and generating, by the processor, at least one depth image or point cloud comprising a plurality of super pixels having values respectively set to the feature values.
1 . A method for operating a lidar system, comprising:
performing operations by each of a plurality of photodetectors to facilitate measurements of an intensity of a light signal reflected off an object external to the lidar system;
receiving, by a processor, result values from the photodetectors that indicate measured reflected intensities of the light signal;
performing, by the processor, at least one convolutional algorithm to combine different sets of the result values to produce a plurality of feature values, wherein the different sets of the result values partially overlap with each other;
generating, by the processor, at least one depth image or point cloud comprising a plurality of super pixels having values respectively set to the feature values; and
performing, by the processor, post processing operations for correcting an edge effect of the at least one depth image, wherein the edge effect results from the performing of the at least one convolutional algorithm.
2 . The method according to claim 1 , wherein the at least one convolutional algorithm is (i) configured to combine non-overlapping or overlapping sets of the result values or (ii) implemented by even and odd sized convolutional computing kernels with half-pixel strides.
3 . The method according to claim 1 , further comprising selecting by the processor the at least one convolutional algorithm from a plurality of convolutional algorithms based on at least one of a time of day, a location of the lidar system, or at least one condition of an environment in which the lidar system is being used.
4 . The method according to claim 1 , further comprising selecting, by the processor, at least one of a size or a stride for a convolutional computing kernel implementing the at least one convolutional algorithm based on at least one of a time of day, a location of the lidar system, and at least one condition of an environment in which the lidar system is being used.
5 . The method according to claim 1 , wherein the post processing operations comprise:
identifying a transition region between adjacent super pixels where a difference in the feature values exceeds a distance threshold; and
adjusting the feature value of a target super pixel within the transition region based on a comparison of an estimated intensity value associated with the target super pixel against an intensity threshold.
6 . The method according to claim 5 , wherein the post processing operations comprise:
identifying super pixels of the at least one depth image that are associated with estimated intensity values or noise floor values below a pre-defined value; and
increasing the feature values of the identified super pixels which were identified.
7 . The method according to claim 6 , wherein the feature values of the identified super pixels are increased using a convolutional computing kernel with a size that is inversely proportional to a respective one of the estimated intensity values.
8 . The method according to claim 5 , wherein the post processing operations comprise:
identifying super pixels of the at least one depth image that are associated with estimated intensity values or noise floor values above a pre-defined value; and
decreasing a confidence value associated with the identified super pixels or discarding the identified super pixels.
9 . The method according to claim 5 , wherein the post processing operations comprise:
identifying super pixels of the at least one depth image that are associated with estimated intensity values or noise floor values above a pre-defined value;
detecting whether a super pixel of the identified super pixels is associated with one of said photodetectors which generated two or more measures of reflected intensity for the light signal during a certain period of time;
swapping a first measured reflectivity value of a first return with a second measured reflectivity value of a second return for the super pixel; and
computing a new feature value using the second measured reflectivity value.
10 . A system, comprising:
a processor;
a non-transitory computer-readable storage medium comprising programming instructions that are configured to cause the processor to implement a method for operating a lidar system, wherein the programming instructions comprise instructions to:
cause each of a plurality of photodetectors to facilitate measurements of an intensity of a light signal reflected off an object external to the lidar system;
receive result values from the plurality of photodetectors that indicate measured reflected intensities of the light signal;
perform at least one convolutional algorithm to combine different sets of the result values to produce a plurality of feature values, wherein the different sets of the result values partially overlap with each other;
generate at least one depth image or point cloud comprising a plurality of super pixels having values respectively set to the feature values; and
perform post processing operations for correcting an edge effect of the at least one depth image, wherein the edge effect results from the performing of the at least one convolutional algorithm.
11 . The system according to claim 10 , wherein the at least one convolutional algorithm is (i) configured to combine non-overlapping or overlapping sets of the result values or (ii) implemented by even and odd sized convolutional computing kernels with half-pixel strides.
12 . The system according to claim 10 , wherein the programming instructions further comprise instructions to select the at least one convolutional algorithm from a plurality of convolutional algorithms based on at least one of a time of day, a location of the lidar system, or at least one condition of an environment in which the lidar system is being used.
13 . The system according to claim 10 , wherein the programming instructions further comprise instructions to select at least one of a size or a stride for a convolutional computing kernel implementing the at least one convolutional algorithm based on at least one of a time of day, a location of the lidar system, and at least one condition of an environment in which the lidar system is being used.
14 . The system according to claim 10 , wherein the post processing operations comprise:
identifying a transition region between adjacent super pixels where a difference in the feature values exceeds a distance threshold; and
adjusting the feature value of a target super pixel within the transition region based on a comparison of an estimated intensity value associated with the target super pixel against an intensity threshold.
15 . The system according to claim 14 , wherein the post processing operations comprise:
identifying super pixels of the at least one depth image that are associated with estimated intensity values or noise floor values below a pre-defined value; and
increasing the feature values of the identified super pixels which were identified.
16 . The system according to claim 15 , wherein the feature values of the identified super pixels are increased using a convolutional computing kernel with a size that is inversely proportional to a respective one of the estimated intensity values.
17 . The system according to claim 14 , wherein the post processing operations comprise:
identifying super pixels of the at least one depth image that are associated with estimated intensity values or noise floor values above a pre-defined value; and
decreasing a confidence value associated with the identified super pixels or discarding the identified super pixels.
18 . A non-transitory computer-readable medium that stores instructions that are configured to, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
causing a plurality of photodetectors to facilitate measurements of an intensity of a light signal reflected off an object external to a lidar system;
receiving result values from the photodetectors that indicate measured reflected intensities of the light signal;
performing at least one convolutional algorithm to combine different sets of the result values to produce a plurality of feature values, wherein the different sets of the result values partially overlap with each other;
generating at least one depth image or point cloud comprising a plurality of super pixels having values respectively set to the feature values; and
perform post processing operations for correcting an edge effect of the at least one depth image, wherein the edge effect results from the performing of the at least one convolutional algorithm.
19 . The non-transitory computer-readable medium according to claim 18 , wherein the at least one convolutional algorithm is (i) configured to combine non-overlapping or overlapping sets of the result values or (ii) implemented by even and odd sized convolutional computing kernels with half-pixel strides.
20 . The non-transitory computer-readable medium according to claim 18 , wherein the at least one computing device to further caused to select the at least one convolutional algorithm from a plurality of convolutional algorithms based on at least one of a time of day, a location of the lidar system, or at least one condition of an environment in which the lidar system is being used.