Method and a system of determining LiDAR data degradation degree
A system and method for for determining a degree of point cloud data degradation of a LiDAR sensor of a Self-Driving Car (SDC) using a machine-learning algorithm (MLA) are provided. The method comprises: determining, based on a training point cloud generated by the LiDAR sensor representative of surroundings of the SDC, a plurality of LiDAR features; determining, for each training object in the surroundings, based on statistical data of coverage of training objects with LiDAR points, a plurality of enrichment features; receiving a respective label indicative of a degradation degree of the training point cloud; generating, based on the plurality of LiDAR features, the plurality of enrichment features, and the respective label, a given feature vector of a plurality of feature vectors; training, based on the plurality of feature vectors, the MLA to determine an in-use degree of degradation of in-use sensed data further generated by the LiDAR sensor.
1 . A computer-implemented method of determining a degree of point cloud data degradation of a Light Detection and Ranging (LiDAR) sensor of a Self-Driving Car (SDC) using a Machine-Learning Algorithm (MLA), the method being executable by at least one processor communicatively coupled to the LiDAR sensor, the method comprising, during a training phase, training the MLA by:
receiving, by the at least one processor from the LiDAR sensor, training sensed data including a given training point cloud representative of a given portion of a surrounding area of the SDC;
determining, by the at least one processor, based on the given training point cloud, a plurality of LiDAR features associated with the given portion of the surrounding area of the SDC;
receiving, by the at least one processor from a camera sensor of the SDC, a respective training image representative of the given portion of the surrounding area of the SDC,
the respective training image having been generated, by the camera sensor, at a same time as the sensed data by the LiDAR sensor;
projecting, by the at least one processor, the given training point cloud onto the respective training image to generate a merged representation of the given portion of the surrounding area of the SDC including statistical data of coverage of training objects present in the respective image with LiDAR points;
determining, by the at least one processor, for each training object present in the given portion of the surrounding area of the SDC, based on the statistical data, a plurality of enrichment features;
receiving, by the at least one processor, for the merged representation of the given portion, a respective label indicative of a degradation degree of the training sensed data generated by the LiDAR sensor;
generating, by the at least one processor, based on the plurality of LiDAR features, the plurality of enrichment features associated with each training object present in the given portion of the surrounding area of the SDC, and the respective label, a given training feature vector of a plurality of training feature vectors;
training, by the at least one processor, the MLA based on the plurality of training feature vectors to determine an in-use degree of degradation of in-use sensed data of the LiDAR sensor representative of in-use objects in the surrounding area of the SDC.
2 . The method of claim 1 , wherein the plurality of LiDAR features includes: (i) a number of LiDAR points in the given training point cloud; and (ii) light intensity values of the LiDAR points in the given training point cloud.
3 . The method of claim 2 , wherein the determining the plurality of LiDAR features comprises:
defining, along a horizontal field of view (HFOV) of the LiDAR sensor, a plurality of angles of the HFOV; and
determining the plurality of LiDAR features for LiDAR points of the given training point cloud having fallen in each angle of the plurality of angles the HFOV of the LiDAR sensor.
4 . The method of claim 1 , wherein the determining the plurality of enrichment features for a given training object in the given portion of the surrounding area of the SDC includes:
generating, by the at least one processor, a respective bounding box representative of the given training object; and
determining, by the at least one processor, at least one of:
a respective distance value from the SDC to the given training object;
a number of the training objects in the given portion of the surrounding area of the SDC;
a surface area of the respective bounding box associated with the given training object;
a number of LiDAR points having fallen within the respective bounding box;
a density of the LiDAR points having fallen within the respective bounding box;
light intensity values of the LiDAR points having fallen within the respective bounding box;
average distance values from the LiDAR sensor to the LiDAR points having fallen within the respective bounding box; and
energy values of the LiDAR points having fallen within the respective bounding box.
5 . The method of claim 4 , wherein the respective label comprises data indicative of whether at least one reference training object is covered with LiDAR points at the respective distance value from the SDC.
6 . The method of claim 5 , wherein the given training object is of a first predetermined object class, and the at least one reference training object is of a second predetermined object class, different from the first predetermined object class.
7 . The method of claim 6 , wherein the first predetermined object class is a vehicle.
8 . The method of claim 4 , wherein the generating the given training feature vector further comprises:
determining, by the at least one processor, based on the merged representation, a number of bounding boxes void of LiDAR points; and
determining, by the at least one processor, surface area values of the bounding boxes void of LiDAR points.
9 . The method of claim 1 , wherein the respective label has been generated by a human assessor, and the degradation degree of the training sensed data has been determined as being perceived by the human assessor.
10 . The method of claim 1 , wherein the generating the given training feature vector comprises adjusting the respective label using a machine-generated label indicative of a predicted degradation degree of the training sensed data, the predicted degradation degree having been determined by a secondary MLA, which has been trained to determine predicted degradation degrees of the training sensed data of the LiDAR sensor, the secondary MLA being different from the MLA, the adjusting comprising:
in response to every one of: (i) the predicted degradation degree of the training sensed data generated by the secondary MLA being misaligned with the degradation degree indicated by the respective label; (ii) the predicted degradation degree of the training sensed data being higher than the degradation degree; and (iii) a confidence level associated with the predicted degradation degree being equal to or greater than a predetermined confidence level threshold,
replacing the respective label with the machine-generated label in the given training feature vector.
11 . The method of claim 10 , wherein the secondary MLA has been trained based on at least a portion of the plurality of training feature vectors.
12 . The method of claim 10 , wherein each one of the MLA and the secondary MLA is a decision tree-based MLA.
13 . The method of claim 12 , wherein each one of the MLA and the secondary MLA is a respective decision tree forest.
14 . The method of claim 13 , wherein the secondary MLA has a lower dimensionality than that of the MLA.
15 . The method of claim 12 , wherein the decision-tree based MLA is a CatBoost decision tree MLA.
16 . The method of claim 9 , further comprising, in response to the replacing respective label with the machine-generated label in the given training feature vector:
generating a given reduced training feature vector based on the plurality of LiDAR features and the respective label; and
including the given reduced training feature vector in the plurality of training feature vectors.
17 . The method of claim 1 , further comprising, during an in-use phase following the training phase, using the MLA to determine the in-use degree of degradation of the in-use sensed data of the LiDAR sensor of the SDC, the using comprising:
receiving, by the at least one processor from the LiDAR sensor, the in-use sensed data including an in-use point cloud representative of the given portion of the surrounding area of the SDC;
determining, by the at least one processor, based on the in-use point cloud, the plurality of LiDAR features associated with the given portion of the surrounding area of the SDC;
receiving, by the at least one processor from a camera sensor of the SDC, a respective in-use image representative of the given portion of the surrounding area of the SDC,
the respective in-use image having been generated, by the camera sensor, at the same time as the in-use sensed data by the LiDAR sensor;
projecting, by the at least one processor, the in-use point cloud onto the respective in-use image to generate an in-use merged representation of the given portion of the surrounding area of the SDC including the statistical data of coverage of in-use objects present in the respective in-use image with LiDAR points of the in-use point cloud;
determining, by the at least one processor, for each in-use object present in the given portion of the surrounding area of the SDC, based on the statistical data, the plurality of enrichment features;
generating, by the at least one processor, based on the plurality of LiDAR features, the plurality of enrichment features associated with each in-use object present in the given portion of the surrounding area of the SDC, an in-use feature vector;
feeding, by the at least one processor, the in-use feature vector to the MLA to determine the in-use degree of degradation of the in-use sensed data generated by the LiDAR sensor,
the in-use degree of degradation being indicative of a current visibility distance at which the LiDAR sensor is capable of recognizing in-use objects in the given portion of the surrounding area of the SDC; and
in response to the in-use degree of degradation of the in-use sensed data exceeding a predetermined image data degradation threshold, causing, by the at least one processor, execution of at least one predetermined preventive action.
18 . The method of claim 17 , wherein the recognizing the in-use objects comprises recognizing in-use objects of a predetermined object class.
19 . The method of claim 17 , wherein the at least one predetermined preventive action comprises one of causing cleaning of the LiDAR sensor, generating a notification for maintenance of the LiDAR sensor, causing the SDC to decelerate, and causing the SDC to stop.
20 . A system for determining a degree of image data degradation of a Light Detection and Ranging (LiDAR) sensor of a Self-Driving Car (SDC) using a Machine-Learning Algorithm (MLA), the system comprising: at least one processor communicatively coupled to the LiDAR sensor; and a non-transitory computer-readable medium storing instructions, which, when executed by the at least one processor, cause the system to: receive, from the LiDAR sensor, training sensed data including a given training point cloud representative of a given portion of a surrounding area of the SDC; determine, based on the given training point cloud, a plurality of LiDAR features associated with the given portion of the surrounding area of the SDC; receive, from a camera sensor of the SDC, a respective training image representative of the given portion of the surrounding area of the SDC, the respective training image having been generated, by the camera sensor, at a same time as the training sensed data by the LiDAR sensor; project the given training point cloud onto the respective training image to generate a merged representation of the given portion of the surrounding area of the SDC including statistical data of coverage of training objects present in the respective image with LiDAR points; determine, for each training object present in the given portion of the surrounding area of the SDC, based on the statistical data, a plurality of enrichment features; receive, for the merged representation of the given portion, a respective label indicative of a degradation degree of the training sensed data generated by the LiDAR sensor; generate, based on the plurality of LiDAR features, the plurality of enrichment features associated with each training object present in the given portion of the surrounding area of the SDC, and the respective label, a given training feature vector of a plurality of training feature vectors; train the MLA based on the plurality of training feature vectors to determine an in-use degree of degradation of in-use sensed data of the LiDAR sensor representative of in-use objects in the surrounding area of the SDC.