On-road localization methodologies and equipment utilizing road surface characteristics
View Patent ↗Disclosed embodiments provide a technical improvement for providing localization for a transportation vehicle by detecting road wear reference lines in a roadway on which the transportation vehicle is travelling and controlling, guiding or otherwise facilitating alignment of the transportation vehicle wheel centers with the detected centers of the road wear.
1. Transportation vehicle equipment for performing on-roadway localization, the equipment comprising:
at least one LiDAR sensor;
at least one processor running software configured to:
generate LiDAR point cloud data based on data received from the at least one LiDAR sensor, wherein the LiDAR point cloud data comprise speckle characteristics for samples of the generated data, wherein the speckle characteristics are based on optical interference of back scattered light that result from coherent illumination of object surfaces with laser light generated by the LiDAR sensor;
analyze the LiDAR point cloud data to detect at least one road wear reference line in a roadway on which the transportation vehicle is travelling, wherein speckle pattern analysis is utilized to provide information for detecting road wear based on a variation of the road surface roughness, wherein the speckle pattern analysis analyzes the speckle characteristics in the LiDAR point cloud data for characteristic patterns indicative of the road surface roughness; and
facilitate alignment of a center of at least one wheel surface of a transportation vehicle wheel with the detected center of the road wear reference line,
wherein the analysis of the LiDAR point cloud data analyzes LiDAR point cloud data including x, y, and z coordinates, intensity data, and radial velocity data.
2. The transportation vehicle equipment of claim 1 , wherein the at least one processor is further configured to provide autonomous and/or assistive functionality for operating the transportation vehicle to travel on the roadway, which includes the facilitation of the alignment of the center of the at least one wheel surface of the transportation vehicle wheel with the detected center of the road wear reference line.
3. The transportation vehicle equipment of claim 2 , wherein the transportation vehicle includes a plurality of sensors including the at least one sensor, and wherein the equipment for providing autonomous and/or assistive functionality analyzes data generated by the plurality of sensors to provide the autonomous and/or assistive functionality in the transportation vehicle.
4. The transportation vehicle equipment of claim 3 , wherein the equipment for providing autonomous and/or assistive functionality alters a weighting of sensor data from at least some of the plurality of sensors based on analysis of data indicating reliability and/or availability of the sensor data from the plurality of sensors.
5. The transportation vehicle equipment of claim 1 , wherein the at least one LiDAR sensor is a Frequency Modulated Continuous Wave (FMCW) LiDAR sensor.
6. The transportation vehicle equipment of claim 5 , wherein the FMCW LiDAR provides coherent LiDAR using a continuously-emitting laser that varies frequency through a range and compares the frequency of returned signals with corresponding local reference signal to determine distance to a target.
7. The transportation vehicle equipment of claim 1 , wherein the analysis of the LiDAR point cloud data uses deep neural network operations to perform feature extraction and classification based on low-dimensional characteristic features.
8. The transportation vehicle equipment of claim 7 , wherein the deep neural network operations perform machine learning methods for data analysis to transform high-dimensional sensor data generated by the at least one sensor to characteristic low dimensional features.
9. The transportation vehicle equipment of claim 7 , wherein the deep neural network operations utilize deep neural networks for performing machine learning methods for data analysis including speckle pattern analysis.
10. The transportation vehicle of equipment of claim 7 , wherein the deep neural network operations perform machine learning methods for data analysis include as least one of application of Convolutional Neural Networks (CNNs), Recurrent Neural Network (RNNs), use of convolutional autoencoders, application of Generative Adversarial Networks (GANs), Siamese/Triplet Networks, and/or Graph Neural Networks (GNNs).
11. The transportation vehicle equipment of claim 1 , wherein the alignment facilitation includes a lane departure warning system configured to warn a driver of the transportation vehicle when the vehicle begins to move out of its lane of traffic on the roadway.
12. The transportation vehicle equipment of claim 1 , wherein the alignment facilitation includes a lane departure warning system configured to control steering of the transportation vehicle to maintain its lane of traffic on the roadway.
13. The transportation vehicle equipment of claim 1 , wherein the analysis of image data depicting a roadway on which the transportation vehicle is travelling detects a presence or absence of road treatment material on the roadway based on the at least one sensor mounted to the transportation vehicle, wherein the data indicating the presence or absence of the road treatment material is taken into consideration during the alignment facilitation so as to provide lane departure warning functionality.
14. The transportation vehicle equipment of claim 1 , wherein the characteristic patterns result from a lack of points within the LiDAR point cloud data that results where speckle is present.
15. A method of performing on-roadway localization using equipment included in a transportation vehicle, the method comprising:
generating LiDAR point cloud data based on data received from at least one LiDAR sensor mounted to the transportation vehicle, wherein the LiDAR point cloud data comprise speckle characteristics for samples of the generated data, wherein the speckle characteristics are based on optical interference of back scattered light that result from coherent illumination of object surfaces with laser light generated by the LiDAR sensor;
analyzing the LiDAR point cloud data to detect at least one road wear reference line in a roadway on which the transportation vehicle is travelling, wherein speckle patterning analysis is utilized to provide information for detecting road wear based on a variation of the road surface roughness, wherein the speckle pattern analysis analyzes the speckle characteristics in the LiDAR point cloud data for characteristic patterns indicative of the road surface roughness; and
facilitating, based on the analysis of LiDAR point cloud data, of alignment of a center of at least one wheel surface of a wheel of the transportation vehicle with the detected center of the road wear reference line,
wherein the analyzing LiDAR point cloud data analyzes LiDAR point cloud data including x, y, and z coordinates, intensity data, and radial velocity data.
16. The on-roadway transportation vehicle localization method of claim 15 , providing autonomous and/or assistive functionality for operating the transportation vehicle to travel on the roadway by the facilitating alignment of the center of the at least one wheel surface of the transportation vehicle wheel with the detected center of the road wear reference line.
17. The on-roadway transportation vehicle localization method of claim 16 , wherein the transportation vehicle includes a plurality of sensors including the at least one sensor, and wherein the providing of the autonomous and/or assistive functionality analyzes data generated by the plurality of sensors to provide the autonomous and/or assistive functionality in the transportation vehicle.
18. The on-roadway transportation vehicle localization method of claim 17 , wherein the autonomous and/or assistive functionality is provided by altering a weighting of sensor data from at least some of the plurality of sensors based on analysis of data indicating reliability and/or availability of the sensor data from the plurality of sensors.
19. The on-roadway transportation vehicle localization method of claim 15 , wherein the at least one LiDAR sensor is a Frequency Modulated Continuous Wave (FMCW) LiDAR sensor.
20. The on-roadway transportation vehicle localization method of claim 19 , wherein the FMCW LiDAR provides coherent LiDAR using a continuously-emitting laser that varies frequency through a range and compares the frequency of returned signals with corresponding local reference signal to determine distance to a target.
21. The on-roadway transportation vehicle localization method of claim 15 , wherein the analysis of LiDAR point cloud data uses deep neural network operations to perform feature extraction and classification based on low-dimensional characteristic features.
22. The on-roadway transportation vehicle localization method of claim 21 , wherein the deep neural network operations utilize deep neural networks for performing machine learning methods for data analysis to transform high-dimensional sensor data generated by the at least one sensor to characteristic low dimensional features.
23. The on-roadway transportation vehicle localization method of claim 21 , wherein the deep neural network operations utilize deep neural networks for performing machine learning methods for data analysis including speckle pattern analysis.
24. The on-roadway transportation vehicle localization method of claim 21 , wherein the deep neural network operations utilize deep neural networks for performing machine learning methods for data analysis include as least one of application of Convolutional Neural Networks (CNNs), Recurrent Neural Network (RNNs), use of convolutional autoencoders, application of Generative Adversarial Networks (GANs), Siamese/Triplet Networks, and/or Graph Neural Networks (GNNs).
25. The on-roadway transportation vehicle localization method of claim 15 , wherein the facilitating alignment includes performing operations for warning of a lane departure to warn a driver of the transportation vehicle when the vehicle begins to move out of its lane of traffic on the roadway.
26. The on-roadway transportation vehicle localization method of claim 15 , wherein the facilitating alignment includes performing lane operations for warning of a lane departure to control steering of the transportation vehicle to maintain its lane of traffic on the roadway.
27. The on-roadway transportation vehicle localization method of claim 15 , wherein the analyzing of the image data depicting a roadway on which the transportation vehicle is travelling detects a presence or absence of road treatment material on the roadway based on the at least one sensor mounted to the transportation vehicle, wherein the data indicating the presence or absence of the road treatment material is taken into consideration for facilitating alignment so as to provide lane departure warning functionality.
28. The on-roadway transportation vehicle localization method of claim 15 , wherein the characteristic patterns result from a lack of points within the LiDAR point cloud data that results where speckle is present.
29. A non-transitory, machine readable medium including machine readable software code, which, when executed on a processor, controls a method of performing on-roadway localization using equipment included in a transportation vehicle, the method comprising:
generating LiDAR point cloud data based on data received from at least one LiDAR sensor mounted to the transportation vehicle, wherein the LiDAR point cloud data comprise speckle characteristics for samples of the generated data, wherein the speckle characteristics are based on optical interference of back scattered light that result from coherent illumination of object surfaces with laser light generated by the LiDAR sensor;
analyzing the LiDAR point cloud data to detect at least one road wear reference line in a roadway on which the transportation vehicle is travelling, wherein speckle patterning analysis is utilized to provide information for detecting road wear based on a variation of the road surface roughness, wherein the speckle pattern analysis analyzes the speckle characteristics in the LiDAR point cloud data for characteristic patterns indicative of the road surface roughness; and
facilitating, based on the analysis of LiDAR point cloud data, of alignment of a center of at least one wheel surface of a transportation vehicle wheel with the detected center of the road wear reference line,
wherein the analyzing of the LiDAR point cloud data analyzes LiDAR point cloud data including x, y, and z coordinates, intensity data, and radial velocity data.