World model generation and correction for autonomous vehicles
Systems and methods of generating and updating a world model for autonomous vehicle navigation are disclosed. An autonomous vehicle system can receive sensor data from a plurality of sensors of an autonomous vehicle, where the sensor data is captured during operation of the autonomous vehicle; access a world model generated based at least on map information corresponding to a location of the operation of the autonomous vehicle; determine at least one semantic correction for the world model based on the sensor data; determine at least one geometric correction for the world model based on the sensor data and the map information; and generate an updated world model based on the at least one semantic correction and the at least one geometric correction.
1 . A system, comprising:
at least one processor coupled to non-transitory memory, the at least one processor configured to:
retrieve, from a world model, expected semantic data for a road traveled by an autonomous vehicle;
receive sensor data from a plurality of sensors of the autonomous vehicle, the sensor data captured during operation of the autonomous vehicle;
detect semantic attributes of the road based on the sensor data;
detect an error in the expected semantic data based on the sensor data by:
comparing the detected semantic attributes with corresponding semantic attributes of the road in the expected semantic data,
wherein the error indicates a semantic error in the world model that mismatches an attribute of the road;
generate a correction to the world model based on the error;
modify the world model based on the correction; and
navigate the autonomous vehicle based at least in part on the modified world model.
2 . The system of claim 1 , wherein the at least one processor is further configured to modify a speed limit identified in the world model based on the correction.
3 . The system of claim 1 , wherein the plurality of sensors comprises one or more of a light detection and ranging (LiDAR) sensor, a radar sensor, a camera, or an inertial measurement unit (IMU).
4 . The system of claim 1 , wherein the expected semantic data comprises one or more of a speed limit for the road, a lane type of a lane of the road, a presence of a road sign corresponding of the road, or a type of the road sign.
5 . The system of claim 1 , wherein the at least one processor is further configured to transmit the correction to at least one server to correct corresponding map information.
6 . The system of claim 1 , wherein the at least one processor is further configured to:
detect, based on the sensor data, one or more objects corresponding to the road traveled by the autonomous vehicle; and
generate additional semantic data for the road based on a classification of the one or more objects.
7 . The system of claim 6 , wherein the at least one processor is further configured to generate the correction based on the additional semantic data.
8 . A method, comprising:
retrieving, by at least one processor coupled to non-transitory memory, from a world model, expected semantic data for a road traveled by an autonomous vehicle;
receiving, by the at least one processor, sensor data from a plurality of sensors of the autonomous vehicle, the sensor data captured during operation of the autonomous vehicle;
detecting semantic attributes of the road based on the sensor data;
detecting, by the at least one processor, an error in the expected semantic data based on the sensor data by:
comparing the detected semantic attributes with corresponding semantic attributes of the road in the expected semantic data,
wherein the error indicates a semantic error in the world model that mismatches an attribute of the road;
generating a correction to the world model based on the error;
modifying the world model based on the correction; and
navigating the autonomous vehicle based at least in part on the modified world model.
9 . The method of claim 8 , further comprising modifying, by the at least one processor, a speed limit identified in the world model based on the correction.
10 . The method of claim 8 , wherein the plurality of sensors comprises one or more of a light detection and ranging (LiDAR) sensor, a radar sensor, a camera, or an inertial measurement unit (IMU).
11 . The method of claim 8 , wherein the expected semantic data comprises one or more of a speed limit for the road, a lane type of a lane of the road, a presence of a road sign corresponding of the road, or a type of the road sign.
12 . The method of claim 8 , further comprising transmitting, by the at least one processor, the correction to at least one server to correct corresponding map information.
13 . The method of claim 8 , further comprising:
detecting, by the at least one processor, based on the sensor data, one or more objects corresponding to the road traveled by the autonomous vehicle; and
generating, by the at least one processor, additional semantic data for the road based on a classification of the one or more objects.
14 . The method of claim 13 , further comprising generating, by the at least one processor, the correction based on a comparison of the additional semantic data and the expected semantic data retrieved from the world model.
15 . An autonomous vehicle, comprising:
a plurality of sensors; and
at least one processor coupled to non-transitory memory, the at least one processor configured to:
receive, during operation of the autonomous vehicle, sensor data from the plurality of sensors;
detect semantic attributes of a road based on the sensor data;
detect, based on the sensor data, an error in expected semantic data of a world model used in navigation of the autonomous vehicle by:
comparing the detected semantic attributes with corresponding semantic attributes of the road in the expected semantic data,
wherein the error indicates a semantic error in the world model that mismatches an attribute of the road;
generate an updated world model based on the error; and
navigate the autonomous vehicle based at least in part on the updated world model.
16 . The autonomous vehicle of claim 15 , wherein the plurality of sensors comprises one or more of a light detection and ranging (LiDAR) sensor, a radar sensor, a camera, or an inertial measurement unit (IMU).
17 . The autonomous vehicle of claim 15 , wherein the at least one processor is further configured to:
detect, based on the sensor data, one or more objects corresponding to a road traveled by the autonomous vehicle; and
generate additional semantic data for the road based on a classification of the one or more objects.
18 . The autonomous vehicle of claim 17 , wherein the at least one processor is further configured to detect the error based on a mismatch between the expected semantic data of the world model and the additional semantic data.