Systems and methods for using attention masks to improve motion planning
Systems and methods for generating attention masks are provided. In particular, a computing system can access sensor data and map data for an area around an autonomous vehicle. The computing system can generate a voxel grid representation of the sensor data and map data. The computing system can generate an attention mask based on the voxel grid representation. The computing system can generate, by using the voxel grid representation and the attention mask as input to a machine-learned model, an attention weighted feature map. The computing system can determine using the attention weighted feature map, a planning cost volume for an area around the autonomous vehicle. The computing system can select a trajectory for the autonomous vehicle based, at least in part, on the planning cost volume.
1. A computer-implemented method for improving autonomous vehicle motion planning, the method comprising:
accessing, by a computing system including one or more processors, sensor data and map data for an area around an autonomous vehicle:
generating, by the computing system, a voxel grid representation of the sensor data and the map data;
generating, by the computing system, an attention mask based on the voxel grid representation, wherein the attention mask is generated using a first machine-learned model trained to generate a grid of attention values, each of the attention values representing an importance of the map data to navigating the autonomous vehicle and wherein the importance of an attention value of the attention values is determined, at least in part, on a position of the respective attention value relative to the autonomous vehicle;
generating, by the computing system by using the voxel grid representation and the attention mask as input to a second machine-learned model, an attention weighted feature map;
determining, by the computing system, using the attention weighted feature map, a planning cost volume for the area around the autonomous vehicle;
selecting, by the computing system, a trajectory for the autonomous vehicle based, at least in part, on the planning cost volume; and
controlling, by the computing system, the autonomous vehicle based on the trajectory.
2. The computer-implemented method of claim 1 , wherein each respective attention value is associated with a particular location in the voxel grid representation.
3. The computer-implemented method of claim 2 , wherein the attention value is a scalar value.
4. The computer-implemented method of claim 3 , wherein the scalar value is associated with an importance associated with the particular location of the voxel grid representation.
5. The computer-implemented method of claim 4 , wherein the scalar value is between zero and one and wherein generating, by the computing system by using the voxel grid representation and the attention mask as input to the second machine-learned model, the attention weighted feature map further comprises:
for each respective scalar value:
determining whether the respective scalar value exceeds a predetermined threshold value;
in accordance with a determination that the respective scalar value exceeds the predetermined threshold value, setting the respective scalar value to one; and
in accordance with a determination that the respective scalar value does not exceed the predetermined threshold value, setting the respective scalar value to zero.
6. The computer-implemented method of claim 1 , wherein generating, by the computing system by using the voxel grid representation and the attention mask as input to the second machine-learned model, the attention weighted feature map further comprises:
generating, by the computing system, a sparse feature map by multiplying the voxel grid representation and the attention mask; and
receiving, by the computing system, the attention weighted feature map as output from one or more machine-learned models using the sparse feature map and the attention mask as input.
7. The computer-implemented method of claim 6 , wherein generating, by the computing system by using the voxel grid representation and the attention mask as input to the second machine-learned model, the attention weighted feature map further comprises:
concatenating, by the computing system, the attention weighted feature map with the voxel grid representation.
8. The computer-implemented method of claim 1 , wherein the planning cost volume comprises a cost associated with each position in the planning cost volume.
9. The computer-implemented method of claim 8 , wherein the planning cost volume comprises costs associated with each said position and time step described by the planning cost volume.
10. The computer-implemented method of claim 9 , wherein selecting, by the computing system, the trajectory for the autonomous vehicle based, at least in part, on the planning cost volume further comprises:
generating, by the computing system, a plurality of candidate trajectories for the autonomous vehicle, each candidate trajectory comprising a series of points that represent a route of the autonomous vehicle through the area around the autonomous vehicle, each point being associated with a particular point in time.
11. The computer-implemented method of claim 10 , wherein selecting, by the computing system, the trajectory for the autonomous vehicle based, at least in part, on the planning cost volume further comprises:
for a respective candidate trajectory in the plurality of candidate trajectories:
determining, by the computing system, a cost for each respective point in the series of points associated with the respective candidate trajectory by using a location of the respective point and the associated point in time to identify a predicted cost of the point in the planning cost volume; and
aggregating, by the computing system, the costs for each position in the series of points into a total cost associated for the respective candidate trajectory.
12. The computer-implemented method of claim 10 , wherein selecting, by the computing system, the trajectory for the autonomous vehicle based, at least in part, on the planning cost volume further comprises:
selecting, by the computing system, the candidate trajectory with a lowest total cost.
13. The computer-implemented method of claim 1 , further comprises:
generating, by the computing system, a list of detected objects based, at least in part, on the attention weighted feature map.
14. A computing system for improving autonomous vehicle motion planning for an autonomous vehicle, the system comprising:
one or more processors and one or more non-transitory computer-readable memories;
wherein the one or more non-transitory computer-readable memories store instructions that, when executed by the processors, cause the computing system to perform operations, the operations comprising:
accessing sensor data and map data for an area around the autonomous vehicle;
generating a voxel grid representation of the sensor data and the map data;
generating an attention mask based on the voxel grid representation, wherein the attention mask is generated using a first machine-learned model trained to generate a grid of attention values, each of the attention values representing an importance of the map data to navigating the autonomous vehicle and wherein the importance of an attention value in the attention values is determined, at least in part, on a position of the respective attention value relative to the autonomous vehicle;
generating, by using the voxel grid representation and the attention mask as input to a second machine-learned model, an attention weighted feature map;
determining, by using the attention weighted feature map, a planning cost volume for the area around the autonomous vehicle;
selecting a trajectory for the autonomous vehicle based on the planning cost volume; and
controlling, by the computing system, the autonomous vehicle based on the trajectory.