Divide-and-conquer for lane-aware diverse trajectory prediction
A method for driving path prediction is provided. The method concatenates past trajectory features and lane centerline features in a channel dimension at an agent's respective location in a top view map to obtain concatenated features thereat. The method obtains convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene the vehicle and agent interactions. The method extracts hypercolumn descriptor vectors which include the convolutional features from the agent's respective location in the top view map. The method obtains primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors. The method generates a respective score for each of the primary and auxiliary trajectory predictions. The method trains a vehicle trajectory prediction neural network using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions.
1. A computer-implemented method for driving path prediction and current trajectory control by an advanced driver-assistance system (ADAS) that is integrated into an autonomous vehicle, comprising:
obtaining a top view map, a past trajectory, and lane centerlines for the autonomous vehicle in a training scene as initial training inputs;
ranking the lane centerlines based on heuristics including trajectory distance along a lane score and a centerline yaw score;
concatenating past trajectory features and lane centerline features in a channel dimension at an agent's respective location in the top view map of the training scene to obtain concatenated features thereat;
obtaining, by a convolutional encoder of the ADAS in a single forward pass, convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene that includes the vehicle and interactions with agents in the training scene;
extracting, by a hypercolumn trajectory encoder of the ADAS, hypercolumn descriptor vectors from the convolutional features, the hypercolumn descriptor vectors including the convolutional features from the agent's respective location in the top view map and an interpolated location in subsequent lower convolutional layers;
obtaining, by a hypercolumn trajectory decoder of the ADAS, primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors;
generating, by an Inverse Optimal Control (IOC) based ranking module of the ADAS, a respective score for each of the primary and auxiliary trajectory predictions;
training a vehicle trajectory prediction neural network of the ADAS using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions;
generating a trajectory prediction of the autonomous vehicle using the vehicle trajectory prediction neural network; and
using the ADAS to autonomously control a current trajectory of the autonomous vehicle based on the trajectory prediction, where the ADAS autonomously controls the vehicle using one or more of a steering system, a braking system, and an accelerating system of the vehicle.
2. The computer-implemented method of claim 1 , further comprising encoding, by a past trajectory encoder, the past trajectory to obtain the past trajectory features.
3. The computer-implemented method of claim 1 , further comprising encoding, by a centerline encoder, the lane centerlines to obtain the lane centerline features.
4. The computer-implemented method of claim 1 , further comprising:
generating, by the trained vehicle trajectory prediction neural network, a trajectory prediction of the vehicle based on a current scene; and
controlling a vehicle system to control a current vehicle trajectory for collision avoidance based on the trajectory prediction.
5. The computer-implemented method of claim 1 , wherein the hypercolumn trajectory decoder comprises a plurality of 1×1 convolutions producing a plurality of outputs for each of the agents.
6. The computer-implemented method of claim 1 , wherein the primary predictions are in normal tangential coordinates, and wherein the auxiliary features are in global cartesian coordinates of the top view map to regularize the primary predictions.
7. The computer-implemented method of claim 1 , wherein the IOC loss objective maximizes a cumulative rewards for each of the primary trajectory predictions.
8. The computer-implemented method of claim 1 , wherein the hypercolumn descriptors vectors capture interactions and a global context of the training scene at different scales with respect to each of the agents present in the scene.
9. A computer program product for driving path prediction and current trajectory control by an advanced driver-assistance system (ADAS) that is integrated into an autonomous vehicle, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
obtaining a top view map, a past trajectory, and lane centerlines for the autonomous vehicle in a training scene as initial training inputs;
ranking the lane centerlines based on heuristics including trajectory distance along a lane score and a centerline yaw score;
concatenating past trajectory features and lane centerline features in a channel dimension at an agent's respective location in the top view map of the training scene to obtain concatenated features thereat;
obtaining, by a convolutional encoder of the ADAS in a single forward pass, convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene that includes the vehicle and interactions with agents in the training scene;
extracting, by a hypercolumn trajectory encoder of the ADAS, hypercolumn descriptor vectors from the convolutional features, the hypercolumn descriptor vectors including the convolutional features from the agent's respective location in the top view map and an interpolated location in subsequent lower convolutional layers;
obtaining, by a hypercolumn trajectory decoder of the ADAS, primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors;
generating, by an Inverse Optimal Control (IOC) based ranking module of the ADAS, a respective score for each of the primary and auxiliary trajectory predictions;
training a vehicle trajectory prediction neural network of the ADAS using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions;
generating a trajectory prediction of the autonomous vehicle using the vehicle trajectory prediction neural network; and
using the ADAS to autonomously controlling a current trajectory of the autonomous vehicle based on the trajectory prediction, where the ADAS autonomously controls the vehicle using one or more of a steering system, a braking system, and an accelerating system of the vehicle.
10. The computer program product of claim 9 , further comprising encoding, by a past trajectory encoder, the past trajectory to obtain the past trajectory features.
11. The computer program product of claim 9 , further comprising encoding, by a centerline encoder, the lane centerlines to obtain the lane centerline features.
12. The computer program product of claim 9 , further comprising:
generating, by the trained vehicle trajectory prediction neural network, a trajectory prediction of the vehicle based on a current scene; and
controlling a vehicle system to control a current vehicle trajectory for collision avoidance based on the trajectory prediction.
13. The computer program product of claim 9 , wherein the hypercolumn trajectory decoder comprises a plurality of 1×1 convolutions producing a plurality of outputs for each of the agents.
14. The computer program product of claim 9 , wherein the primary predictions are in normal tangential coordinates, and wherein the auxiliary features are in global cartesian coordinates of the top view map to regularize the primary predictions.
15. The computer program product of claim 9 , wherein the IOC loss objective maximizes a cumulative rewards for each of the primary trajectory predictions.
16. A computer processing system for driving path prediction and current trajectory control by an advanced driver-assistance system (ADAS) that is integrated into an autonomous vehicle, comprising:
a memory device for storing program code; and
a processor device operatively coupled to the memory device for running the program code to:
obtain a top view map, a past trajectory, and lane centerlines for the autonomous vehicle in a training scene as initial training inputs;
rank the lane centerlines based on heuristics including trajectory distance along a lane score and a centerline yaw score;
concatenate past trajectory features and lane centerline features in a channel dimension at an agent's respective location in the top view map of the training scene to obtain concatenated features thereat;
obtain, by a convolutional encoder of the ADAS in a single forward pass, convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene that includes the vehicle and interactions with agents in the training scene;
extract, by a hypercolumn trajectory encoder of the ADAS, hypercolumn descriptor vectors from the convolutional features, the hypercolumn descriptor vectors including the convolutional features from the agent's respective location in the top view map and an interpolated location in subsequent lower convolutional layers;
obtain, by a hypercolumn trajectory decoder of the ADAS, primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors;
generate, by an Inverse Optimal Control (IOC) based ranking module of the ADAS, a respective score for each of the primary and auxiliary trajectory predictions;
train a vehicle trajectory prediction neural network of the ADAS using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions;
generate a trajectory prediction of the autonomous vehicle using the vehicle trajectory prediction neural network; and
use the ADAS to autonomously control a current trajectory of the autonomous vehicle based on the trajectory prediction, where the ADAS autonomously controls the vehicle using one or more of a steering system, a braking system, and an accelerating system of the vehicle.