SURROUNDING AWARE TRAJECTORY PREDICTION
A method of predicting vehicle trajectory includes operating a scene encoder on an environmental representation surrounding a vehicle; concatenating an output of the scene encoder with a history trajectory; applying a sequence encoder to a result of the concatenating; refining an output of the sequence encoder based on the history trajectory; and generating one or more predicted future trajectories by operating a decoder on an output of the refining.
1 . A method of predicting vehicle trajectory, comprising:
receiving information indicative of a surrounding environment of a vehicle;
receiving a history of vehicle trajectories;
determining learned patterns by separately operating a first encoder on the surrounding information and a second encoder on the history of vehicles trajectories; and
determining one or more predicted future trajectories for the vehicle based on the learned patterns.
2 . The method of claim 1 , wherein the information indicative of the surrounding environment is represented as a combination of a vectorized representation and a rasterized representation.
3 . The method of claim 1 , wherein the information indicative of the surrounding environment includes a first channel indicating a surrounding environment and a surrounding vehicle dynamics and a second channel indicating legally reachable areas for the vehicle.
4 . The method of claim 1 , wherein the first encoder comprises a convolutional neural network.
5 . The method of claim 1 , wherein the first encoder comprises a three-dimensional 3D convolutional layer, followed by a 3D average pooling stage, followed by a squeeze and a two-dimensional 2D convolutional layer.
6 . The method of claim 1 , wherein the second encoder comprises a cascade of a long short-term memory encoder, a 1D convolutional layer, a maxpooling stage and a 1D convolutional layer.
7 . The method of claim 1 , wherein learned patterns are determined by refining a weighted combination of an output of the second encoder and the history of vehicle trajectories.
8 . The method of claim 1 , wherein the one or more predicted future trajectories are determined by operating a decoder comprising a recurrent neural network and a stage for conversion to a dense layer that operates on an output of a refining stage.
9 . The method of claim 1 , wherein the surrounding environment comprises a street intersection.
10 . The method of claim 1 , wherein the one or more future trajectories includes a trajectory of the vehicle near an intersection, along a curvy road, while turning or while making a lane change.
11 . A method of predicting vehicle trajectory, comprising:
operating a scene encoder on an environmental representation surrounding a vehicle;
concatenating an output of the scene encoder with a history trajectory;
applying a sequence encoder to a result of the concatenating;
refining an output of the sequence encoder based on the history trajectory; and
generating one or more predicted future trajectories by operating a decoder on an output of the refining.
12 . The method of claim 11 , wherein the environmental representation surrounding the vehicle is represented as a combination of a vectorized representation and a rasterized representation.
13 . The method of claim 11 , wherein the environmental representation is a two channel image in which a first channel indicating a surrounding environment and a surrounding vehicle dynamics and a second channel indicating legally reachable areas for the vehicle.
14 . The method of claim 11 , wherein the scene encoder comprises a convolutional neural network.
15 . The method of claim 11 , wherein the scene encoder comprises a 3D convolutional layer, followed by a 3D average pooling stage, followed by a squeeze and a 2D convolutional layer.
16 . The method of claim 11 , wherein the sequence encoder comprises a cascade of a long short-term memory encoder, a 1D convolutional layer, a maxpooling stage and a 1D convolutional layer.
17 . The method of claim 11 , wherein the refining uses a weighted combination of the output of the sequence encoder and the history trajectory.
18 . The method of claim 11 , wherein the decoder comprises a recurrent neural network and a stage for conversion to a dense layer that is used for generating one or more predicted future trajectories.
19 . The method of claim 11 , wherein the one or more future trajectories includes a trajectory of the vehicle near an intersection, along a curvy road, while turning or while making a lane change.
20 . An apparatus comprising one or more processors configured to implement a method, comprising;
receiving information indicative of a surrounding environment of a vehicle;
receiving a history of vehicle trajectories;
determining learned patterns by separately operating a first encoder on the surrounding information and a second encoder on the history of vehicles trajectories; and
determining one or more predicted future trajectories for the vehicle based on the learned patterns.