Method, device and storage media for multi-agent motion prediction

1 . A multi-agent motion prediction method, based on coordinate information obtained using GPS and position information captured by a camera in a traffic scenario, which are received by a prediction module for symmetrical modeling, the method comprising:

a. taking each of a plurality of agents in the traffic scenario as a central agent respectively, and dividing the traffic scenario into different areas, each area associated with a respective central agent and serving as a local area for the respective central agent;

b. obtaining a local eigenvector for each of the central agents in its associated local area, comprising:

applying one or more multilayer perceptron (MLP) models and one or more space-domain learnable matrices that are optimized during a training process using a stochastic gradient descent algorithm to:

obtain time-dependent information of the central agent from features of the central agent at each of one or more time steps;

obtain agent-agent interaction information of the central agent and its adjacent agent;

obtain agent-road segment interaction information of the central agent and road segments in its associated local area; and

aggregate the time-dependent information and the agent-agent interaction information and the agent-road segment interaction information in generating the local eigenvector;

wherein each of the local eigenvectors represents (i) a relationship between the central agent and its adjacent agent, (ii) a relationship between the central agent and a lane, and (iii) a relationship between a past status and a current status of the central agent in its associated local area;

c. correcting a coordinate system of local eigenvectors between all of the central agents, comprising:

obtaining a mapping matrix using a mapping-matrix MLP model; and

applying the mapping matrix together with global-domain learnable matrices obtained through a training process to the local eigenvectors in obtaining long-range dependencies between each of the central agents representing relationships or interactions among the central agents across different local areas;

d. predicting a motion for each of the central agents in accordance with the local eigenvectors of the central agents and the long-range dependencies, wherein the predicting comprises:

splicing a trajectory segment vector together with a first semantic attribute corresponding to the trajectory segment vector, wherein the trajectory segment vector is derived from coordinate data representing movement of each of the agents, and the first semantic attribute provides descriptive information related to the corresponding agent;

splicing a lane segment vector together with a second semantic attribute corresponding to the lane segment vector, wherein the lane segment vector is derived from coordinate data representing lane geometry, and the second semantic attribute provides descriptive information related to the lane;

inputting a result of either the spliced trajectory segment vector or the spliced lane segment vector to a trained decoder comprising a trajectory-decoder MLP model, the trained decoder comprising learnable parameters that are trained using a stochastic gradient descent algorithm; and

outputting, by the trained decoder, six predicted trajectories for each of the agents consisting of several two-dimensional coordinate points and corresponding probability values based on the long-range dependencies.

2 . A computing device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program is executable by the processor that implements the multi-agent motion prediction method of claim 1 .

3 . A multi-agent motion prediction method, comprising:

determining coordinate information of a first traffic participant in a traffic scenario using GPS;

capturing, by a camera, position information of a plurality of second traffic participants within a current field of view of the camera in the traffic scenario;

obtaining, by a prediction module, the coordinate information provided by using GPS and the position information from the camera; and

performing, by the prediction module, symmetrical modeling of a plurality of agents in the traffic scenario based on the second traffic participants, wherein the multi-agent motion prediction method further comprises:

a. taking, by the prediction module, each of the agents in the traffic scenario as a central agent respectively, and dividing the traffic scenario into different areas, each area associated with a respective central agent and serving as a local area for the respective central agent;

b. obtaining, by the prediction module, a local eigenvector for each of the central agents in its associated local area, comprising:

applying one or more multilayer perceptron (MLP) models and one or more space-domain learnable matrices that are optimized during a training process using a stochastic gradient descent algorithm to:

obtain time-dependent information of the central agent from features of the central agent at each of one or more time steps;

obtain agent-agent interaction information of the central agent and its adjacent agent;

obtain agent-road segment interaction information of the central agent and road segments in its associated local area; and

aggregate the time-dependent information and the agent-agent interaction information and the agent-road segment interaction information in generating the local eigenvector;

wherein each of the local eigenvectors represents (i) a relationship between the central agent and its adjacent agent, (ii) a relationship between the central agent and a lane, and (iii) a relationship between a past status and a current status of the central agent in its associated local area;

c. correcting, by the prediction module, a coordinate system of local eigenvectors between all of the central agents, comprising:

obtaining a mapping matrix using a mapping-matrix MLP model; and

applying the mapping matrix together with global-domain learnable matrices obtained through a training process to the local eigenvectors in obtaining long-range dependencies between each of the central agents representing relationships or interactions among the central agents across different local areas; and

d. predicting, by the prediction module, a motion for each of the central agents in accordance with the local eigenvectors of the central agents and the long-range dependencies, wherein the predicting comprises:

splicing a trajectory segment vector together with a first semantic attribute corresponding to the trajectory segment vector, wherein the trajectory segment vector is derived from coordinate data representing movement of each of the agents, and the first semantic attribute provides descriptive information related to the corresponding agent;

splicing a lane segment vector together with a second semantic attribute corresponding to the lane segment vector, wherein the lane segment vector is derived from coordinate data representing lane geometry, and the second semantic attribute provides descriptive information related to the lane;

inputting a result of either the spliced trajectory segment vector or the spliced lane segment vector to a trained decoder comprising a trajectory-decoder MLP model, the trained decoder comprising learnable parameters that are trained using a stochastic gradient descent algorithm; and

outputting, by the trained decoder, six predicted trajectories for each of the agents consisting of several two-dimensional coordinate points and corresponding probability values based on the long-range dependencies.

4 . The method of claim 3 , wherein the traffic scenario is divided into different areas by determining, for each central agent, a local area within a radius of 50 meters centered on the central agent.

5 . The method of claim 3 , wherein an environment where the central agent is located comprises traffic participants within a radius of 50 meters from the central agent and map elements including lanes within the radius of 50 meters from the central agent.

6 . The method of claim 5 , wherein the traffic participants comprise vehicles, motorcycles, bicycles, and pedestrians that are encountered during vehicle driving and are within the radius of 50 meters from the central agent.