Control parameter based search space for vehicle motion planning
A method for generating a trajectory for a vehicle based on an abstract space of control parameters associated with the vehicle may include applying a machine learning model to determine an abstract space representation of the trajectory, which includes a sequence of control parameters associated with the vehicle. For instance, application of the machine learning model may include performing a search of the abstract space parameterized by control parameters that include derivatives of the position of the vehicle. Examples of control parameters include velocity, acceleration, jerk, and snap. A physical space representation of the trajectory may be determined by at least mapping the sequence of control parameters to a sequence of positions for the vehicle. A motion of the vehicle may be controlled based at least on the physical space representation of the trajectory. Related systems and computer program products are also provided.
1. A method, comprising:
applying, using at least one data processor, a machine learning model to determine an abstract space representation of a trajectory for a vehicle, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle, the machine learning model determining the abstract space representation of the trajectory by at least performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters;
determining, using the at least one data processor, a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
controlling, using the at least one data processor, a motion of the vehicle based at least on the physical space representation of the trajectory.
2. The method of claim 1 , wherein each transition between successive nodes comprises an action that is associated with a reward determined by a reward function, and wherein the search is performed based on the reward function such that the sequence of control parameters comprising the abstract space representation of the trajectory is associated with a maximum cumulative reward.
3. The method of claim 2 , further comprising:
training, using the at least one data processor, the machine learning model to learn the reward function based on one or more demonstrations of expert behavior.
4. The method of claim 3 , wherein the machine learning model is trained by applying one or more of inverse reinforcement learning (IRL), associative reinforcement learning, deep reinforcement learning, safe reinforcement learning, and partially supervised reinforcement learning (PSRL).
5. The method of claim 2 , wherein the search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.
6. The method of claim 1 , wherein the abstract space representation of the trajectory and the physical space representation of the trajectory are determined based on one or more initial conditions of the vehicle.
7. The method of claim 1 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and determining, based at least on the integrated first function and one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.
8. The method of claim 1 , wherein the sequence of control parameters include a first control parameter of the vehicle at a first time and a second control parameter of the vehicle at a second time.
9. The method of claim 1 , wherein each control parameter in the sequence of control parameters comprise a velocity, an acceleration, a jerk, or a snap.
10. The method of claim 1 , wherein each control parameter in the sequence of control parameters further comprise a low-degree polynomial modeling a change in two or more successive control parameters.
11. The method of claim 1 , wherein the sequence of positions include a first position of the vehicle at a first time and a second position of the vehicle at a second time.
12. The method of claim 11 , wherein the first position and the second position each comprise a set of two-dimensional spatial coordinates or a set of three-dimensional spatial coordinates.
13. A system, comprising:
at least one processor, and
at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
apply a machine learning model to determine an abstract space representation of a trajectory for a vehicle, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle, the machine learning model determining the abstract space representation of the trajectory by at least performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters;
determine a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
control a motion of the vehicle based at least on the physical space representation of the trajectory.
14. The system of claim 13 , wherein each transition between successive nodes comprises an action that is associated with a reward determined by a reward function, and wherein the search is performed based on the reward function such that the sequence of control parameters comprising the abstract space representation of the trajectory is associated with a maximum cumulative reward.
15. The system of claim 14 , wherein the operations further comprise:
training, using the at least one data processor, the machine learning model to learn the reward function based on one or more demonstrations of expert behavior.
16. The system of claim 15 , wherein the machine learning model is trained by applying one or more of inverse reinforcement learning (IRL), associative reinforcement learning, deep reinforcement learning, safe reinforcement learning, and partially supervised reinforcement learning (PSRL).
17. The system of claim 13 , wherein the search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.
18. The system of claim 13 , wherein the abstract space representation of the trajectory and the physical space representation of the trajectory are determined based on one or more initial conditions of the vehicle.
19. The system of claim 13 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and determining, based at least on the integrated first function and one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.
20. The system of claim 13 , wherein the sequence of control parameters include a first control parameter of the vehicle at a first time and a second control parameter of the vehicle at a second time.
21. The system of claim 13 , wherein each control parameter in the sequence of control parameters comprise a velocity, an acceleration, a jerk, or a snap.
22. The system of claim 13 , wherein each control parameter in the sequence of control parameters further comprise a low-degree polynomial modeling a change in two or more successive control parameters.
23. The system of claim 13 , wherein the sequence of positions include a first position of the vehicle at a first time and a second position of the vehicle at a second time.
24. The system of claim 23 , wherein the first position and the second position each comprise a set of two-dimensional spatial coordinates or a set of three-dimensional spatial coordinates.
25. At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:
apply, using at least one data processor, a machine learning model to determine an abstract space representation of a trajectory for a vehicle, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle, the machine learning model determining the abstract space representation of the trajectory by at least performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters;
determine, using the at least one data processor, a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
control, using the at least one data processor, a motion of the vehicle based at least on the physical space representation of the trajectory.
26. The at least one non-transitory storage media of claim 25 , wherein the search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.
27. The at least one non-transitory storage media of claim 25 , wherein the abstract space representation of the trajectory and the physical space representation of the trajectory are determined based on one or more initial conditions of the vehicle.
28. The at least one non-transitory storage media of claim 25 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and determining, based at least on the integrated first function and one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.