Control parameter based search space for vehicle motion planning
A method for generating a trajectory for a vehicle based on an abstract space of control parameters associated with the vehicle may include applying a machine learning model to determine an abstract space representation of the trajectory, which includes a sequence of control parameters associated with the vehicle. For instance, application of the machine learning model may include performing a search of the abstract space parameterized by control parameters that include derivatives of the position of the vehicle. Examples of control parameters include velocity, acceleration, jerk, and snap. A physical space representation of the trajectory may be determined by at least mapping the sequence of control parameters to a sequence of positions for the vehicle. A motion of the vehicle may be controlled based at least on the physical space representation of the trajectory. Related systems and computer program products are also provided.
1 . A method, comprising:
generating, using at least one data processor, an abstract space representation of a trajectory of a vehicle by application of a machine learning model, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle;
generating, using the at least one data processor, a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
generating, using the at least one data processor, instructions for causing a motion of the vehicle based at least on the physical space representation of the trajectory.
2 . The method of claim 1 , wherein:
each transition between successive nodes comprises an action that is associated with a reward generated by a reward function, and wherein a search is performed based on the reward function such that the sequence of control parameters comprising the abstract space representation of the trajectory is associated with a maximum cumulative reward.
3 . The method of claim 2 , further comprising:
training, using the at least one data processor, the machine learning model to learn the reward function based on one or more demonstrations of expert behavior.
4 . The method of claim 3 , wherein:
the machine learning model is trained by applying one or more of inverse reinforcement learning (IRL), associative reinforcement learning, deep reinforcement learning, safe reinforcement learning, and partially supervised reinforcement learning (PSRL), and the generating of the abstract space representation of the trajectory by the application of the machine learning model is based at least on performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters.
5 . The method of claim 4 , wherein the search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.
6 . The method of claim 4 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and generating, based at least on the first function that is integrated and one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.
7 . The method of claim 4 , wherein the sequence of positions include a first position of the vehicle at a first time and a second position of the vehicle at a second time.
8 . The method of claim 7 , wherein each control parameter in the sequence of control parameters further comprise a low-degree polynomial modeling a change in two or more successive control parameters.
9 . The method of claim 8 , wherein the first position and the second position each comprise a set of two-dimensional spatial coordinates or a set of three-dimensional spatial coordinates.
10 . The method of claim 1 , wherein the abstract space representation of the trajectory and the physical space representation of the trajectory are generated based on one or more initial conditions of the vehicle.
11 . The method of claim 1 , wherein the sequence of control parameters include a first control parameter of the vehicle at a first time and a second control parameter of the vehicle at a second time.
12 . The method of claim 1 , wherein each control parameter in the sequence of control parameters comprise a velocity, an acceleration, a jerk, or a snap.
13 . A system, comprising:
at least one processor, and
at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
generating, using at least one data processor, an abstract space representation of a trajectory of a vehicle by application of a machine learning model, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle;
generating, using the at least one data processor, a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
generating, using the at least one data processor, instructions for causing a motion of the vehicle based at least on the physical space representation of the trajectory.
14 . The system of claim 13 , wherein:
each transition between successive nodes comprises an action that is associated with a reward generated by a reward function, and wherein a search is performed based on the reward function such that the sequence of control parameters comprising the abstract space representation of the trajectory is associated with a maximum cumulative reward.
15 . The system of claim 14 , wherein the operations further comprise:
training, using the at least one processor, the machine learning model to learn the reward function based on one or more demonstrations of expert behavior.
16 . The system of claim 15 , wherein:
the machine learning model is trained by applying one or more of inverse reinforcement learning (IRL), associative reinforcement learning, deep reinforcement learning, safe reinforcement learning, and partially supervised reinforcement learning (PSRL), and
the generating of the abstract space representation of the trajectory by the application of the machine learning model is based at least on performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters.
17 . The system of claim 16 , wherein the search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.
18 . The system of claim 16 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and generating, based at least on the first function that is integrated and one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.
19 . The system of claim 16 , wherein the sequence of positions include a first position of the vehicle at a first time and a second position of the vehicle at a second time.
20 . The system of claim 19 , wherein each control parameter in the sequence of control parameters further comprise a low-degree polynomial modeling a change in two or more successive control parameters.
21 . The system of claim 20 , wherein the first position and the second position each comprise a set of two-dimensional spatial coordinates or a set of three-dimensional spatial coordinates.
22 . The system of claim 13 , wherein the abstract space representation of the trajectory and the physical space representation of the trajectory are generated based on one or more initial conditions of the vehicle.
23 . The system of claim 13 , wherein the sequence of control parameters include a first control parameter of the vehicle at a first time and a second control parameter of the vehicle at a second time.
24 . The system of claim 13 , wherein each control parameter in the sequence of control parameters comprise a velocity, an acceleration, a jerk, or a snap.
25 . At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:
generating, using at least one data processor, an abstract space representation of a trajectory of a vehicle by application of a machine learning model, the abstract space representation of the trajectory including a sequence of control parameters associated with the vehicle, each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle;
generating, using the at least one data processor, a physical space representation of the trajectory by at least mapping the sequence of control parameters to a sequence of positions for the vehicle; and
generating, using the at least one data processor, instructions for causing a motion of the vehicle based at least on the physical space representation of the trajectory.
26 . The at least one non-transitory storage media of claim 25 , wherein:
the machine learning model is trained by applying one or more of inverse reinforcement learning (IRL), associative reinforcement learning, deep reinforcement learning, safe reinforcement learning, and partially supervised reinforcement learning (PSRL),
the generating of the abstract space representation of the trajectory by the application of the machine learning model is based at least on performing a Monte Carlo Tree Search (MCTS) of a search tree having a plurality of nodes representative of a plurality of different control parameters,
the abstract space representation of the trajectory and the physical space representation of the trajectory are generated based on one or more initial conditions of the vehicle, and
each control parameter in the sequence of control parameters corresponding to a derivative of a position of the vehicle.
27 . The at least one non-transitory storage media of claim 26 , wherein the sequence of control parameters is mapped to the sequence of positions for the vehicle by at least integrating a first function corresponding to the sequence of control parameters and generating, based at least on the first function that is integrated and the one or more initial conditions of the vehicle, a second function corresponding to the sequence of positions.
28 . The at least one non-transitory storage media of claim 26 , wherein a search of the search tree is confined, based on one or more kinematic constraints and/or physical constraints associated with the vehicle, to prevent one or more control parameter values from being included in the abstract space representation of the trajectory.