System and method for utilizing model predictive control for optimal interactions

1 . A computer-implemented method for utilizing model predictive control for optimal interactions comprising:

receiving sensor data from a plurality of vehicle subsystems, including at least one of a LiDAR sensor and a camera;

processing the sensor data to generate environment data associated with a surrounding environment of an ego agent;

receiving dynamic data associated with an operation of the ego agent;

inputting the environment data and the dynamic data to variational autoencoders;

utilizing the model predictive control through functional approximation with the variational autoencoders that output encoded data based on analysis of the environment data and the dynamic data and decoders that decode the encoded data to output parameterized probabilistic action estimates;

outputting an estimated optimal control trajectory based on analysis of the parameterized probabilistic action estimates to control at least one system of the ego agent to operate within the surrounding environment of the ego agent;

generating actuator commands for steering, acceleration, and braking of the ego agent based on the estimated optimal control trajectory;

executing the actuator commands in real-time on the ego agent; and

iteratively updating the actuator commands based on the environment data.

2 . The computer-implemented method of claim 1 , wherein the variational autoencoders are configured as a multi-pronged ensemble of variational autoencoders that are configured to model a single optimization process of the model predictive control.

3 . The computer-implemented method of claim 1 , further including conditioning the variational autoencoders on a cost of moving from one state to another as well as costs related to where each state action pair produces optimal future interactions.

4 . The computer-implemented method of claim 1 , wherein the variational autoencoders are configured to reduce an environment state dimension associated with global constraints of the surrounding environment of the ego agent to a latent space.

5 . The computer-implemented method of claim 4 , wherein the variational autoencoders are configured to reduce the environment state as well as specific optimizations through designed cost functions and input respective encoded data to the decoders.

6 . The computer-implemented method of claim 1 , wherein utilizing the model predictive control includes recombining outputs of the decoders to form the estimated optimal control trajectory.

7 . The computer-implemented method of claim 6 , wherein recombining the outputs of the decoder includes recombining data pertaining to the parameterized probabilistic action estimates output by the decoders with Gaussian assumptions to form the estimated optimal control trajectory.

8 . The computer-implemented method of claim 1 , wherein the model predictive control is utilized to exploit suboptimal interactions to search a space of surrounding environment of the ego agent for optimality, to learn system dynamics and constraints based on the dynamic data and the environment data, and to learn a relative importance of optimality to one or more interacting agents.

9 . The computer-implemented method of claim 1 , wherein controlling the at least one system of the ego agent includes analyzing data pertaining to the estimated optimal control trajectory and communicating at least one command to an autonomous controller of the ego agent to provide autonomous control of the ego agent to enable optimal interactions based on future state predictions.

10 . A system for utilizing model predictive control for optimal interactions comprising:

a memory storing instructions when executed by a processor cause the processor to:

receive sensor data from a plurality of vehicle subsystems, including at least one of a LIDAR sensor and a camera;

process the sensor data to generate environment data associated with a surrounding environment of an ego agent;

receive dynamic data associated with an operation of the ego agent;

input the environment data and the dynamic data to a multi-pronged ensemble of variational autoencoders, wherein each variational autoencoder is conditioned on different cost functions related to state transitions and future interaction optimization;

utilize the model predictive control through functional approximation with the variational autoencoders that output encoded data based on analysis of the environment data and the dynamic data and decoders that decode the encoded data to output parameterized probabilistic action estimates;

recombine outputs of the decoders using Gaussian assumptions to generate a control trajectory prediction based on the parameterized probabilistic action estimate;

output an estimated optimal control trajectory based on analysis of the parameterized probabilistic action estimates to control at least one system of the ego agent to operate within the surrounding environment of the ego agent;

generate actuator commands for steering, acceleration, and braking of the ego agent based on the estimated optimal control trajectory;

execute the actuator commands in real-time on the ego agent; and

iteratively update the actuator commands based on the environment data.

11 . The system of claim 10 , wherein the multi-pronged ensemble of variational autoencoders that are configured to model a single optimization process of the model predictive control.

12 . The system of claim 10 , wherein the variational autoencoders are conditioned on a cost of moving from one state to another as well as costs related to where each state action pair produces optimal future interactions.

13 . The system of claim 10 , wherein the variational autoencoders are configured to reduce an environment state dimension associated with global constraints of the surrounding environment of the ego agent to a latent space.

14 . The system of claim 13 , wherein the variational autoencoders are configured to reduce the environment state as well as specific optimizations through designed cost functions and input respective encoded data to the decoders.

15 . The system of claim 10 , wherein utilizing the model predictive control includes recombining outputs of the decoders to form the estimated optimal control trajectory.

16 . The system of claim 15 , wherein recombining the outputs of the decoder includes recombining data pertaining to the parameterized probabilistic action estimates output by the decoders with Gaussian assumptions to form the estimated optimal control trajectory.

17 . The system of claim 10 , wherein the model predictive control is utilized to exploit suboptimal interactions to search a space of surrounding environment of the ego agent for optimality, to learn system dynamics and constraints based on the dynamic data and environment data, and to learn a relative importance of optimality to one or more interacting agents.

18 . The system of claim 10 , wherein controlling the at least one system of the ego agent includes analyzing data pertaining to the estimated optimal control trajectory and communicating at least one command to an autonomous controller of the ego agent to provide autonomous control of the ego agent to enable optimal interactions based on future state predictions.

19 . A non-transitory computer readable storage medium storing instruction that when executed by a computer, which includes a processor perform a method, the method comprising:

receiving sensor data from a plurality of vehicle subsystems, including at least one of a LiDAR sensor and a camera;

processing the sensor data to generate environment data associated with a surrounding environment of an ego agent;

receiving dynamic data associated with an operation of the ego agent;

inputting the environment data and the dynamic data to variational autoencoders;

utilizing model predictive control through functional approximation with the variational autoencoders that output encoded data based on analysis of the environment data and the dynamic data and decoders that decode the encoded data to output parameterized probabilistic action estimates;

outputting an estimated optimal control trajectory based on analysis of the parameterized probabilistic action estimates to control at least one system of the ego agent to operate within the surrounding environment of the ego agent;

generating actuator commands for steering, acceleration, and braking of the ego agent based on the estimated optimal control trajectory;

executing the actuator commands in real-time on the ego agent; and

iteratively updating the actuator commands based on the environment data.

20 . The non-transitory computer readable storage medium of claim 19 , wherein the model predictive control is utilized to exploit suboptimal interactions to search a space of surrounding environment of the ego agent for optimality, to learn system dynamics and constraints based on the dynamic data and the environment data, and to learn a relative importance of optimality to one or more interacting agents.