IP Library › Granted Patent US 12662137
Granted Patent B2
US 12662137 · App. 18/603,887 · Granted Jun 23, 2026

Shared decision-making with cognitive priors

Inventors: Jean Marcel dos Reis Costa (San Jose, CA); Guy Rosman (Cambridge, MA); Deepak Edakkattil Gopinath (Washington, DC); Emily Sumner (Mountain View, CA); Thomas Balch (Somerville, MA); Jonathan DeCastro (Arlington, MA); Andrew Michael Silva (Cambridge, MA); Laporsha Trinati Dees (Boston, MA)
Assignees: Toyota Research Institute, Inc.; Toyota Jidosha Kabushiki Kaisha
B60W50/0097G06N3/09B60W2050/0028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12662137
App. No.
18/603,887
Granted
Jun 23, 2026
Kind
B2
Abstract

Systems, methods, and other embodiments described herein relate to integrating human decision-making into a model-based system. In one embodiment, a method includes acquiring sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle. The method includes encoding, using a world encoder, the sensor data into a latent representation. The method includes determining human decision-making characteristics according to the latent representation. The method includes generating a control signal for providing shared control of the vehicle according to the human decision-making characteristics and the latent representation.

Claims (41)

1 . A cognitive system, comprising:

one or more processors;

a memory communicably coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:

acquire sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle;

encode, using a world encoder, the sensor data into a latent representation, the latent representation being encoded features of the sensor data within a defined latent space;

determine human decision-making characteristics that define at least an intent of the driver and latencies of the driver in making a decision according to the latent representation; and

generate a control signal that provides shared control of the vehicle according to the human decision-making characteristics and the latent representation, the shared control including blending driver control inputs with automated control inputs.

2 . The cognitive system of claim 1 , wherein the instructions to encode the sensor data into the latent representation include instructions to encode a latent state of the driver to integrate human decision-making within the latent representation.

3 . The cognitive system of claim 1 , wherein the instructions to determine the human decision-making characteristics include instructions to apply a decision model to predict human decision-making and latency in making a decision from the latent representation.

4 . The cognitive system of claim 3 , wherein the decision model implements one of nonlinear opinion dynamics and drift-diffusion modeling to infer what the driver is considering, limitations of the driver in relation to the human decision-making, and to account for latencies in making the decision, and

wherein the instructions to determine the human decision-making characteristics include instructions to infer, using a dynamics model, a subsequent representation of the latent representation at a subsequent time step to further account for future considerations of the driver.

5 . The cognitive system of claim 1 , wherein the instructions to determine the human decision-making characteristics includes applying a decision model to generate outputs indicating the intent of the driver, preferences for decisions, and preferences for a mode of interaction according to the latent representation.

6 . The cognitive system of claim 1 , wherein generating the control signal includes applying a policy model that determines how to blend driver control inputs with automated control inputs for the shared control according to the human decision-making characteristics and the latent representation, and

wherein the policy model is a semi-Markov model (SMM).

7 . The cognitive system of claim 1 , further comprising:

training at least the world encoder and a policy model using a reward network that predicts rewards according to the latent representation, wherein training includes training a decision model using labeled data.

8 . The cognitive system of claim 1 , wherein the cognitive system is integrated within the vehicle that operates at least semi-autonomously.

9 . A non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to:

acquire sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle;

encode, using a world encoder, the sensor data into a latent representation, the latent representation being encoded features of the sensor data within a defined latent space;

determine human decision-making characteristics that define at least an intent of the driver and latencies of the driver in making a decision according to the latent representation; and

generate a control signal for providing shared control of the vehicle according to the human decision-making characteristics and the latent representation, the shared control including blending driver control inputs with automated control inputs.

10 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to encode the sensor data into the latent representation include instructions to encode a latent state of the driver to integrate human decision-making within the latent representation.

11 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to determine the human decision-making characteristics include instructions to apply a decision model to predict human decision-making and latency in making a decision from the latent representation.

12 . The non-transitory computer-readable medium of claim 11 , wherein the decision model implements one of nonlinear opinion dynamics and drift-diffusion modeling to infer what the driver is considering, limitations of the driver in relation to the human decision-making, and to account for latencies in making the decision, and

wherein the instructions to determine the human decision-making characteristics include instructions to infer, using a dynamics model, a subsequent representation of the latent representation at a subsequent time step to further account for future considerations of the driver.

13 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to determine the human decision-making characteristics includes applying a decision model to generate outputs indicating intent of the driver, preferences for decisions, and preferences for a mode of interaction according to the latent representation.

14 . A method, comprising:

acquiring sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle;

encoding, using a world encoder, the sensor data into a latent representation, the latent representation being encoded features of the sensor data within a defined latent space;

determining human decision-making characteristics that define at least an intent of the driver and latencies of the driver in making a decision according to the latent representation; and

generating a control signal that provides shared control of the vehicle according to the human decision-making characteristics and the latent representation, the shared control including blending driver control inputs with automated control inputs.

15 . The method of claim 14 , wherein encoding the sensor data into the latent representation includes encoding a latent state of the driver to integrate human decision-making within the latent representation.

16 . The method of claim 14 , wherein determining the human decision-making characteristics includes applying a decision model to predict human decision-making and latency in making a decision from the latent representation.

17 . The method of claim 16 , wherein the decision model implements one of nonlinear opinion dynamics and drift-diffusion modeling to infer what the driver is considering, limitations of the driver in relation to the human decision-making, and to account for latencies in making the decision, and

wherein determining the human decision-making characteristics includes inferring, using a dynamics model, a subsequent representation of the latent representation at a subsequent time step to further account for future considerations of the driver.

18 . The method of claim 14 , wherein determining the human decision-making characteristics includes applying a decision model to generate outputs indicating the intent of the driver, preferences for decisions, and preferences for a mode of interaction according to the latent representation.

19 . The method of claim 14 , wherein generating the control signal includes applying a policy model that determines how to blend driver control inputs with automated control inputs for the shared control according to the human decision-making characteristics and the latent representation,

wherein the policy model is a semi-Markov model (SMM).

20 . The method of claim 14 , further comprising:

training at least the world encoder and a policy model using a reward network that predicts rewards according to the latent representation, wherein training includes training a decision model using labeled data.