Granted Patent B2

US 12,415,544 · App. 18/143,475 · Granted Sep 16, 2025

Inventors: Xianghong Liu (Ann Arbor, MI); Shuo Feng (Ann Arbor, MI); Haowei Sun (Ann Arbor, MI); Xintao Yan (Ann Arbor, MI); Haojie Zhu (Ann Arbor, MI); Zhengxia Zou (Ann Arbor, MI); Shengyin Shen (Ann Arbor, MI)

Assignee: The Regents of the University of Michigan

B60W60/0015G06N3/08

Abstract

Claims (29)

1. A method of safety testing a host autonomous vehicle (AV), comprising the steps of:

generating a trained machine learning (ML) agent by: (i) obtaining a testing state model having non-safety-critical states and safety-critical states, (ii) editing the testing state model to obtain an edited testing state model that omits data concerning the non-safety-critical states, and (iii) training a ML agent using the edited state testing model so as to generate the trained ML agent; and

testing the host AV in an environment that includes one or more background vehicles configured to operate according to the trained ML agent.

2. The method of claim 1 , further comprising a step of configuring the one or more background vehicles to operate according to the trained ML agent, wherein the configuring step includes storing data representing the trained ML agent in computer-readable memory.

3. The method of claim 1 , wherein the trained ML agent is a deep reinforcement learning (DRL) agent, and wherein the trained ML agent employs a neural network.

4. The method of claim 3 , wherein the trained ML agent is a dense DRL (D2RL) agent.

5. The method of claim 4 , wherein the testing state model is or is based on a Markov decision process (MDP).

6. The method of claim 5 , wherein a D2RL approach is used to densify safety-critical data used to train the ML agent.

7. The method of claim 5 , wherein the editing sub-step includes removing the non-safety-critical states and reconnecting the safety-critical states.

8. The method of claim 1 , wherein the environment in which the host AV is tested is a real environment having one or more roadways on which the host AV travels during testing, and wherein the one or more background vehicles are virtual or simulated vehicles.

9. The method of claim 8 , wherein the testing step includes carrying out a simulation that is synchronized with the host AV and the environment in which the host AV is tested, and wherein the simulation includes at least one of the one or more background vehicles as a virtual background vehicle.

10. The method of claim 1 , wherein the method is carried out by an augmented reality (AR) autonomous vehicle (AV) testing system.

11. An autonomous vehicle (AV) testing system, comprising:

at least one electronic processor and memory accessible by the at least one electronic processor, wherein the memory stores computer instructions;

wherein the AV testing system is configured so that, when the at least one electronic processor executes the computer instructions, the AV testing system:

generates a trained machine learning (ML) agent by: (i) obtaining a testing state model having non-safety-critical states and safety-critical states, (ii) editing the testing state model to obtain an edited testing state model that omits data concerning the non-safety-critical states, and (iii) training a ML agent using the edited state testing model so as to generate the trained ML agent; and

tests the host AV in an environment that includes one or more background vehicles configured to operate according to the trained ML agent.

12. The AV testing system of claim 11 , wherein the AV testing system is configured so that, when the at least one electronic processor executes the computer instructions, the AV testing system: configures the one or more background vehicles to operate according to the trained ML agent, wherein the configuring step includes storing data representing the trained ML agent in computer-readable memory.

13. The AV testing system of claim 11 , wherein the trained ML agent is a deep reinforcement learning (DRL) agent, and wherein the trained ML agent employs a neural network.

14. The AV testing system of claim 13 , wherein the trained ML agent is a dense DRL (D2RL) agent.

15. The AV testing system of claim 14 , wherein the testing state model is or is based on a Markov decision process (MDP).

16. The AV testing system of claim 15 , wherein a D2RL approach is used to densify safety-critical data used to train the ML agent.

17. The AV testing system of claim 15 , wherein the editing sub-step includes removing the non-safety-critical states and reconnecting the safety-critical states.

18. The AV testing system of claim 11 , wherein the environment in which the host AV is tested is a real environment having one or more roadways on which the host AV travels during testing, and wherein the one or more background vehicles are virtual or simulated vehicles.

19. The AV testing system of claim 18 , wherein the testing step includes carrying out a simulation that is synchronized with the host AV and the environment in which the host AV is tested, and wherein the simulation includes at least one of the one or more background vehicles as a virtual background vehicle.

20. A method of safety testing a host autonomous vehicle (AV), comprising the steps of:

generating a trained dense deep reinforcement learning (D2RL) agent by (i) obtaining a testing state model having non-safety-critical states and safety-critical states, (ii) editing the testing state model to omit at least one non-safety-critical state and reconnect at least two safety-critical states, and (iii) training a D2RL agent using the edited state testing model so as to generate the trained D2RL agent;

configuring one or more background vehicles to operate according to the trained D2RL agent; and

after configuring one or more background vehicles to operate according to the trained D2RL agent, testing the AV in an environment that includes the one or more background vehicles.

Assignments (1)

Continuity (2)

References Cited (65)

Cited By (1)

US 12,679,342