IP Library Granted Patent US 12,394,311
Granted Patent B2
US 12,394,311 · App. 18/388,107 · Granted Aug 19, 2025

Systems and methods for implementing multimodal safety operations with an autonomous agent

Inventors: Steve Vozar (Ann Arbor, MI); Edwin Olson (Ann Arbor, MI); Sean M. Messenger (Ann Arbor, MI); Collin Johnson (Ann Arbor, MI)
Assignee: May Mobility, Inc.
G08G1/096725G08G1/0116
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,394,311
App. No.
18/388,107
Granted
Aug 19, 2025
Kind
B2
Abstract

A system and method includes an autonomous agent having a communication interface that enables the autonomous agent to communicate with a plurality of infrastructure sensing devices; a plurality of distinct health monitors that monitor distinct operational aspects of the autonomous agent; an autonomous state machine that computes a plurality of allowed operating states of the autonomous agent based on inputs from the plurality of distinct health monitors; a plurality of distinct autonomous controllers that generate a plurality of distinct autonomous control instructions; and an arbiter of autonomous control instructions that: collects, as a first input, the plurality of autonomous control instructions generated by each of the plurality of distinct autonomous controllers; collects, as a second input, data relating to the plurality of allowed operating state of the autonomous agent; and selectively enables only a subset of the autonomous control instructions to pass to driving components of the autonomous agent.

Claims (36)

1. A method for operating an autonomous agent, the method comprising:

during operation of the autonomous agent in a first operation mode, collecting a set of data from a set of sources;

determining an operating state of the autonomous agent based on the set of data, wherein the operating state is indicative of a set of capabilities associated with the autonomous agent;

mapping the operating state to a second operation mode;

switching a control of the autonomous agent from the first operation mode to the second operation mode; and

simulating a set of trajectories associated with the second operation mode.

2. The method of claim 1 , wherein mapping the operating state to the second operation mode comprises determining a highest level of autonomous operation satisfied by the set of data.

3. The method of claim 1 , wherein the first operation mode and the second operation mode each relate to distinct levels of autonomous operation within an autonomous operation cascade comprising a plurality of distinct levels of autonomous operation.

4. The method of claim 1 , wherein determining the operating state comprises determining a set of capabilities of a plurality of devices associated with the autonomous agent.

5. The method of claim 4 , wherein the plurality of devices associated with the autonomous agent comprise an onboard sensor system.

6. The method of claim 1 , wherein the set of sources comprises one or more offboard data sources.

7. The method of claim 1 , further comprising controlling the autonomous agent based on a simulated trajectory of the set of trajectories having a highest probability of occurring.

8. The method of claim 1 , wherein determining the operating state of the autonomous agent based on the set of data comprises determining a set of health values of one or more features of the autonomous agent.

9. The method of claim 1 , wherein switching between modes comprises selectively activating a subset of a plurality of instruction filters.

10. A system for operating an autonomous agent, the system comprising:

a set of health monitors in communication with a set of sensors, wherein each health monitor of the set of health monitors computes a set of outputs;

a state machine that determines an autonomy level, wherein determining the autonomy level comprises mapping the sets of outputs to a set of expected health statistics; and

an arbiter that selectively permits a subset of autonomous control instructions received from a plurality of distinct planning modules to reach the autonomous agent based on the autonomy level.

11. The system of claim 10 , wherein the set of outputs comprises a sensor health of a sensor of the autonomous agent.

12. The system of claim 10 , wherein the set of expected health statistics comprises a set of minimum capabilities associated with each of a set of distinct autonomy levels.

13. The system of claim 10 , wherein the arbiter comprises selection circuitry comprising a switch that operably switches between autonomous control instructions based on the autonomy level.

14. The system of claim 10 , wherein the plurality of distinct planning modules comprise a multi-policy decision-making module configured to:

generate a plurality of behavioral policies; and

select a subset of the behavioral policies to execute as control instructions.

15. The system of claim 10 , wherein the arbiter is further configured to switch an operation mode of the autonomous agent based on the autonomy level determined by the state machine.

16. The system of claim 10 , wherein selectively permitting the subset of autonomous control instructions to reach the autonomous agent comprises mapping the control instructions to the autonomy level.

17. The system of claim 10 , wherein the autonomous agent operates within a defined zone of operation comprising a structured route of the autonomous agent.

18. The system of claim 10 , wherein the plurality of distinct planning modules:

generate sets of autonomous control instructions based on data streams received from the set of sensors; and

selectively accept and reject data streams from each of the set of sensors based on the set of outputs computed by the health monitors.

19. The system of claim 10 , wherein mapping the set of outputs to the set of expected health statistics comprises determining a highest level of autonomous operation satisfied by the set of outputs.

20. A method for operating an autonomous agent, the method comprising:

during operation of the autonomous agent in a first operation mode, collecting a set of data from a set of sources;

determining an operating state of the autonomous agent based on the set of data, wherein the operating state is indicative of a set of capabilities associated with the autonomous agent;

mapping the operating state to a second operation mode;

switching a control of the autonomous agent from the first operation mode to the second operation mode, wherein switching between modes comprises selectively activating a subset of a plurality of instruction filters.

Assignments (2)
SECURITY INTEREST Recorded May 15, 2026
From: MAY MOBILITY, INC.
To: ACP REDSTONE CREDIT, LLC
Reel/Frame 075610/0696 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2023
From: VOZAR, STEVE; OLSON, EDWIN; MESSENGER, SEAN M.; JOHNSON, COLLIN
To: MAY MOBILITY, INC.
Reel/Frame 065501/0502 →
Continuity (5)
Continuation 17014578 · Sep 8, 2020
Continuation 16776993 · Jan 30, 2020
Continuation 16505372 · Jul 8, 2019
Provisional Application 62702715 · Jul 24, 2018
Related Publication 20240071221A1 · Feb 29, 2024
References Cited (188)
US 5544282A · Chen et al. · 1996 [cited by applicant]
US 6199013B1 · Oshea · 2001 [cited by applicant]
US 9129519B2 · Aoude et al. · 2015 [cited by applicant]
US 9274525B1 · Ferguson et al. · 2016 [cited by applicant]
US 9368026B1 · Herbach et al. · 2016 [cited by applicant]
US 9495874B1 · Zhu et al. · 2016 [cited by applicant]
US 9618938B2 · Olson et al. · 2017 [cited by applicant]
US 9646428B1 · Konrardy et al. · 2017 [cited by applicant]
US 9720412B1 · Zhu et al. · 2017 [cited by applicant]
US 9811760B2 · Richardson et al. · 2017 [cited by applicant]
US 9914452B1 · Ferguson et al. · 2018 [cited by applicant]
US 10012981B2 · Gariepy et al. · 2018 [cited by applicant]
US 10062294B2 · Kunzi et al. · 2018 [cited by applicant]
US 10156848B1 · Konrardy et al. · 2018 [cited by applicant]
US 10235882B1 · Aoude et al. · 2019 [cited by applicant]
US 10248120B1 · Siegel et al. · 2019 [cited by applicant]
US 10386856B2 · Wood et al. · 2019 [cited by applicant]
US 10467891B1 · Bart et al. · 2019 [cited by applicant]
US 10518770B2 · Kroop et al. · 2019 [cited by applicant]
US 10518783B2 · Tanimichi et al. · 2019 [cited by applicant]
US 10540892B1 · Fields et al. · 2020 [cited by applicant]
US 10558224B1 · Lin et al. · 2020 [cited by applicant]
US 10564641B2 · Vozar et al. · 2020 [cited by applicant]
US 10564643B2 · Lui et al. · 2020 [cited by applicant]
US 10571916B2 · Tschanz et al. · 2020 [cited by applicant]
US 10586254B2 · Singhal · 2020 [cited by applicant]
US 10599155B1 · Konrardy et al. · 2020 [cited by applicant]
US 10614709B2 · Vozar et al. · 2020 [cited by applicant]
US 10642276B2 · Huai · 2020 [cited by applicant]
US 10654476B2 · Wray et al. · 2020 [cited by applicant]
US 10671076B1 · Kobilarov et al. · 2020 [cited by applicant]
US 10796581B2 · Herbach et al. · 2020 [cited by applicant]
US 10860019B2 · Censi et al. · 2020 [cited by applicant]
US 10969470B2 · Voorheis et al. · 2021 [cited by applicant]
US 11086318B1 · Davis et al. · 2021 [cited by applicant]
US 11087200B2 · Olson et al. · 2021 [cited by applicant]
US 11242054B2 · Isele · 2022 [cited by applicant]
US 11260855B2 · Zhang · 2022 [cited by applicant]
US 11281213B2 · Cross et al. · 2022 [cited by applicant]
US 11300957B2 · Wray et al. · 2022 [cited by applicant]
US 11396302B2 · Ye et al. · 2022 [cited by applicant]
US 11525887B2 · Voorheis et al. · 2022 [cited by applicant]
US 20020062207A1 · Faghri · 2002 [cited by applicant]
US 20040100563A1 · Sablak et al. · 2004 [cited by applicant]
US 20050004723A1 · Duggan et al. · 2005 [cited by applicant]
US 20060184275A1 · Hosokawa et al. · 2006 [cited by applicant]
US 20060200333A1 · Dalal et al. · 2006 [cited by applicant]
US 20070193798A1 · Allard et al. · 2007 [cited by applicant]
US 20070276600A1 · King et al. · 2007 [cited by applicant]
US 20080033684A1 · Vian et al. · 2008 [cited by applicant]
US 20100114554A1 · Misra · 2010 [cited by applicant]
US 20110142283A1 · Huang et al. · 2011 [cited by applicant]
US 20120089275A1 · Yao-Chang et al. · 2012 [cited by applicant]
US 20130054106A1 · Schmuedderich et al. · 2013 [cited by applicant]
US 20130141576A1 · Lord et al. · 2013 [cited by applicant]
US 20130253816A1 · Caminiti et al. · 2013 [cited by applicant]
US 20140195138A1 · Stelzig et al. · 2014 [cited by applicant]
US 20140244198A1 · Mayer · 2014 [cited by applicant]
US 20140309815A1 · Ricci et al. · 2014 [cited by applicant]
US 20150105961A1 · Callow · 2015 [cited by applicant]
US 20150284010A1 · Beardsley et al. · 2015 [cited by applicant]
US 20150302756A1 · Guehring et al. · 2015 [cited by applicant]
US 20150316928A1 · Guehring et al. · 2015 [cited by applicant]
US 20150321337A1 · Stephens · 2015 [cited by applicant]
US 20150344030A1 · Damerow et al. · 2015 [cited by applicant]
US 20160005333A1 · Naouri · 2016 [cited by applicant]
US 20160209840A1 · Kim · 2016 [cited by applicant]
US 20160314224A1 · Wei et al. · 2016 [cited by applicant]
US 20170031361A1 · Olson et al. · 2017 [cited by applicant]
US 20170032671A1 · Toyama et al. · 2017 [cited by applicant]
US 20170072853A1 · Matsuoka et al. · 2017 [cited by applicant]
US 20170155696A1 · Dong et al. · 2017 [cited by applicant]
US 20170199523A1 · Barton-Sweeney et al. · 2017 [cited by applicant]
US 20170248952A1 · Perkins et al. · 2017 [cited by applicant]
US 20170268896A1 · Bai et al. · 2017 [cited by applicant]
US 20170289341A1 · Rodriguez et al. · 2017 [cited by applicant]
US 20170291560A1 · Schroeder et al. · 2017 [cited by applicant]
US 20170301111A1 · Zhao et al. · 2017 [cited by applicant]
US 20170320500A1 · Yoo et al. · 2017 [cited by applicant]
US 20170323568A1 · Inoue et al. · 2017 [cited by applicant]
US 20170356748A1 · Iagnemma · 2017 [cited by applicant]
US 20180011485A1 · Ferren · 2018 [cited by applicant]
US 20180018895A1 · Chan et al. · 2018 [cited by applicant]
US 20180046182A1 · Joyce et al. · 2018 [cited by applicant]
US 20180047291A1 · Konishi et al. · 2018 [cited by applicant]
US 20180053102A1 · Martinson et al. · 2018 [cited by applicant]
US 20180065625A1 · Tijerina et al. · 2018 [cited by applicant]
US 20180070056A1 · Deangelis et al. · 2018 [cited by applicant]
US 20180082596A1 · Whitlow · 2018 [cited by applicant]
US 20180089563A1 · Redding et al. · 2018 [cited by applicant]
US 20180100743A1 · Diaz et al. · 2018 [cited by applicant]
US 20180183873A1 · Wang et al. · 2018 [cited by applicant]
US 20180184352A1 · Lopes et al. · 2018 [cited by applicant]
US 20180196427A1 · Majumdar et al. · 2018 [cited by applicant]
US 20180220283A1 · Condeixa et al. · 2018 [cited by applicant]
US 20180224851A1 · Park · 2018 [cited by applicant]
US 20180251126A1 · Linscott et al. · 2018 [cited by applicant]
US 20180267550A1 · Kopetz et al. · 2018 [cited by applicant]
US 20180268281A1 · Olson et al. · 2018 [cited by applicant]
US 20180293537A1 · Kwok · 2018 [cited by applicant]
US 20180299898A1 · Luo et al. · 2018 [cited by applicant]
US 20180330481A1 · Watanabe et al. · 2018 [cited by applicant]
US 20180365908A1 · Liu et al. · 2018 [cited by applicant]
US 20180367997A1 · Shaw et al. · 2018 [cited by applicant]
US 20190027034A1 · Xu et al. · 2019 [cited by applicant]
US 20190039545A1 · Kumar et al. · 2019 [cited by applicant]
US 20190066399A1 · Jiang et al. · 2019 [cited by applicant]
US 20190096244A1 · Guruva Reddiar et al. · 2019 [cited by applicant]
US 20190101914A1 · Coleman et al. · 2019 [cited by applicant]
US 20190101919A1 · Kobilarov et al. · 2019 [cited by applicant]
US 20190106117A1 · Goldberg · 2019 [cited by applicant]
US 20190113919A1 · Englard et al. · 2019 [cited by applicant]
US 20190113929A1 · Mukadam et al. · 2019 [cited by applicant]
US 20190130878A1 · Bradley · 2019 [cited by applicant]
US 20190138524A1 · Singh et al. · 2019 [cited by applicant]
US 20190147610A1 · Frossard et al. · 2019 [cited by applicant]
US 20190180529A1 · Smith · 2019 [cited by applicant]
US 20190196465A1 · Hummelshøj · 2019 [cited by applicant]
US 20190220011A1 · Della Penna · 2019 [cited by applicant]
US 20190227553A1 · Kentley-Klay et al. · 2019 [cited by applicant]
US 20190235516A1 · Zhang et al. · 2019 [cited by applicant]
US 20190236950A1 · Li et al. · 2019 [cited by applicant]
US 20190258246A1 · Liu et al. · 2019 [cited by applicant]
US 20190258251A1 · Ditty et al. · 2019 [cited by applicant]
US 20190265059A1 · Warnick et al. · 2019 [cited by applicant]
US 20190271981A1 · Oba · 2019 [cited by applicant]
US 20190329771A1 · Wray et al. · 2019 [cited by applicant]
US 20190331758A1 · Malkes et al. · 2019 [cited by applicant]
US 20190332106A1 · Belloni Mourao et al. · 2019 [cited by applicant]
US 20190337509A1 · Shalev-Shwartz et al. · 2019 [cited by applicant]
US 20200017114A1 · Santoni et al. · 2020 [cited by applicant]
US 20200020226A1 · Stenneth et al. · 2020 [cited by applicant]
US 20200057441A1 · Wang et al. · 2020 [cited by applicant]
US 20200086837A1 · Le Cornec · 2020 [cited by applicant]
US 20200094850A1 · Chi et al. · 2020 [cited by applicant]
US 20200097003A1 · Wray et al. · 2020 [cited by applicant]
US 20200098269A1 · Wray et al. · 2020 [cited by applicant]
US 20200122830A1 · Anderson et al. · 2020 [cited by applicant]
US 20200124447A1 · Schwindt et al. · 2020 [cited by applicant]
US 20200159227A1 · Cohen et al. · 2020 [cited by applicant]
US 20200189731A1 · Mistry et al. · 2020 [cited by applicant]
US 20200209853A1 · Leach et al. · 2020 [cited by applicant]
US 20200209864A1 · Chen · 2020 [cited by applicant]
US 20200210777A1 · Valois et al. · 2020 [cited by applicant]
US 20200217668A1 · Cionca et al. · 2020 [cited by applicant]
US 20200233060A1 · Lull et al. · 2020 [cited by applicant]
US 20200249684A1 · Onofrio et al. · 2020 [cited by applicant]
US 20200255027A1 · Kulkarni et al. · 2020 [cited by applicant]
US 20200269843A1 · Wissing et al. · 2020 [cited by applicant]
US 20200290619A1 · Mehdi et al. · 2020 [cited by applicant]
US 20200294401A1 · Kerecsen · 2020 [cited by applicant]
US 20200309543A1 · Voznesensky · 2020 [cited by applicant]
US 20200339151A1 · Batts et al. · 2020 [cited by applicant]
US 20200346666A1 · Wray et al. · 2020 [cited by applicant]
US 20200355820A1 · Zeng et al. · 2020 [cited by applicant]
US 20200369294A1 · Jeon et al. · 2020 [cited by applicant]
US 20200388159A1 · Degerman · 2020 [cited by applicant]
US 20200400781A1 · Voorheis et al. · 2020 [cited by applicant]
US 20210042535A1 · Abbott et al. · 2021 [cited by applicant]
US 20210116907A1 · Altman · 2021 [cited by applicant]
US 20210132606A1 · Basich et al. · 2021 [cited by applicant]
US 20210163021A1 · Frazzoli et al. · 2021 [cited by applicant]
US 20210181758A1 · Das et al. · 2021 [cited by applicant]
US 20210197864A1 · Oltmann et al. · 2021 [cited by applicant]
US 20210208244A1 · Voorheis et al. · 2021 [cited by applicant]
US 20210229697A1 · Lee et al. · 2021 [cited by applicant]
US 20210245785A1 · Suzuki et al. · 2021 [cited by applicant]
US 20210269063A1 · Lee et al. · 2021 [cited by applicant]
US 20210286651A1 · Ho et al. · 2021 [cited by applicant]
US 20210339741A1 · Rezvan Behbahani et al. · 2021 [cited by applicant]
US 20210365701A1 · Eshet et al. · 2021 [cited by applicant]
US 20220076032A1 · Jain et al. · 2022 [cited by applicant]
US 20220081005A1 · Brown et al. · 2022 [cited by applicant]
US 20220126878A1 · Moustafa et al. · 2022 [cited by applicant]
US 20220185325A1 · Chen et al. · 2022 [cited by applicant]
US 20220204010A1 · Zhu et al. · 2022 [cited by applicant]
JP 2011100492A · 2011 [cited by applicant]
JP 2015083417A · 2015 [cited by applicant]
JP 2016091039A · 2016 [cited by applicant]
JP 2016184276A · 2016 [cited by applicant]
WO 2015160900A1 · 2015 [cited by applicant]
Cunningham, A. , et al., “MPDM: Multipolicy Decision-Making in Dynamic, Uncertain Environments for Autonomous Driving”, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2015). [cited by applicant]
Mehta, D. , et al., “Autonomous Navigation in Dynamic Social Environments Using Multi-Policy Decision Making”, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016). [cited by applicant]
Mehta, D. , et al., “Fast Discovery of Influential Outcomes for Risk-Aware MPDM”, Proceedings go the IEEE International Conference on Robotics and Automation (ICRA) (2017). [cited by applicant]
Neumeier Stefan, et al., “Towards a Driver Support System for Teleoperated Driving”, 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (Year: 2019). [cited by applicant]
Paden, B. , et al., “A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles”, IEEE Transactions on Intelligent Vehicles, vol. 1, Ids. 1, (Jun. 13, 2016). [cited by applicant]
Straub, J. , “Comparing the Effect of Pruning on a Best Path and a Naive-approach Blackboard Solver”, International Journal of Automation and Computing (Oct. 2015). [cited by applicant]
Wuthishuwong, Chairit , et al., “Vehicle to Infrastructure based Safe Trajectory Planning for Autonomous Intersection Management”, 2013 13th International Conference on ITS Telecommunications (ITST) (Year: 2013). [cited by applicant]
Cited By (1)
US 12,555,043