IP Library Granted Patent US 12,099,140
Granted Patent B2
US 12,099,140 · App. 17/982,858 · Granted Sep 24, 2024

Systems and methods for intelligently calibrating infrastructure devices using onboard sensors of an autonomous agent

Inventors: Tom Voorheis (Ann Arbor, MI); Rob Goeddel (Ann Arbor, MI); Steve Vozar (Ann Arbor, MI); Edwin Olson (Ann Arbor, MI)
Assignee: May Mobility, Inc.
G01S7/40B60R11/04G01S7/497G01S13/865G01S13/867G01S13/931G01S17/931G02B7/40G05D1/0088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,099,140
App. No.
17/982,858
Granted
Sep 24, 2024
Kind
B2
Abstract

A system for intelligently implementing an autonomous agent that includes an autonomous agent, a plurality of infrastructure devices, and a communication interface. A method for intelligently calibrating infrastructure (sensing) devices using onboard sensors of an autonomous agent includes identifying a state of calibration of an infrastructure device, collecting observation data from one or more data sources, identifying or selecting mutually optimal observation data, specifically localizing a subject autonomous agent based on granular mutually optimal observation data, identifying dissonance in observation data from a perspective of a subject infrastructure device, and recalibrating a subject infrastructure device.

Claims (39)

1. A method comprising:

receiving a first set of observation data, collected with a sensor onboard an autonomous agent, and a second set of observation data, collected with a secondary device within an environment of the autonomous agent, each of the first and second sets of observation data comprising observations of the environment of the autonomous agent;

determining an association between the first and second sets of observation data;

localizing the autonomous agent, in near real-time during a traversal of a route by the autonomous agent, based on the first and second sets of observation data;

determining a calibration error based on the association and the localization of the autonomous agent;

updating a state of calibration based on the calibration error; and

based upon the updated state of calibration, controlling the autonomous agent.

2. The method of claim 1 , wherein the association comprises a spatiotemporal association.

3. The method of claim 1 , further comprising: determining a source of the calibration error based on a third set of observation data, wherein the state of calibration is updated based on the source of the calibration error.

4. The method of claim 1 , further comprising: based on the association, determining mutually optimal observation data for the first and second sets of observation data, wherein the calibration error is determined with the mutually optimal observation data.

5. The method of claim 4 , wherein determining the mutually optimal observation data further comprises removing noise from the mutually optimal observation data.

6. The method of claim 1 , wherein the association comprises at least one granular instance of coincident mutual observability between the first and second sets of observation data.

7. The method of claim 6 , wherein the autonomous agent is localized with joint observation data, from both the first and second sets of observation data, which corresponds to the at least one granular instance of coincident mutual observability.

8. The method of claim 1 , further comprising: determining a dissonance between the first and second data sets based on the association, wherein the calibration error is determined using the dissonance.

9. The method of claim 1 , wherein updating the state of calibration comprises updating the state of calibration of the sensor.

10. The method of claim 1 , wherein updating the state of calibration comprises updating the state of calibration of the secondary device.

11. The method of claim 1 , wherein the secondary device is remote from the autonomous agent.

12. The method of claim 11 , wherein the secondary device is fixed infrastructure within the environment of the autonomous agent.

13. The method of claim 1 , wherein the secondary device comprises a field of sensing, the second set of observation data associated with the field of sensing, wherein the autonomous agent is arranged within the field of sensing.

14. The method of claim 1 , further comprising: based on the updated state of calibration, triggering a recalibration routine to update a set of calibration parameters for the sensor or the secondary device.

15. A sensor device comprising a state of calibration and configured to output the state of calibration, the state of calibration determined by:

determining a first set of observation data with the sensor device;

determining a second set of observation data with a sensor onboard a vehicle;

determining an association between the first and second sets of observation data, wherein the first and second sets of observation data each comprise observations of an environment of the vehicle;

localizing the vehicle, in near real-time during a traversal of a route by the vehicle, based on the first and second sets of observation data;

determining a calibration error based on the association and the localization of the vehicle;

determining the state of calibration based on the calibration error; and

based upon the updated state of calibration, performing at least one of: modifying a behavior of the vehicle, or controlling the vehicle.

16. The sensor device of claim 15 , wherein the association comprises a spatiotemporal association.

17. The sensor device of claim 15 , wherein the state of calibration is further determined by: determining a source of the calibration error based on a third set of observation data, wherein the state of calibration is updated based on the source of the calibration error.

18. The sensor device of claim 15 , wherein the state of calibration is further determined by: based on the association, determining mutually optimal observation data for the first and second sets of observation data, wherein the calibration error is determined with the mutually optimal observation data.

19. A non-transitory computer readable medium comprising software instructions that, when executed by a computing system, cause the computing system to perform:

determining a first set of observation data, collected with a sensor onboard an autonomous agent, and a second set of observation data, collected with a secondary device within an environment of the autonomous agent, each of the first and second sets of observation data comprising observations of the environment of the autonomous agent;

determining an association between the first and second sets of observation data;

localizing the autonomous agent in near real-time during a traversal of a route by the autonomous agent based on the first and second sets of observation data;

determining a calibration error based on the association and the localization of the autonomous agent;

updating a state of calibration based on the calibration error; and

based upon the updated state of calibration, controlling the autonomous agent.

20. The non-transitory computer readable medium of claim 19 , wherein the computing system is onboard the autonomous agent.

Assignments (2)
SECURITY INTEREST Recorded May 15, 2026
From: MAY MOBILITY, INC.
To: ACP REDSTONE CREDIT, LLC
Reel/Frame 075610/0696 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2022
From: VOORHEIS, TOM; GOEDDEL, ROB; VOZAR, STEVE; OLSON, EDWIN
To: MAY MOBILITY, INC.
Reel/Frame 061692/0941 →
Continuity (4)
Continuation 17011037 · Sep 3, 2020
Continuation 16792780 · Feb 17, 2020
Provisional Application 62806564 · Feb 15, 2019
Related Publication 20230059510A1 · Feb 23, 2023