IP Library Granted Patent US 10,989,538
Granted Patent B2
US 10,989,538 · App. 16/140,897 · Granted Apr 27, 2021

IMU data offset compensation for an autonomous vehicle

Inventors: Marc Leach (Pittsburgh, PA); Mark Sibenac (Pittsburgh, PA); Robert Zlot (Pittsburgh, PA); Aaron Greenfield (Pittsburgh, PA)
Assignee: UATC, LLC
G01C21/165G01C21/30G01C25/005G01S17/86G01S17/931G05D1/0212G05D1/0231G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,989,538
App. No.
16/140,897
Granted
Apr 27, 2021
Kind
B2
Abstract

A sensor data processing system of an autonomous vehicle can receive inertial measurement unit (IMU) data from one or more IMUs of the autonomous vehicle. Based at least in part on the IMU data, the system can identify an IMU data offset from a deficient IMU of the one or more IMUs, and generate an offset compensation transform to compensate for the IMU data offset from the deficient IMU. The system may then dynamically execute the offset compensation transform on the IMU data from the deficient IMU to dynamically compensate for the IMU data offset.

Claims (54)

1. A sensor data processing system for an autonomous vehicle, comprising:

one or more processors; and

a memory storing an instruction set that, when executed by the one or more processors, causes the sensor data processing system to:

receive inertial measurement unit (IMU) data from one or more IMUs of the autonomous vehicle;

based at least in part on the IMU data, identify an IMU data offset from a deficient IMU of the one or more IMUs;

determine whether the IMU data offset from the deficient IMU is within a set of plausibility thresholds;

in response to determining that the IMU data offset from the deficient IMU is within the set of plausibility thresholds, generate an offset compensation transform to compensate for the IMU data offset from the deficient IMU; and

dynamically execute the offset compensation transform on the IMU data from the deficient IMU to dynamically compensate for the IMU data offset.

2. The sensor data processing system of claim 1 , wherein the executed instruction set further causes the sensor data processing system to:

discard the IMU data from the deficient IMU in response to determining that the IMU data offset from the deficient IMU is not within the set of plausibility thresholds.

3. The sensor data processing system of claim 1 , wherein the one or more IMUs comprise three or more IMUs, and wherein the executed instruction set causes the sensor data processing system to identify the IMU data offset from the deficient IMU based on comparing the IMU data from at least two other IMUs of the three or more IMUs with the IMU data from the deficient IMU.

4. The sensor data processing system of claim 1 , wherein the executed instruction set causes the sensor data processing system to:

receive sensor data from a perception sensor system of the autonomous vehicle, the sensor data comprising a sensor view of a surrounding environment of the autonomous vehicle;

wherein the executed instruction set causes the sensor data processing system to identify the IMU data offset from the deficient IMU based on comparing the IMU data from the deficient IMU with the sensor data from the perception sensor system.

5. The sensor data processing system of claim 4 , wherein the executed instruction set further causes the sensor data processing system to:

dynamically perform sequential localization operations by dynamically comparing the sensor view with a current localization map of a set of stored localization maps on the autonomous vehicle, the current localization map comprising ground truth data of a particular road segment on which the autonomous vehicle currently operates.

6. The sensor data processing system of claim 5 , wherein the sequential localization operations are dynamically performed at a first rate, wherein the IMU data from the one or more IMUs are received at a second rate that is greater than the first rate, wherein the executed instruction set further causes the sensor data processing system to:

based on the IMU data, determine a current state of the autonomous vehicle at the second rate;

for each localization operation, determine a map-relative state of the autonomous vehicle; and

between each sequential localization operation, correlate the map-relative state from a previous localization operation with the current state to determine whether the map-relative state from the previous localization operation is within a set of tolerance thresholds from the current state of the autonomous vehicle as determined from the IMU data.

7. The sensor data processing system of claim 6 , wherein each of the map-relative state and the current state correspond to at least one of an orientation, a trajectory, a position in a local reference frame, a position in a global reference frame, or a velocity of the autonomous vehicle.

8. The sensor data processing system of claim 6 , wherein the sequential set of localization operations corresponds to a map-relative filter, and wherein determining the current state of the autonomous vehicle based on the IMU data corresponds to a continuous pose filter.

9. The sensor data processing system of claim 4 , wherein the perception sensor system comprises at least one of a camera system or a LIDAR sensor system.

10. The sensor data processing system of claim 1 , wherein the one or more IMUs comprise at least one of an original equipment manufacturer's (OEM) IMU of the autonomous vehicle or an aftermarket IMU.

11. An autonomous vehicle comprising:

acceleration, braking, and steering systems;

a perception sensor system generating sensor data comprising a sensor view of a surrounding environment of the autonomous vehicle;

one or more inertial measurement units (IMUs); and

a control system comprising one or more processors executing an instruction set, causing the control system to:

process the sensor data to perform object detection operations and autonomously operate the acceleration, braking, and steering systems;

receive IMU data from the one or more IMUs;

based at least in part on the IMU data, identify an IMU data offset from a deficient IMU of the one or more IMUs;

determine whether the IMU data offset from the deficient IMU is within a set of plausibility thresholds;

in response to determining that the IMU data offset from the deficient IMU is within the set of plausibility thresholds, generate an offset compensation transform to compensate for the IMU data offset from the deficient IMU; and

dynamically execute the offset compensation transform on the IMU data from the deficient IMU to dynamically compensate for the IMU data offset.

12. The autonomous vehicle of claim 11 , wherein the executed instruction set further causes the control system to:

discard the IMU data from the deficient IMU in response to determining that the IMU data offset from the deficient IMU is not within the set of plausibility thresholds.

13. The autonomous vehicle of claim 11 , wherein the one or more IMUs comprise three or more IMUs, and wherein the executed instruction set causes the control system to identify the IMU data offset from the deficient IMU based on comparing the IMU data from at least two other IMUs of the three or more IMUs with the IMU data from the deficient IMU.

14. The autonomous vehicle of claim 11 , wherein the executed instruction set causes the control system to:

receive the sensor data from the perception sensor system;

wherein the executed instruction set causes the control system to identify the IMU data offset from the deficient IMU based on comparing the IMU data from the deficient IMU with the sensor data from the perception sensor system.

15. The autonomous vehicle of claim 14 , wherein the executed instruction set further causes the control system to:

dynamically perform sequential localization operations by dynamically comparing the sensor view with a current localization map of a set of stored localization maps on the autonomous vehicle, the current localization map comprising ground truth data of a particular road segment on which the autonomous vehicle currently operates.

16. The autonomous vehicle of claim 15 , wherein the sequential localization operations are dynamically performed at a first rate, and wherein the executed instruction set further causes the control system to:

based on the IMU data, determine a current state of the autonomous vehicle at a second rate that is greater than the first rate;

for each of the sequential localization operations, determine a map-relative state of the autonomous vehicle; and

between each sequential localization operation, correlate the map-relative state of a previous localization operation with the current state to determine whether the map-relative state from the previous localization operation is within a set of tolerance thresholds from the current state of the autonomous vehicle as determined from the IMU data.

17. The autonomous vehicle of claim 16 , wherein each of the map-relative state and the current state correspond to at least one of an orientation, a trajectory, a position in a local reference frame, a position in a global reference frame, or a velocity of the autonomous vehicle.

18. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of an autonomous vehicle, cause the one or more processors to:

receive inertial measurement unit (IMU) data from one or more IMUs of the autonomous vehicle;

based at least in part on the IMU data, identify an IMU data offset from a deficient IMU of the one or more IMUs;

determine whether the IMU data offset from the deficient IMU is within a set of plausibility thresholds;

in response to determining that the IMU data offset from the deficient IMU is within the set of plausibility thresholds, generate an offset compensation transform to compensate for the IMU data offset from the deficient IMU; and

dynamically execute the offset compensation transform on the IMU data from the deficient IMU to dynamically compensate for the IMU data offset.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2024
From: UATC, LLC
To: AURORA OPERATIONS, INC.
Reel/Frame 067733/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE CORRECT ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 050912 FRAME: 0757. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 10, 2020
From: UBER TECHNOLOGIES, INC.
To: UATC, LLC
Reel/Frame 052133/0436 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY DATA PREVIOUSLY RECORDED ON REEL 050912 FRAME 0757. ASSIGNOR(S) HEREBY CONFIRMS THE RECEIVING PARTY DATA/ASSIGNEE SHOULD BE UATC, LLC. Recorded Mar 3, 2020
From: UBER TECHNOLOGIES, INC.
To: UATC, LLC
Reel/Frame 052084/0590 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2019
From: UBER TECHNOLOGIES, INC.
To: UTAC, LLC
Reel/Frame 050912/0757 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2019
From: GREENFIELD, AARON; SIBENAC, MARK; LEACH, MARC; ZLOT, ROBERT
To: UBER TECHNOLOGIES, INC.
Reel/Frame 048509/0696 →