IP Library Granted Patent US 11,163,045
Granted Patent B2
US 11,163,045 · App. 16/920,983 · Granted Nov 2, 2021

Automated detection of sensor miscalibration

Inventor: David Pfeiffer (Foster City, CA)
Assignee: Zoox, Inc.
G01S7/4972G01S17/08G01S17/42G01S17/87G01S17/931G05D1/0088G05D1/024G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,163,045
App. No.
16/920,983
Granted
Nov 2, 2021
Kind
B2
Abstract

A vehicle control system includes various sensors. The system can include, among others, LIDAR, RADAR, SONAR, cameras, microphones, GPS, and infrared systems for monitoring and detecting environmental conditions. In some implementations, one or more of these sensors may become miscalibrated. Using data collected by the sensors, the system can detect a miscalibrated sensor and generate an indication that one or more sensors have become miscalibrated. For example, data captured by a sensor can be processed to determine an average height represented by the sensor data and compared to an average height of data captured by other sensors. Based on a difference in heights, an indication can be generated identifying a miscalibrated sensor.

Claims (89)

1. A method comprising:

receiving sensor data of an environment from a plurality of sensors;

determining, as region data, a subset of data of the sensor data associated with a region of the environment proximate a vehicle;

determining a first value based at least in part on a first subset of the region data, the first subset associated with a first sensor of the plurality of sensors;

determining a second value based at least in part on a second subset of the region data, the second subset associated with a second sensor of the plurality of sensors;

determining a difference between the first value and the second value;

determining that the difference meets or exceeds a threshold based at least in part on a number of tests generated for the first sensor or the second sensor; and

performing an action by the vehicle based at least in part on the difference meeting or exceeding the threshold.

2. The method of claim 1 , further comprising:

generating, based at least in part on the difference meeting or exceeding the threshold, an indication that the first sensor is miscalibrated;

generating, based at least in part on the indication, a trajectory; and

controlling, based at least in part on the trajectory, the vehicle.

3. The method of claim 1 , further comprising:

receiving segmentation information associated with the sensor data; and

determining, based at least in part on the segmentation information, that the region is associated with a ground surface of the environment.

4. The method of claim 1 , further comprising determining that the first subset of the region data meets or exceeds a threshold number of data values.

5. The method of claim 1 , further comprising:

determining a third value based at least in part on a third subset of the region data;

determining a fourth value based at least in part on a fourth subset of the region data;

determining a second difference between the third value and the fourth value;

determining that the second difference meets or exceeds the threshold; and

generating an indication that the second sensor is miscalibrated.

6. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein executing the instructions causes the system to:

receive sensor data of an environment from a plurality of sensors disposed about a vehicle;

determine, as region data, a subset of data of the sensor data associated with a region of the environment proximate the vehicle;

determine a first value of a first subset of the region data, the first subset associated with a first sensor of the plurality of sensors;

determine a second value of a second subset of the region data, the second subset associated with at least a second sensor of the plurality of sensors;

determine a difference between the first value and the second value;

determine that the difference meets or exceeds a threshold based at least in part on a number of tests generated for the first sensor or the second sensor; and

perform an action by the vehicle based at least in part on the difference meeting or exceeding the threshold.

7. The system of claim 6 , wherein:

the sensor data represents data captured by three or more sensors, and

at least one of the first value or the second value comprises an average height dimension value.

8. The system of claim 6 , wherein the instructions further causes the system to:

determine a third value based at least in part on a third subset of the region data, the third subset captured by the second sensor;

determine a fourth value based at least in part on a fourth subset of the region data, the fourth subset captured by the first sensor and a third sensor;

determine a second difference between the third value and the fourth value; and

determine that the second difference meets or exceeds the threshold; and

generate an indication that the second sensor is miscalibrated.

9. The system of claim 6 , wherein the instructions further cause the system to:

determine a highest value for the first subset;

determine a lowest value for the first subset;

determine a value difference between the highest value and the lowest value;

determine that the value difference is less than or equal to a value threshold; and

determine, based at least in part on the value difference being less than or equal to the value threshold, that the region data is associated with a flat surface.

10. The system of claim 6 , wherein the instructions further cause the system to:

receive segmentation information associated with the sensor data; and

determine, based at least in part on the segmentation information, that the region data is associated with to a ground surface.

11. The system of claim 6 , wherein the instructions further cause the system to determine that the first subset of the region data meets or exceeds a threshold number of data values.

12. The system of claim 6 , wherein the vehicle is an autonomous vehicle and the instructions further cause the system to:

determine, based at least in part on the difference meeting or exceeding the threshold, that the first sensor is miscalibrated;

generate, based at least in part on the second sensor and a third sensor, a signal configured to cause the autonomous vehicle to traverse a trajectory; and

transmit the signal to the autonomous vehicle.

13. The system of claim 6 , wherein the instructions further cause the system to:

increment a count of a number of tests attempted;

determine a second sensor average of data associated with the second sensor;

determine a third sensor average of data associated with a third sensor;

determine a sensor average difference between the second sensor average and the third sensor average;

determine the sensor average difference does not exceed a threshold; and

increment a count of a number of tests successfully performed.

14. The system of claim 13 , wherein the instructions further cause the system to:

generate a quality score associated with a sensor of the plurality of sensors,

wherein the quality score is based at least in part on the number of tests successfully performed and the number of tests attempted.

15. The system of claim 6 , wherein the instructions further cause the system to determine that the region data is associated with a flat surface.

16. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:

capturing sensor data of an environment using a plurality of sensors of a vehicle;

determining, as region data, a subset of data of the sensor data associated with a region of the environment proximate the vehicle;

determining a first value of a first subset of the region data, the first subset associated with a first sensor of the plurality of sensors;

determining a second value of a second subset of the region data, the second subset associated with a second sensor of the plurality of sensors;

determining a difference between the first value and the second value;

determining that the difference meets or exceeds a threshold value based at least in part on a number of tests generated for the first sensor or the second sensor; and

performing an action by the vehicle based at least in part on the difference meeting or exceeding the threshold.

17. The one or more non-transitory computer-readable media of claim 16 , wherein the operations further comprise:

generating a test rate, the test rate indicative of a number of times the difference is determined; and

generating a success rate, the success rate indicative of a number of times the difference is less than or equal to the threshold value.

18. The one or more non-transitory computer-readable media of claim 16 , wherein the operations further comprise:

disabling, based at least in part on an indication that the first sensor is miscalibrated, the first sensor of the plurality of sensors,

wherein disabling the first sensor comprises indicating a subset of the plurality of sensors as active sensors.

19. The one or more non-transitory computer-readable media of claim 18 , wherein the vehicle is an autonomous vehicle and the operations further comprise:

generating, based at least in part on sensor data captured from one or more sensors, a signal configured to cause the autonomous vehicle to traverse a trajectory; and

transmitting the signal to the autonomous vehicle.

20. The one or more non-transitory computer-readable media of claim 16 , wherein operations further comprise:

determining a highest value for the first subset;

determining a lowest value for the first subset;

determining a value difference between the highest value and the lowest value;

determining that the value difference is less than or equal to a value threshold; and

determining, based at least in part on the value difference, that the region data is associated with to a flat surface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2021
From: PFEIFFER, DAVID
To: ZOOX, INC.
Reel/Frame 056838/0152 →
Continuity (2)
Continuation 15927315 · Mar 21, 2018
Related Publication 20200400799A1 · Dec 24, 2020
Cited By (2)
US 1,063,662 US 1,095,334