IP Library Granted Patent US 11,544,161
Granted Patent B1
US 11,544,161 · App. 16/428,048 · Granted Jan 3, 2023

Identifying anomalous sensors

Inventors: Pradeep Krishna Yarlagadda (Redmond, WA); Jean-Guillaume Dominique Durand (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G06F11/1608B64C39/024G01D18/00G05D1/0077
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,544,161
App. No.
16/428,048
Granted
Jan 3, 2023
Kind
B1
Abstract

A sensor system may include first and second sensors configured to be coupled to a vehicle and generate respective first and second sensor signals indicative of operation of the vehicle. The sensor system may also include a sensor anomaly detector including an anomalous sensor model configured to receive the first and second sensor signals and determine that one or more of the first sensor or the second sensor is an anomalous sensor generating inaccurate sensor data. The sensor system may also be configured to identify one or more of the first sensor or the second sensor as the anomalous sensor generating inaccurate sensor data.

Claims (77)

1. An unmanned aerial vehicle (UAV) comprising:

a vehicle controller configured to receive sensor signals from a sensor system and control operation of the UAV; and

the sensor system in communication with the vehicle controller, the sensor system comprising:

a first sensor coupled to the UAV and configured to generate first sensor signals indicative of operation of the UAV;

a second sensor coupled to the UAV and configured to generate second sensor signals indicative of the operation of the UAV; and

a sensor anomaly detector in communication with the first sensor and the second sensor, the sensor anomaly detector comprising an analytical sensor model configured to:

receive the first sensor signals and the second sensor signals; and

determine that one or more of the first sensor or the second sensor

is an anomalous sensor generating inaccurate sensor data,

wherein the sensor anomaly detector is configured to:

generate a sensor fault signal indicative of the anomalous sensor; and

identify the anomalous sensor,

wherein the vehicle controller is configured to receive the sensor fault signal and control operation of the UAV based on the sensor fault signal;

wherein the vehicle controller is configured to cause the UAV to perform a maneuver, and wherein the sensor anomaly detector is configured to confirm the anomalous sensor based on at least one of the first sensor signals generated during the maneuver or the second sensor signals generated during the maneuver.

2. The UAV of claim 1 , wherein the analytical sensor model comprises a mathematical model trained via machine learning and training data comprising sensor data generated prior in time during operation of a UAV by a plurality of sensors coupled to the UAV.

3. The UAV of claim 1 , wherein the vehicle controller, upon receipt of the sensor fault signal, is configured to one of prevent take-off of the UAV or cause the UAV to land.

4. The UAV of claim 1 , wherein the sensor system further comprises a third sensor coupled to the UAV and configured to generate third sensor signals indicative of the operation of the UAV, and wherein the sensor anomaly detector is configured identify as the anomalous sensor one of the first sensor, the second sensor, or the third sensor, wherein identifying the anomalous sensor comprises:

monitoring, during a first time period, the first sensor signals and the second sensor signals;

monitoring, during a second time period, the second sensor signals and the third sensor signals;

monitoring, during a third time period, the first sensor signals and the third sensor signals; and

identifying one or more of the first sensor, the second sensor, or the third sensor as the anomalous sensor generating the inaccurate sensor data, based on the monitoring during the first time period, the monitoring during the second time period, and the monitoring during the third time period.

5. The UAV of claim 1 , wherein generating the sensor fault signal is based on a confidence level associated with at least one of the first sensor signals or the second sensor signals meeting or exceeding a threshold confidence level, wherein the threshold confidence level is based on one or more correlations associated with third sensor signals generated by the first sensor or the second sensor during prior operation of the UAV.

6. The UAV of claim 1 , where determining that the one or more of the first sensor or the second sensor is the anomalous sensor generating the inaccurate sensor data comprises:

determining expected sensor signals expected to be generated by the first sensor or the second sensor, the expected sensor signals being based on first previous operation of the UAV or second previous operation of a different UAV;

comparing the expected sensor signals to at least one of the first sensor signals or the second sensor signals; and

determining that a difference between the expected sensor signals and the at least one of the first sensor signals or the second sensor signals meets or exceeds a threshold amount.

7. A sensor system for a vehicle, the sensor system comprising:

a first sensor configured to be coupled to the vehicle and generate one or more first sensor signals indicative of operation of the vehicle;

a second sensor configured to be coupled to the vehicle and generate one or more second sensor signals indicative of the operation of the vehicle; and

a sensor anomaly detector in communication with the first sensor and the second sensor, the sensor anomaly detector comprising an anomalous sensor model configured to:

receive the one or more first sensor signals and the one or more second sensor signals;

determine, based at least in part on at least one of the one or more first sensor signals or the one or more second sensor signals, that one or more of the first sensor or the second sensor is an anomalous sensor generating of inaccurate sensor data, wherein the sensor anomaly detector is configured to generate a sensor fault signal indicative of the anomalous sensor and

cause the vehicle to perform a maneuver and recalibrate the anomalous sensor based at least in part on the maneuver.

8. The sensor system of claim 7 , wherein the anomalous sensor model comprises mathematical model trained via machine learning and training data comprising sensor data generated prior in time during operation of a UAV by a plurality of sensors coupled to the UAV.

9. The sensor system of claim 7 , further comprising a third sensor configured to be coupled to the vehicle and generate one or more third sensor signals indicative of the operation of the vehicle, and wherein the sensor anomaly detector is configured to identify as the anomalous sensor one of the first sensor, the second sensor, or the third sensor, wherein identifying the anomalous sensor comprises:

monitoring, during a first time period, the one or more first sensor signals and the one or more second sensor signals;

monitoring, during a second time period, the one or more second sensor signals and the one or more third sensor signals;

monitoring, during a third time period, the one or more first sensor signals and the one or more third sensor signals; and

identifying one or more of the first sensor, the second sensor, or the third sensor as the anomalous sensor generating the inaccurate sensor data, based at least in part on the monitoring during the first time period, the monitoring during the second time period, and the monitoring during the third time period.

10. The sensor system of claim 9 , wherein one or more of the first time period, the second time period, or the third time period at least partially overlap.

11. The sensor system of claim 9 , wherein identifying the anomalous sensor further comprises:

determining that the one or more first sensor signals are inconsistent with the one or more second sensor signals during the first time period;

determining that the one or more first sensor signals are inconsistent with the one or more third sensor signals during the third time period;

determining that the one or more second sensor signals are consistent with the one or more third sensor signals during the second time period; and

identifying the first sensor as the anomalous sensor generating the inaccurate sensor data.

12. The sensor system of claim 7 , wherein identifying one or more of the first sensor or the second sensor as the anomalous sensor generating the inaccurate sensor data comprises:

monitoring the one or more first sensor signals during a first time period;

monitoring the one or more first sensor signals during a second time period;

determining a first sensor signal difference between the one or more first sensor signals during the first time period and the one or more first sensor signals during the second time period; and

identifying the first sensor as the anomalous sensor based at least in part on the first sensor signal difference being greater than a threshold magnitude.

13. The sensor system of claim 7 , wherein the first sensor comprises one of a temperature sensor, an optical sensor, a global positioning system (GPS), an inertial measurement unit (IMU), an accelerometer, a gyroscope, a laser sensor, a sound navigation and ranging (SONAR) sensor, a radio detection and ranging (RADAR) sensor, or a pressure sensor, and wherein the second sensor comprises one of a temperature sensor, an optical sensor, a GPS, an IMU, an accelerometer, a gyroscope, a laser sensor, a SONAR sensor, a RADAR sensor, or a pressure sensor.

14. The sensor system of claim 7 , wherein the one or more first sensor signals comprise first sensor data indicative of a first second maneuver of the vehicle and the one or more second sensor signals comprise second sensor data indicative of the second maneuver of the vehicle, and wherein the sensor anomaly detector is configured to determine that the first sensor data is inconsistent with the second sensor data.

15. The sensor system of claim 14 , further comprising:

a vehicle controller in communication with the sensor system and configured to control operation of the vehicle, wherein the vehicle controller is further configured to:

receive the sensor fault signal;

wherein the sensor anomaly detector is configured to identify the anomalous sensor based at least in part on the one or more first sensor signals generated during the second maneuver and the one or more second sensor signals generated during the second maneuver.

16. A method comprising:

receiving one or more first sensor signals generated by a first sensor in an analytical sensor model, the one or more first sensor signals indicative of operation of a vehicle;

receiving one or more second sensor signals generated by a second sensor in the analytical sensor model, the one or more second sensor signals indicative of the operation of the vehicle;

determining, via the analytic sensor model and based at least in part on at least one of the one or more first sensor signals or the one or more second sensor signals, that one or more of the first sensor or the second sensor is an anomalous sensor generating inaccurate sensor data; and

causing the vehicle to maneuver and recalibrating the anomalous sensor based at least in part on the maneuver.

17. The method of claim 16 , wherein the first sensor comprises the anomalous sensor and the method further comprises:

recalibrating the first sensor based at least in part on the one or more first sensor signals generated during the maneuver, and the one or more second sensor signals generated during the maneuver.

18. The method of claim 16 , further comprising:

generating a sensor fault signal indicative of the anomalous sensor;

causing, upon generation of the sensor fault signal, the vehicle to perform the maneuver; and

identifying the anomalous sensor based at least in part on the one or more first sensor signals and the one or more second sensor signals generated during the maneuver.

19. The method of claim 16 , wherein the method further comprises:

monitoring the one or more first sensor signals during a first time period;

monitoring the one or more first sensor signals during a second time period;

determining a first sensor signal difference between the one or more first sensor signals during the first time period and the one or more first sensor signals during the second time period; and

identifying the first sensor as the anomalous sensor based at least in part on the first sensor signal difference being greater than a threshold magnitude.

20. The method of claim 16 , further comprising:

monitoring, during a first time period, the one or more first sensor signals and the one or more second sensor signals;

monitoring, during a second time period, the one or more second sensor signals and one or more third sensor signals indicative of the operation of the vehicle;

monitoring, during a third time period, the one or more first sensor signals and the one or more third sensor signals; and

identifying one or more of the first sensor, the second sensor, or the third sensor as the anomalous sensor generating the inaccurate sensor data, based at least in part on the monitoring during the first time period, the monitoring during the second time period, and the monitoring during the third time period.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2019
From: YARLAGADDA, PRADEEP KRISHNA; DURAND, JEAN-GUILLAUME DOMINIQUE
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 049332/0423 →
Cited By (3)
US 12,270,628 US 12,351,337 US 12,467,720