Granted Patent B2

US 12,322,292 · App. 17/138,063 · Granted Jun 3, 2025

Inventors: Keenan A. Wyrobek (Half Moon Bay, CA); Matthew O. Derry (Ann Arbor, MI); Rohit H. Sant (San Mateo, CA); Gavin K Ananda Krishnan (San Carlos, CA); Thomas O. Teisberg (Menlo Park, CA); Michael J. Demertzi (Half Moon Bay, CA)

Assignee: Zipline International, Inc.

G08G5/80G06N3/08G08G5/21G08G5/34G08G5/55G08G5/59B64U2101/30

Abstract

Claims (47)

1. A method comprising:

receiving an audio signal at audio sensors of an aircraft;

analyzing, by a detection and avoidance (DAA) model at computing resources associated with the aircraft, the received audio signal to determine directional information for a source of the audio signal;

determining, by the DAA model, a location of the source of the audio signal based on the directional information;

determining, by the DAA model, that the received audio signal is associated with an intruder aircraft at a range and an elevation relative to the aircraft and requires a maneuver to maintain an avoidance zone based on a classification of the intruder aircraft and the range and elevation of the intruder aircraft using a machine learning model of the DAA model, wherein the machine learning model is trained on acoustic data generated by different intruder sources and matched with location data for the intruder sources at different positions; and

controlling, autonomously by a flight controller of the aircraft, flight of the aircraft to perform the maneuver based on the location of the source of the audio signal and the determination that the received audio signal is associated with the intruder aircraft.

2. The method of claim 1 , wherein the location of the source of the audio signal is determined using at least one of a second machine learning model or a probabilistic tracking.

3. The method of claim 1 , wherein analyzing the received audio signal comprises:

determining first directional information of the signal source based on the received audio signal at a first point in time and a geometry of the sensors; and

determining second directional information of the signal source based at least on changes in the received audio signal at a second point in time.

4. The method of claim 3 , wherein the second directional information is determined based on changes in the received signal in response to an information-gathering maneuver of the aircraft.

5. The method of claim 1 , wherein the directional information is determined using a deep learning model of the DAA model.

6. The method of claim 1 , further comprising:

executing, by the aircraft, an initial maneuver responsive to the determination that the audio signal is associated with an intruder.

7. The method of claim 6 , further comprising:

executing one or more additional maneuvers based on information about the intruder gathered after initiation of the initial maneuver.

8. The method of claim 1 , further comprising:

beamforming the audio signal received at the sensors of the aircraft to generate a beamformed signal;

comparing the beamformed signal to a plurality of filters including known aircraft frequencies to determine that the audio signal is associated with an intruding aircraft; and

classifying the intruding aircraft as one of a plurality of possible types of aircraft based on the beamformed signal.

9. One or more non-transitory computer readable media encoded with instructions which, when executed by one or more processors of an acoustic aircraft detection system of an aircraft, cause the acoustic aircraft detection system to:

analyze an audio signal received by the acoustic aircraft detection system to determine directional information for a source of the audio signal;

generate, based on the directional information, an estimation for a location of the source of the audio signal;

analyze, by a machine learning model, the audio signal to determine that the audio signal is associated with an intruding aircraft, wherein the machine learning model is trained on aircraft data comprising acoustic data and position data, and wherein the machine learning model is trained to determine whether the audio signal is associated with an intruding aircraft;

determine a classification of the source of the audio signal; and

provide the estimation for the location of the source of the audio signal to a flight controller of the aircraft, wherein the flight controller is configured to control flight of the aircraft based on the estimation of the location of the source of the audio signal, the determination that the audio signal is associated with the intruding aircraft, and the classification of the intruder aircraft.

10. The one or more non-transitory computer readable media of claim 9 , wherein the audio signal received by the acoustic aircraft detection system is provided to a second machine learning model to determine directional information for the source of the audio signal.

11. The one or more non-transitory computer readable media of claim 10 , wherein the instructions further cause the acoustic aircraft detection system to beamform the received audio signal and to provide the beamformed audio signal to the machine learning model.

12. An unmanned aerial vehicle (UAV) comprising:

a plurality of acoustic sensors connected to the UAV;

an acoustic aircraft detection system comprising one or more processors, wherein the acoustic aircraft detection system is configured to:

analyze the audio signal using a machine learning model to determine that the audio signal is associated with an intruding aircraft, wherein the machine learning model is trained on aircraft data comprising acoustic data and position data;

analyze the audio signal to determine a classification of the source of the audio signal; and

analyze audio signal received by the plurality of acoustic sensors to generate an estimation in a three dimensional space of the intruding aircraft relative to the UAV; and

a flight controller configured to communicate with one or more control systems to control motion of the UAV, wherein the flight controller is further configured to control flight of the aircraft based on the estimation in the three dimensional space of the intruding aircraft relative to the UAV, the determination that the audio signal is associated with the intruding aircraft, and the classification of the intruding aircraft.

13. The UAV of claim 12 , wherein the flight controller is further configured to receive communications from the acoustic aircraft detection system to initiate an information-gathering maneuver of the UAV.

14. The UAV of claim 13 , wherein the acoustic aircraft detection system is configured to generate the estimation in space of the intruding aircraft relative to the UAV based at least partially on the audio signal from the intruding aircraft as the UAV executes the information-gathering maneuver.

15. The UAV of claim 12 , wherein the acoustic aircraft detection system is configured to generate the estimation in space of the intruding aircraft relative to the UAV based at least partially on a geometry of the plurality of acoustic sensors connected to the UAV.

16. The UAV of claim 12 , wherein the acoustic aircraft detection system is configured to generate the estimation in space of the intruding aircraft using at least a second machine learning model.

17. The method of claim 1 , wherein determining the location of the source of the audio signal based on the directional information comprises:

generating, by the DAA model, a state estimation of the location of the source of the audio signal based on the directional information; and

updating, by the DAA model, the state estimation over time to track a change in the location of the source of the audio signal, wherein an algorithm is used to update the state estimation.

18. The method of claim 1 , wherein the range of the intruder aircraft relative to the aircraft comprises an estimated range of the intruder aircraft relative to the aircraft and a confidence interval associated with the estimated range.

19. The method of claim 1 , wherein the elevation of the intruder aircraft relative to the aircraft comprises an estimated angle of the intruder aircraft relative to the aircraft and a confidence interval associated with the estimated angle.

20. The method of claim 1 , wherein the acoustic data used to train the machine learning model of the DAA comprises acoustic data of vehicles collected by audio sensors and synthetic acoustic data of vehicles generated by simulation, wherein the vehicles comprise airborne and non-airborne vehicles.

21. The method of claim 9 , wherein determining the classification of the source of the audio signal comprises determining the classification of the intruder aircraft associated with the audio signal, wherein the classification comprises a class of aircraft of the intruder aircraft.

22. The method of claim 12 , wherein analyzing the audio signal to determine a classification of the source of the audio signal comprises analyzing the audio signal to determine a classification of the intruding aircraft associated with the audio signal, wherein the classification comprises a class of aircraft of the intruder aircraft.

Assignments (2)

Continuity (5)

References Cited (242)