System and/or method for directed aircraft perception
The method can include performing inference using the system; and can optionally include training the system components. The method functions to automatically interpret flight commands from a stream of air traffic control (ATC) radio communications. The method can additionally or alternatively function to train and/or update a natural language processing system based on ATC communications. Additionally, the method can include or be used in conjunction with collision avoidance and/or directed perception for traffic detection associated therewith.
1. A method for air traffic control (ATC)-directed collision avoidance on an aircraft comprising:
receiving an air traffic control (ATC) audio signal from a communication system;
determining an utterance hypothesis from the ATC audio signal with automatic speech recognition (ASR);
autonomously determining a traffic advisory by querying a pre-trained neural network model with the utterance hypothesis and at least one semantic query, the traffic advisory comprising an estimated ego-relative position of an object;
locating the object associated with the traffic advisory, comprising: based on aircraft perception data, performing an extended range search with a pretrained classifier, the extended range search directed based on the estimated ego-relative position; and
performing an action based on the location of the object.
2. The method of claim 1 , wherein performing the action comprises controlling the aircraft based on the object.
3. The method of claim 1 , wherein performing the action comprises determining a resolution advisory.
4. The method of claim 1 , wherein the aircraft perception data comprises a set of camera images collected onboard the aircraft.
5. The method of claim 1 , wherein a detection range of the extended range search is between 2 and 5 nautical miles.
6. The method of claim 1 , wherein the object comprises a second aircraft.
7. The method of claim 1 , wherein the estimated ego-relative position is autonomously determined by the pre-trained neural network model.
8. A method for vehicle collision avoidance comprising:
receiving an audio signal;
determining an utterance hypothesis for the audio signal;
based on the utterance hypothesis, autonomously determining a traffic alert comprising a position estimate;
based on the traffic alert, performing a directed object detection routine with a pretrained classifier to search for an object associated with the traffic alert using vehicle perception data and the position estimate;
based on the directed object detection routine, locating the object associated with the traffic alert; and
determining a vehicle command based on the identification of the object.
9. The method of claim 8 , wherein the vehicle comprises an aircraft and the object comprises a second aircraft, wherein the traffic alert comprises a traffic advisory from air traffic control (ATC).
10. The method of claim 8 , wherein the position estimate comprises a position estimate for the object which is autonomously determined by natural language processing (NLP) of the utterance hypothesis.
11. The method of claim 8 , further comprising: automatically determining a resolution advisory, wherein the vehicle command is associated with the resolution advisory.
12. The method of claim 8 , wherein object detection routine comprises an extended-range search of a restricted search space within a set of camera images, wherein the restricted search space is based on the position estimate.