System and/or method for semantic parsing of air traffic control audio
The method S 200 can include: at an aircraft, receiving an audio utterance from air traffic control S 210 , converting the audio utterance to text, determining commands from the text using a question-and-answer model S 240 , and optionally controlling the aircraft based on the commands S 250 . The method functions to automatically interpret flight commands from the air traffic control (ATC) stream.
1. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft; and
a computing system communicatively coupled to the communication system and configured to receive an ATC audio signal from the communication system, the computing system comprising:
a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal, the utterance hypothesis comprising a speaker identification; and
a question-and-answer (Q/A) module configured to determine aircraft commands by querying the utterance hypothesis with a pre-trained neural network model according to a sequence of natural language queries, wherein the sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to a determination, based on the speaker identification, that the aircraft is an intended recipient of an utterance corresponding to the utterance hypothesis.
2. The system of claim 1 , further comprising a second computing system connected to the computing system and configured to control the aircraft based on the aircraft commands.
3. The system of claim 2 , wherein the ATC utterances are syntactically non-standardized, wherein the computing system is configured to automatically provide the commands to the second computing system in a standardized format.
4. The system of claim 1 , wherein the pre-trained neural network model is tuned with ATC audio.
5. The system of claim 1 , wherein the utterance hypothesis comprises a text transcript.
6. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft;
a computing system communicatively coupled to the communication system and configured to receive an ATC audio signal from the communication system, the computing system comprising:
a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal comprising a first pre-trained neural network, wherein the first pre-trained neural network is pre-trained to identify a transition speaker associated with audio artifacts; and
a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries.
7. The system of claim 6 , wherein the question-and-answer (Q/A) module is configured to determine aircraft commands by querying the utterance hypothesis with a second pre-trained neural network model according to a structured sequence of the natural language queries.
8. The system of claim 7 , wherein at least one of first or second pre-trained neural network models has been tuned for ATC audio data.
9. The system of claim 6 , wherein the utterance hypothesis comprises a boundary hypothesis, wherein the first pre-trained neural network is further configured to tag entities within the ATC audio signal and generate the boundary hypothesis based on the tagged entities.
10. A method for semantic parsing of air traffic control (ATC) utterances for an aircraft, the method comprising:
receiving an ATC audio signal;
based on an ATC-tuned language model, determining an utterance hypothesis from the ATC audio signal, comprising:
with an integrated automatic speech recognition (ASR) and sentence boundary detection (SBD) module, generating a set of linguistic hypotheses based on the ATC audio signal;
using the ATC-tuned language model, determining a respective language score for each linguistic hypothesis of the set of linguistic hypotheses; and
determining the utterance hypothesis from the set of the linguistic hypotheses based on the respective language scores;
querying the utterance hypothesis with a pre-trained neural network model according to a sequence of the natural language queries; and
based on the sequence of natural language queries, determining a set of aircraft commands associated with the utterance hypothesis.
11. The method of claim 10 , wherein the pre-trained neural network model is ATC-tuned.
12. The method of claim 10 , wherein the sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to an initial query.
13. The method of claim 10 , wherein the utterance hypothesis comprises a text transcript.
14. The method of claim 10 , wherein determining the utterance hypothesis further comprises: determining a respective phonetic score for each linguistic hypothesis, wherein the utterance determined based on a combination of the respective language scores and respective phonetic scores of each linguistic hypothesis of the set of linguistic hypotheses.