System and/or method for semantic parsing of air traffic control audio
The method S 200 can include: at an aircraft, receiving an audio utterance from air traffic control S 210 , converting the audio utterance to text, determining commands from the text using a question-and-answer model S 240 , and optionally controlling the aircraft based on the commands S 250 . The method functions to automatically interpret flight commands from the air traffic control (ATC) stream.
1. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft, the communication subsystem configured to receive an ATC radio signal and transform the ATC radio signal into an audio signal;
a first processing system connected to the communication system, comprising:
a speech-to-text module configured to determine an utterance hypothesis from the audio signal; and
a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries, the aircraft commands determined by querying a pre-trained neural network model tuned to ATC audio according to a structured sequence of the natural language queries; and
a second processing system connected to the first processing system and configured to control the aircraft based on the aircraft commands, wherein the ATC utterances are syntactically non-standardized, wherein the first processing system is configured to automatically provide the commands to the second processing system in a standardized format.
2. The system of claim 1 , wherein the first processing system is configured to convert the syntactically non-standardized utterances into the standardized format.
3. The system of claim 1 , wherein the structured sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to a determination that the aircraft is an intended recipient of an utterance corresponding to the utterance hypothesis.
4. The system of claim 3 , wherein the utterance hypothesis comprises a speaker identification, wherein the determination that the aircraft is the intended recipient is based on the speaker identification.
5. The system of claim 1 , wherein each command comprises a command parameter and a set of values corresponding to the command parameter, wherein the command parameter is selected from a predetermined set of command parameters, wherein the set of values and the command parameter are determined via distinct natural language queries of the structured sequence.
6. The system of claim 5 , wherein the structured sequence comprises a tree-based sequence, wherein the values corresponding to the command parameter are queried in response to a determination that the command parameter is a topic of the utterance.
7. The system of claim 1 , wherein the plurality of natural language queries is selected from a predetermined set.
8. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft, the communication subsystem configured to receive an ATC radio signal and transform the ATC radio signal into an audio signal; and
a first processing system connected to the communication system, comprising:
a speech-to-text module configured to determine an utterance hypothesis from the audio signal, the speech-to-text module comprising an integrated automatic speech recognition (ASR) and sentence boundary detection (SBD) module, wherein the integrated ASR/SBD module comprises a pretrained neural network tuned for ATC audio; and
a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries.
9. The system of claim 8 , wherein each utterance hypothesis comprises a boundary hypothesis, wherein the speech-to-text module comprises a sentence boundary detection (SBD) model configured to tag entities within the audio signal and generate the boundary hypothesis based on the tagged entities.
10. The system of claim 9 , wherein the entities comprise a transition speaker, wherein the SBD model is a neural network pre-trained to identify the transition speaker within multi-utterance ATC audio based on audio artifact annotations.
11. The system of claim 8 , wherein the speech-to-text module further comprises an ATC-tuned language model, wherein determining the utterance hypothesis comprises:
with the integrated ASR/SBD module, generating a plurality of linguistic hypotheses for each utterance;
using the ATC-tuned language model, determining a language score for each the plurality of linguistic hypotheses; and
selecting an utterance hypothesis from the plurality of the linguistic hypotheses based on the corresponding language score.
12. The system of claim 11 , wherein the integrated ASR/SBD module is configured to assign a phonetic score to each of the plurality of linguistic hypotheses, wherein the utterance hypothesis is selected based on a combination of the corresponding language and phonetic scores.
13. The system of claim 8 , wherein determining the utterance hypothesis comprises:
with the speech-to-text module, generating a plurality of utterance hypotheses for an utterance within the audio signal;
using a language model, selecting an utterance hypothesis of the plurality.
14. The system of claim 13 , wherein the language model comprises a neural network pre-trained using entity tagged ATC transcripts.
15. The system of claim 14 , wherein the entity tagged ATC transcripts comprises tags corresponding to phonetically conflicting entities.
16. The system of claim 8 , wherein the utterance hypothesis comprises a text transcript.
17. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft, the communication subsystem configured to receive an ATC radio signal and transform the ATC radio signal into an audio signal; and
a first processing system connected to the communication system, comprising:
a speech-to-text module configured to determine an utterance hypothesis comprising a boundary hypothesis from the audio signal, the speech-to-text module comprising a sentence boundary detection (SBD) model configured to tag entities within the audio signal and generate the boundary hypothesis based on the tagged entities, wherein the entities comprise a transition speaker, wherein the SBD model is a neural network pre-trained to identify the transition speaker within multi-utterance ATC audio based on audio artifact annotations; and
a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries.
18. The system of claim 17 , wherein the transition speaker comprises an audio artifact.
19. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
a communication system onboard the aircraft, the communication subsystem configured to receive an ATC radio signal and transform the ATC radio signal into an audio signal; and
a first processing system connected to the communication system, comprising:
a speech-to-text module comprising an automatic speech recognition (ASR) module and a language model, the speech-to-text module configured to:
generate a plurality of utterance hypotheses for an utterance within the audio signal with the ASR module; and
select an utterance hypothesis of the plurality with the language model, the language model comprising a neural network pre-trained using entity tagged ATC transcripts; and
a question-and-answer (Q/A) module configured to determine aircraft commands based on the utterance hypothesis using a plurality of natural language queries.
20. The system of claim 19 , wherein the entity tagged ATC transcripts comprise tags corresponding to phonetically conflicting entities.