System and/or method for semantic parsing of air traffic control audio
The method S 200 can include: at an aircraft, receiving an audio utterance from air traffic control S 210 , converting the audio utterance to text, determining commands from the text using a question-and-answer model S 240 , and optionally controlling the aircraft based on the commands S 250 . The method functions to automatically interpret flight commands from the air traffic control (ATC) stream.
1. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
an autonomous computing system configured to receive an ATC audio signal from a communication system, the autonomous computing system comprising:
a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal; and
a question-and-answer (Q/A) module configured to autonomously determine an aircraft command based on the utterance hypothesis using a plurality of natural language queries, the aircraft command the comprising at least one command parameter and a set of values corresponding to each command parameter, wherein each command parameter is selected from a predetermined set of command parameters, wherein each of the set of values and the at least one command parameter are determined based on the respective result of a distinct natural language query of the plurality of natural language queries; and
a validation system communicatively coupled to the autonomous computing system and configured to:
provide the aircraft command to a pilot;
determine a pilot validation of the aircraft command, the pilot validation comprising an audit of a respective result of each natural language query of the plurality of natural language queries; and
automatically provide the aircraft command to a flight processing system of the aircraft based on the pilot validation.
2. The system of claim 1 , wherein the ATC utterances are syntactically non-standardized, wherein the aircraft command is provided to the pilot in a human-interpretable format, wherein the aircraft command is automatically provided to the flight processing system in a standardized machine-readable format.
3. The system of claim 2 , wherein the utterance hypothesis comprises a text transcript, wherein the validation system is further configured to provide the utterance hypothesis to the pilot along with the aircraft command.
4. The system of claim 1 , wherein the pilot validation is determined based on a pilot confirmation input associated with the aircraft command.
5. The system of claim 4 , wherein the pilot confirmation input is received from a remote pilot located offboard the aircraft.
6. The system of claim 1 , wherein the pilot validation is determined automatically.
7. The system of claim 1 , wherein the pilot validation comprises an update to a respective result of a natural language query based on the audit.
8. The system of claim 1 , wherein the system further comprises the flight processing system, wherein the flight processing system comprises a flight management system (FMS), a flight control system (FCS), or an autopilot (AP) system.
9. A method for semantic parsing of air traffic control (ATC) utterances for an aircraft, the method comprising:
receiving an ATC audio signal;
determining an utterance hypothesis from the ATC audio signal with automatic speech recognition (ASR);
querying the utterance hypothesis with a pre-trained neural network model according to a sequence of the natural language queries; and
based on the sequence of natural language queries, determining an aircraft commands associated with the utterance hypothesis, the aircraft command comprising at least one command parameter and a set of corresponding values for each command parameter, wherein each command parameter is selected from a predetermined set of command parameters, wherein each of the set of corresponding values and the at least one command parameter are determined based on the respective result of a distinct natural language query of the sequence;
providing the aircraft command to a pilot;
determining a pilot validation comprising an audit of a respective result of each natural language query of the sequence; and
in response to determining the pilot validation, automatically facilitating operation of the aircraft based on the aircraft command.
10. The method of claim 9 , wherein the ATC utterances are syntactically non-standardized, wherein the aircraft command is provided to the pilot in a human-interpretable format, wherein automatically facilitating operation of the aircraft based on the aircraft command comprises providing the aircraft command to a flight processing system in a standardized machine-readable format.
11. The method of claim 9 , wherein the sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to a determination that the aircraft is an intended recipient of an utterance corresponding to the utterance hypothesis.
12. The method of claim 11 , wherein the aircraft command is provided to the pilot is based on the determination that the aircraft is an intended recipient of the utterance.
13. The method of claim 12 , wherein the utterance hypothesis is determined using an integrated automatic speech recognition and sentence boundary detection (ASR/SBD) model, wherein the utterance hypothesis comprises a speaker identification, wherein the determination that the aircraft is the intended recipient is based on the speaker identification.
14. The method of claim 9 , further comprising:
based on the audit, determining an errant result associated with a natural language query of the sequence; and
based on the errant result, updating the aircraft command with a corrected command parameter or value corresponding to the errant result.
15. The method of claim 9 , wherein the pilot validation is determined based on receipt of a pilot confirmation input associated with the aircraft command.
16. The method of claim 9 , wherein the pilot validation is determined automatically.
17. The method of claim 9 , further comprising:
determining a second and a third utterance hypothesis from the ATC audio signal;
querying the second utterance hypothesis with the pre-trained neural network model according to a second set of natural language queries;
based on the second set of natural language queries, determining that the aircraft is not an intended recipient of the utterance hypothesis;
querying the third utterance hypothesis with the pre-trained neural network model according to a third set of natural language queries;
based on the third set of natural language queries, determining a second aircraft command; and
providing the second aircraft command to a pilot,
wherein the first and second aircraft commands are provided along with the utterance hypothesis and the third utterance hypothesis, respectively,
wherein the second utterance hypothesis is associated with a time period between a first time period associated with the utterance hypothesis and a second time period associated with the third utterance hypothesis,
wherein the second utterance hypothesis is not provided to the pilot.
18. A method for semantic parsing of air traffic control (ATC) utterances for an aircraft, the method comprising:
receiving an ATC audio signal;
determining an utterance hypothesis from the ATC audio signal with automatic speech recognition (ASR);
querying the utterance hypothesis with a pre-trained neural network model according to a sequence of the natural language queries;
based on the sequence of natural language queries, determining an aircraft commands associated with the utterance hypothesis;
providing the aircraft command to a pilot;
determining a pilot validation comprising an audit of a respective result of each natural language query of the plurality;
based on the audit, determining an errant result associated with a natural language query of the sequence;
based on the errant result, updating the aircraft command with a corrected command parameter or value corresponding to the errant result; and
in response to determining the pilot validation, automatically facilitating operation of the aircraft based on the aircraft command.
19. A system for an aircraft for semantic parsing of air traffic control (ATC) utterances, the system comprising:
an autonomous computing system configured to receive an ATC audio signal from a communication system, the autonomous computing system comprising:
a speech-to-text module configured to determine an utterance hypothesis from the ATC audio signal; and
a question-and-answer (Q/A) module configured to autonomously determine an aircraft command based on the utterance hypothesis using a plurality of natural language queries; and
a validation system communicatively coupled to the autonomous computing system and configured to:
provide the aircraft command to a pilot;
determine a pilot validation of the aircraft command, wherein the pilot validation is determined based on a pilot confirmation input, received from a remote pilot located offboard the aircraft, associated with the aircraft command; and
automatically provide the aircraft command to a flight processing system of the aircraft based on the pilot validation.
20. A method for semantic parsing of air traffic control (ATC) utterances for an aircraft, the method comprising:
receiving an ATC audio signal;
determining an utterance hypothesis from the ATC audio signal with automatic speech recognition (ASR);
querying the utterance hypothesis with a pre-trained neural network model according to a sequence of the natural language queries; and
based on the sequence of natural language queries, determining an aircraft command associated with the utterance hypothesis, wherein the sequence comprises a tree-based sequence with a plurality of dependencies linking one or more natural language queries to a determination that the aircraft is an intended recipient of an utterance corresponding to the utterance hypothesis;
based on the determination that the aircraft is an intended recipient of the utterance, providing the aircraft command to a pilot;
determining a pilot validation; and
in response to determining the pilot validation, automatically facilitating operation of the aircraft based on the aircraft command.