IP Library › Granted Patent US 11,955,012
Granted Patent B2
US 11,955,012 · App. 17/459,675 · Granted Apr 9, 2024

Transcription systems and message fusion methods

Inventors: Gobinathan Baladhandapani (Madurai, IN); Hariharan Saptharishi (Madurai, IN); Sivakumar Kanagarajan (Bangalore, IN); Vengada Ganeshwaran (Bangalore, IN); Dineshkumar Atchuthan (Madurai, IN)
Assignee: HONEYWELL INTERNATIONAL INC.
G08G5/0004G08G5/0021G08G5/003G08G5/0095
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,955,012
App. No.
17/459,675
Granted
Apr 9, 2024
Kind
B2
Abstract

Methods and systems are provided for assisting operation of a vehicle using speech recognition and transcription to provide a conversation log graphical user interface (GUI) display that consolidates communications with respect to the vehicle. One method involves receiving a data message from a system onboard the vehicle and generating a graphical representation of the data message within the conversation log GUI display, wherein the graphical representation of the data message includes a selectable element and is displayed on the conversation log GUI display concurrently with a graphical representation of a transcription of an audio communication with respect to the vehicle. The depicted data message and transcription are chronologically positioned with respect to one another in accordance with a timestamp associated with the data message. The conversation log GUI display dynamically updates in response to selection of the selectable element.

Claims (65)

1. A method of assisting operation of a vehicle, the method comprising:

obtaining a transcription of an audio communication with respect to the vehicle;

generating a conversation log graphical user interface (GUI) display comprising a first graphical representation of the transcription of the audio communication;

receiving a data message from an onboard system associated with the vehicle;

generating a second graphical representation of the data message within the conversation log GUI display, wherein:

the second graphical representation of the data message comprises a selectable element associated with the data message;

the second graphical representation of the data message and the first graphical representation of the transcription of the audio communication are concurrently displayed on the conversation log GUI display; and

the second graphical representation of the data message is positioned with respect to the first graphical representation of the transcription of the audio communication in accordance with a timestamp associated with the data message; and

dynamically updating the conversation log GUI display in response to selection of the selectable element.

2. The method of claim 1 , wherein:

the vehicle comprises an aircraft;

receiving the data message comprises receiving a text message from an avionics system onboard the aircraft; and

the second graphical representation of the data message comprises indication of the avionics system associated with the text message.

3. The method of claim 1 , further comprising identifying a source of the data message, wherein the second graphical representation of the data message comprises indication of the source of the data message.

4. The method of claim 3 , wherein:

the vehicle comprises an aircraft;

receiving the data message comprises receiving a datalink message from a controller-pilot datalink communications (CPDLC) system onboard the aircraft;

the second graphical representation of the data message comprises a graphical representation of the datalink message; and

the indication of the source comprises an indication of the CPDLC system associated with the datalink message.

5. The method of claim 1 , wherein dynamically updating the conversation log GUI display comprises removing the selectable element from the conversation log GUI display in response to selection of the selectable element.

6. The method of claim 1 , wherein dynamically updating the conversation log GUI display comprises dynamically updating the second graphical representation of the data message to provide an indication of an action associated with the selectable element in response to selection of the selectable element.

7. The method of claim 1 , wherein dynamically updating the conversation log GUI display comprises generating a third graphical representation of a second message within the conversation log GUI display, wherein the third graphical representation of the second message comprises an indication of an action associated with the selectable element.

8. The method of claim 1 , further comprising:

generating an entry associated with the data message in a message log; and

updating a field associated with the entry in the message log to indicate an action associated with the selectable element in response to selection of the selectable element.

9. The method of claim 1 , further comprising providing an output signal indicative of an action associated with the selectable element to a second system onboard the vehicle in response to selection of the selectable element.

10. The method of claim 9 , wherein:

the vehicle comprises an aircraft;

receiving the data message comprises receiving a datalink message comprising a request from a controller-pilot datalink communications (CPDLC) system onboard the aircraft;

the second graphical representation of the data message comprises a graphical representation of the datalink message; and

providing the output signal comprises instructing an avionics system onboard the aircraft to implement the request in response to selection of the selectable element associated with acceptance of the request.

11. The method of claim 1 , wherein generating the second graphical representation comprises:

translating the data message from a standard phraseology into a conversational form, resulting in an augmented message; and

generating the second graphical representation of the augmented message.

12. The method of claim 1 , wherein:

obtaining the transcription comprises obtaining transcribed text of the audio communication from an entry in a clearance table; and

the first graphical representation of the transcription comprises the transcribed text of the audio communication.

13. The method of claim 1 , wherein:

the vehicle comprises an aircraft;

a flight management system (FMS) onboard the aircraft is configured to receive the data message from an avionics system onboard the aircraft vehicle; and

receiving the data message comprises receiving, at an electronic device, the data message from the FMS over a wireless network.

14. The method of claim 13 , wherein the data message comprises a datalink message from a controller-pilot datalink communications (CPDLC) system onboard the aircraft.

15. A computer-readable medium having computer-executable instructions stored thereon that, when executed by a processing system, cause the processing system to:

obtain a transcription of an audio communication with respect to a vehicle;

generate a conversation log graphical user interface (GUI) display comprising a first graphical representation of the transcription of the audio communication;

receive a data message from an onboard system onboard associated with the vehicle;

generate a second graphical representation of the data message within the conversation log GUI display, wherein:

the second graphical representation of the data message comprises a selectable element associated with the data message;

the second graphical representation of the data message and the first graphical representation of the transcription of the audio communication are concurrently displayed on the conversation log GUI display; and

the second graphical representation of the data message is positioned with respect to the first graphical representation of the transcription of the audio communication in accordance with a timestamp associated with the data message; and

dynamically update the conversation log GUI display in response to selection of the selectable element.

16. The computer-readable medium of claim 15 , wherein the computer-executable instructions cause the processing system to generate an entry associated with the data message in a message log and update a field associated with the entry in the message log to indicate an action associated with the selectable element in response to selection of the selectable element.

17. The computer-readable medium of claim 15 , wherein the computer-executable instructions cause the processing system to provide an output signal indicative of an action associated with the selectable element to a second system onboard the vehicle in response to selection of the selectable element.

18. The computer-readable medium of claim 15 , wherein the data message comprises a datalink message from a controller-pilot datalink communications (CPDLC) system onboard an aircraft.

19. The computer-readable medium of claim 18 , wherein the computer-executable instructions cause the processing system to receive the datalink message from a flight management system (FMS) onboard the aircraft over a wireless network.

20. A system comprising:

a display device;

a data storage element to maintain a transcription of an audio communication with respect to a vehicle; and

a processing system coupled to the display device and the data storage element to:

generate, on the display device, a conversation log graphical user interface (GUI) display comprising a first graphical representation of the transcription of the audio communication;

receive a data message from an onboard system associated with the vehicle;

generate a second graphical representation of the data message within the conversation log GUI display, the second graphical representation of the data message comprising a selectable graphical user interface element associated with the data message; and

dynamically update the conversation log GUI display in response to selection of the selectable graphical user interface element associated with the data message, wherein:

the second graphical representation of the data message and the first graphical representation of the transcription of the audio communication are concurrently displayed on the conversation log GUI display; and

the second graphical representation of the data message is positioned with respect to the first graphical representation of the transcription of the audio communication in accordance with a timestamp associated with the data message.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2021
From: BALADHANDAPANI, GOBINATHAN; SAPTHARISHI, HARIHARAN; KANAGARAJAN, SIVAKUMAR; GANESHWARAN, VENGADA; ATCHUTHAN, DINESHKUMAR
To: HONEYWELL INTERNATIONAL INC.
Reel/Frame 057314/0890 →
Priority Claims (1)
IN 202111031126 · Jul 12, 2021 · national
Continuity (1)
Related Publication 20230010103A1 · Jan 12, 2023
Cited By (1)
US 12,322,392