IP Library › Granted Patent US 11,442,614
Granted Patent B2
US 11,442,614 · App. 17/215,512 · Granted Sep 13, 2022

Method and system for generating transcripts of patient-healthcare provider conversations

Inventors: Melissa Strader (San Jose, CA); William Ito (Mountain View, CA); Christopher Co (Saratoga, CA); Katherine Chou (Palo Alto, CA); Alvin Rajkomar (Mountain View, CA); Rebecca Rolfe (Menlo Park, CA)
Assignee: Google LLC
G06F3/04855G06F40/289G06F40/30G10L15/26G16H10/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,442,614
App. No.
17/215,512
Granted
Sep 13, 2022
Kind
B2
Abstract

A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording.

Claims (54)

1. A method for generating a transcript of a conversation between a patient and a healthcare practitioner, comprising:

providing a rendering of an audio recording of the conversation and generating a display of the transcript of the audio recording using a speech-to-text engine in substantial real time with the rendering of the audio recording;

enabling scrolling through the transcript and rendering a portion of the audio recording according to a position of the scrolling;

automatically recognizing, in the transcript or in the audio recording, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and

displaying a note simultaneously with the display of the transcript and populating the note with the words or phrases in substantial real time with the rendering of the audio recording.

2. The method of claim 1 , further comprising:

providing a set of note supplement tools enabling editing of specific portions of the note based on a content of a corresponding portion of the transcript or the audio recording.

3. The method of claim 2 , wherein the set of note supplement tools include at least one of:

a) a display of smart suggestions for words or phrases and a tool for editing, approving, rejecting or providing feedback on the smart suggestions;

b) a display of suggested corrected medical terminology; and

c) a display of an indication of confidence level in suggested words or phrases.

4. The method of claim 3 , wherein the set of note supplement tools include each of the displays a), b) and c) recited in claim 3 .

5. The method of claim 3 , wherein the display of the indication of the confidence level comprises display upon a determination that the confidence level exceeds a threshold confidence level.

6. The method of claim 1 , further comprising:

minimizing display of the transcript and viewing the note only, and wherein the note is generated in substantial real time with the rendering of the audio recording.

7. The method of claim 1 , wherein the words or phrase are placed into appropriate categories or classifications in the note.

8. The method of claim 1 , further comprising:

providing, in the note, supplementary information for symptoms including labels for phrases required for billing.

9. The method of claim 1 , further comprising:

a set of transcript supplement tools to enable editing of specific portions of the transcript based on a content of a corresponding portion of the audio recording.

10. The method of claim 1 , further comprising:

linking words or phrases in the note to relevant parts of the transcript from which the words or phrases in the note originated.

11. The method of claim 1 , wherein the automatically recognizing of the words or phrases is performed by using a trained machine learning model.

12. A server for displaying a transcript of a conversation between a patient and a healthcare practitioner, comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors, cause the server to:

receive an audio recording of the conversation;

apply a machine learning model to generate the transcript of the audio recording using a speech-to-text engine in substantial real time with a rendering of the audio recording, wherein the machine learning model is trained to automatically recognize, in the audio recording or in the transcript, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and

transmit the transcript by an application programming interface to a workstation, and wherein the instructions cause the workstation to:

generate a display of the transcript of the audio recording;

provide a user interface element enabling scrolling through the transcript and rendering a portion of the audio according to a position the scrolling;

automatically recognize, in the transcript, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and

display a note simultaneously with the display of the transcript and populating the note with the words or phrases in substantial real time with the rendering of the audio recording.

13. The server of claim 12 , wherein the instructions cause the server to:

apply the machine learning model to generate the note in substantial real time with the rendering of the audio recording; and

transmit the note by the application programming interface to the workstation.

14. The server of claim 12 , wherein the instructions further cause the workstation to:

provide a set of note supplement tools enabling editing of specific portions of the note based on a content of a corresponding portion of the transcript or the audio recording.

15. The server of claim 14 , wherein the set of note supplement tools include at least one of:

a) a display of smart suggestions for words or phrases and a tool for editing, approving, rejecting or providing feedback on the smart suggestions;

b) a display of suggested corrected medical terminology; and

c) a display of an indication of confidence level in suggested words or phrases.

16. The server of claim 15 , wherein the display of the indication of the confidence level comprises display upon a determination that the confidence level exceeds a threshold confidence level.

17. The server of claim 12 , wherein the instructions further cause the workstation to:

provide, in the note, supplementary information for symptoms including labels for phrases required for billing.

18. The server of claim 12 , wherein the instructions further cause the workstation to:

link words or phrases in the note to relevant parts of the transcript from which the words or phrases in the note originated.

19. The server of claim 12 , wherein the instructions further cause the workstation to:

minimize display of the transcript and enable viewing of the note only.

20. An article manufacture comprising one or more computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to:

provide a rendering of an audio recording of a conversation between a patient and a healthcare practitioner, and generate a display of a transcript of the audio recording using a speech-to-text engine in substantial real time with the rendering of the audio recording;

enable scrolling through the transcript and render a portion of the audio recording according to a position of the scrolling;

automatically recognize, in the transcript or in the audio recording, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and

display a note simultaneously with the display of the transcript and populate the note with the words or phrases in substantial real time with the rendering of the audio recording.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2021
From: STRADER, MELISSA; ITO, WILLIAM; CO, CHRISTOPHER; CHOU, KATHERINE; RAJKOMAR, ALVIN; ROLFE, REBECCA
To: GOOGLE LLC
Reel/Frame 055757/0511 →
Continuity (4)
Continuation 16909115 · Jun 23, 2020
Continuation 15988657 · May 24, 2018
Provisional Application 62575725 · Oct 23, 2017
Related Publication 20210216200A1 · Jul 15, 2021