IP Library Granted Patent US 12,176,080
Granted Patent B2
US 12,176,080 · App. 17/571,799 · Granted Dec 24, 2024

System and method for review of automated clinical documentation from recorded audio

Inventors: Paul Joseph Vozila (Arlington, MA); Guido Remi Marcel Gallopyn (Newburyport, MA); Uwe Helmut Jost (Groton, MA); Matthias Helletzgruber (Vienna, AT); Jeremy Martin Jancsary (Vienna, AT); Kumar Abhinav (Montreal, CA); Joel Praveen Pinto (Aachen, DE); Donald E. Owen (Orlando, FL); Mehmet Mert Öz (Baden, AT)
Assignee: Microsoft Technology Licensing, LLC
G16H10/60G06F3/0482G06F3/0485G06F3/165G06F3/167G06F40/169G06N20/00G10L15/22G10L15/26G16H10/20G16H15/00G16H40/60G06F3/0334G06F3/0362G06F3/0489
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,176,080
App. No.
17/571,799
Granted
Dec 24, 2024
Kind
B2
Abstract

A method, computer program product, and computing system for obtaining, by a computing device, encounter information of a patient encounter, wherein the encounter information may include audio encounter information obtained from at least a first encounter participant. The audio encounter information obtained from at least the first encounter participant may be processed. A user interface may be generated displaying a plurality of layers associated with the audio encounter information obtained from at least the first encounter participant. A user input may be received from a peripheral device to navigate through each of the plurality of layers associated with the audio encounter information displayed on the user interface.

Claims (73)

1. A computer-implemented method comprising:

obtaining, by a computing device, encounter information of a patient encounter, wherein the encounter information includes audio encounter information obtained from at least a first encounter participant;

processing the audio encounter information obtained from at least the first encounter participant including generating an encounter transcript and populating at least a portion of a medical report;

generating a user interface displaying a plurality of layers associated with the audio encounter information obtained from at least the first encounter participant;

determining a confidence level that a time for applying corrections to at least a portion of one or more of the layers is less than a time for manually typing the at least a portion of the one or more layers;

exposing the one or more layers when the confidence level is above a threshold;

determining that at least a portion of the audio encounter information lacks relevance to the medical report including a portion of the audio encounter information that does not attribute significant responsibility for the medical report wherein accumulated attribution of the portion of the audio encounter information toward the medical report is below a threshold; and

automatically speeding up at least the portion of the audio encounter information that is determined to lack relevance to the medical report while the audio encounter information is being playback.

2. The computer-implemented method of claim 1 wherein processing the first audio encounter information includes defining linkages between each of the plurality of layers associated with the audio encounter information.

3. The computer-implemented method of claim 1 further including:

receiving a user input of a selection of a first portion of the audio encounter information at the first layer of the plurality of layers on the user interface; and

displaying an annotation of at least one of the second layer of the plurality of layers and the third layer of the plurality of layers corresponding to the first portion of the audio encounter information of the first layer of the plurality of layers selected on the user interface.

4. The computer-implemented method of claim 3 wherein receiving the user input includes:

receiving, via the user input, a selection of the first portion of the audio encounter information at one of the second layer of the plurality of layers and the third layer of the plurality of layers on the user interface; and

providing audio of the first layer corresponding to the first portion of the audio encounter information of one of the second layer of the plurality of layers and the third layer of the plurality of layers selected on the user interface.

5. The computer-implemented method of claim 3 wherein the user input is received from a peripheral device that includes at least one of a keyboard, a pointing device, a foot pedal, and a dial, and wherein the user input from the peripheral device includes at least one of:

a keyboard shortcut when the peripheral device is the keyboard;

a pointing device action when the peripheral device is the pointing device;

raising and lowering of the foot pedal when the peripheral device is the foot pedal; and

at least one of a rotating action, an up action, a down action, a left action, a right action, and a pressing action of the dial when the peripheral device is the dial.

6. The computer-implemented method of claim 5 wherein the user input from the peripheral device causes the user interface to at least one of:

switch between sentences in an output of the medical report;

switch between sections in the output of the medical report;

switch between the medical report and the transcript;

one of providing audio of the audio signal and ceasing audio of the audio signal; and

one of speeding up the audio of the audio signal and slowing down the audio of the audio signal.

7. The computer-implemented method of claim 1 wherein a first layer of the plurality of layers is an audio signal associated with the audio encounter information, wherein a second layer of the plurality of layers is a transcript associated with the audio encounter information, and wherein a third layer of the plurality of layers is a medical report associated with the audio encounter information.

8. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:

obtaining, by a computing device, encounter information of a patient encounter, wherein the encounter information includes audio encounter information obtained from at least a first encounter participant;

processing the audio encounter information obtained from at least the first encounter participant including generating an encounter transcript and populating at least a portion of a medical report;

generating a user interface displaying a plurality of layers associated with the audio encounter information obtained from at least the first encounter participant;

determining a confidence level that a time for applying corrections to at least a portion of one or more of the layers is less than a time for manually typing the at least a portion of the one or more layers;

exposing the one or more layers when the confidence level is above a threshold;

determining that at least a portion of the audio encounter information lacks relevance to the medical report including a portion of the audio encounter information that does not attribute significant responsibility for the medical report wherein accumulated attribution of the portion of the audio encounter information toward the medical report is below a threshold; and

automatically speeding up at least the portion of the audio encounter information that is determined to lack relevance to the medical report while the audio encounter information is being playback.

9. The computer program product of claim 8 wherein processing the first audio encounter information includes defining linkages between each of the plurality of layers associated with the audio encounter information.

10. The computer program product of claim 8 further including:

receiving a user input of a selection of a first portion of the audio encounter information at the first layer of the plurality of layers on the user interface; and

displaying an annotation of at least one of the second layer of the plurality of layers and the third layer of the plurality of layers corresponding to the first portion of the audio encounter information of the first layer of the plurality of layers selected on the user interface.

11. The computer program product of claim 10 wherein receiving the user input includes:

receiving, via the user input, a selection of the first portion of the audio encounter information at one of the second layer of the plurality of layers and the third layer of the plurality of layers on the user interface; and

providing audio of the first layer corresponding to the first portion of the audio encounter information of one of the second layer of the plurality of layers and the third layer of the plurality of layers selected on the user interface.

12. The computer program product of claim 10 wherein the user input is received from a peripheral device that includes at least one of a keyboard, a pointing device, a foot pedal, and a dial, and wherein the user input from the peripheral device includes at least one of:

a keyboard shortcut when the peripheral device is the keyboard;

a pointing device action when the peripheral device is the pointing device;

raising and lowering of the foot pedal when the peripheral device is the foot pedal; and

at least one of a rotating action, an up action, a down action, a left action, a right action, and a pressing action of the dial when the peripheral device is the dial.

13. The computer program product of claim 8 wherein a first layer of the plurality of layers is an audio signal associated with the audio encounter information, wherein a second layer of the plurality of layers is a transcript associated with the audio encounter information, and wherein a third layer of the plurality of layers is a medical report associated with the audio encounter information.

14. The computer program product of claim 8 wherein the operations further comprise annotating at least a portion of the audio encounter information determined to lack relevance to the medical report.

15. A computing system including a processor and memory configured to perform operations comprising:

obtaining, by a computing device, encounter information of a patient encounter, wherein the encounter information includes audio encounter information obtained from at least a first encounter participant;

processing the audio encounter information obtained from at least the first encounter participant including generating an encounter transcript and populating at least a portion of a medical report;

generating a user interface displaying a plurality of layers associated with the audio encounter information obtained from at least the first encounter participant;

determining a confidence level that a time for applying corrections to at least a portion of one or more of the layers is less than a time for manually typing the at least a portion of the one or more layers;

exposing the one or more layers when the confidence level is above a threshold;

determining that at least a portion of the audio encounter information lacks relevance to the medical report including a portion of the audio encounter information that does not attribute significant responsibility for the medical report wherein accumulated attribution of the portion of the audio encounter information toward the medical report is below a threshold; and

automatically speeding up at least the portion of the audio encounter information that is determined to lack relevance to the medical report while the audio encounter information is being playback.

16. The computing system of claim 15 wherein processing the first audio encounter information includes defining linkages between each of the plurality of layers associated with the audio encounter information.

17. The computing system of claim 16 wherein the user input is received from a peripheral device that includes at least one of a keyboard, a pointing device, a foot pedal, and a dial, and wherein the user input from the peripheral device includes at least one of:

a keyboard shortcut when the peripheral device is the keyboard;

a pointing device action when the peripheral device is the pointing device;

raising and lowering of the foot pedal when the peripheral device is the foot pedal; and

at least one of a rotating action, an up action, a down action, a left action, a right action, and a pressing action of the dial when the peripheral device is the dial.

18. The computing system of claim 17 wherein the user input from the peripheral device causes the user interface to at least one of:

switch between sentences in an output of the medical report;

switch between sections in the output of the medical report;

switch between the medical report and the transcript;

one of providing audio of the audio signal and ceasing audio of the audio signal; and

one of speeding up the audio of the audio signal and slowing down the audio of the audio signal.

19. The computing system of claim 15 further including:

receiving a user input of a selection of a first portion of the audio encounter information at the first layer of the plurality of layers on the user interface and displaying an annotation of at least one of the second layer of the plurality of layers and the third layer of the plurality of layers corresponding to the first portion of the audio encounter information of the first layer of the plurality of layers selected on the user interface; and

receiving, via the user input, a selection of the first portion of the audio encounter information at one of the second layer of the plurality of layers and the third layer of the plurality of layers on the user interface and providing audio of the first layer corresponding to the first portion of the audio encounter information of one of the second layer of the plurality of layers and the third layer of the plurality of layers selected on the user interface.

20. The computing system of claim 15 wherein a first layer of the plurality of layers is an audio signal associated with the audio encounter information, wherein a second layer of the plurality of layers is a transcript associated with the audio encounter information, and wherein a third layer of the plurality of layers is a medical report associated with the audio encounter information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065532/0152 →
Continuity (4)
Continuation 16292920 · Mar 5, 2019
Provisional Application 62803193 · Feb 8, 2019
Provisional Application 62638809 · Mar 5, 2018
Related Publication 20220130502A1 · Apr 28, 2022