Granted Patent B2

US 12,597,512 · App. 19/302,810 · Granted Apr 7, 2026

Inventors: Subhabrata Mukherjee (Palo Alto, CA); Shanil Puri (Palo Alto, CA); Tanmay Laud (Palo Alto, CA); Woojeong Jin (Palo Alto, CA); Jan Schellenberger (Palo Alto, CA)

Assignee: HealthGPT, Inc.

G16H40/20G10L15/183G10L15/22G10L25/66G10L15/1822G10L15/19

Abstract

Claims (104)

1 . A method, comprising:

initiating, by a conversational artificial intelligence executed by one or more processors, a conversation with a human;

receiving, by the conversational artificial intelligence, a human response from the human;

augmenting, by the one or more processors, the human response to create an augmented response;

determining, by the one or more processors, a context of the conversation;

providing, by the one or more processors, the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel;

receiving, by the one or more processors, a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules;

performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the human to create a final text output; and

generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output.

2 . The method of claim 1 , wherein augmenting the human response to create the augmented response comprises at least one of:

performing an intonation analysis of the human response;

performing noise cancellation by reducing an amount of background noise present in the human response; or

any combination thereof.

3 . The method of claim 1 , wherein determining the context of the conversation comprises:

accessing electronic medical records (EMR) associated with the human; and

determining a conversation history of the conversation between the conversational artificial intelligence and the human.

4 . The method of claim 1 , wherein the plurality of ASR modules comprise at least:

a first ASR module implemented using a first artificial intelligence algorithm;

a second ASR module implemented using a second artificial intelligence algorithm; and

a third ASR module implemented using a third artificial intelligence algorithm;

wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.

5 . The method of claim 1 , wherein the plurality of ASR modules comprise at least:

a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;

a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and

a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;

wherein the first type of accent, the second type of accent, and the third type of accent are different from each other.

6 . The method of claim 1 , wherein the conversational artificial intelligence is trained using multi-turn reinforcement learning through human feedback (RLHF).

7 . The method of claim 1 , wherein the conversational artificial intelligence, during the conversation with the human:

identifies a turn-yielding cue;

performs interruption detection to detect when the human is attempting to interrupt the conversationa; artificial intelligence;

identifies a non-verbal cue associated with the human; or

any combination thereof.

8 . A server comprising:

one or more processors; and

one or more computer-readable storage media to store instructions executable by the one or more processors to perform operations comprising:

initiating, by a conversational artificial intelligence, a conversation with a human;

receiving, by the conversational artificial intelligence, a human response from the human;

augmenting the human response to create an augmented response;

determining a context of the conversation;

providing the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel;

receiving a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules;

performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the human to create a final text output; and

generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output.

9 . The server of claim 8 , wherein augmenting the human response to create the augmented response comprises at least one of:

performing an intonation analysis of the human response including determining a volume, a pitch, and a rhythm of individual words in the human response;

performing noise cancellation by identifying speech content spoken by the human and reducing the volume of other content in the human response including other human speech; or

any combination thereof.

10 . The server of claim 8 , wherein determining the context of the conversation comprises:

determining electronic medical records (EMR) associated with the human; and

determining a conversation history of the conversation between the conversational artificial intelligence and the human.

11 . The server of claim 10 , wherein the conversation history of the conversation between the conversational artificial intelligence and the human is stored in a cache memory.

12 . The server of claim 8 , wherein the plurality of ASR modules comprise at least:

a first ASR module implemented using a first artificial intelligence algorithm;

a second ASR module implemented using a second artificial intelligence algorithm; and

a third ASR module implemented using a third artificial intelligence algorithm;

wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.

13 . The server of claim 8 , wherein the plurality of ASR modules comprise at least:

a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;

a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and

a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;

wherein the first type of accent, the second type of accent, and the third type of accent are different from each other.

14 . The server of claim 8 , wherein the conversational artificial intelligence is configured to perform specialized healthcare-related functions comprising one or more of:

gathering data related to performing Healthcare Effectiveness Data and Information Set (HEDIS) calculations;

performing a Health Records Assessment (HRA);

determining a Risk Adjustment Factor (RAF);

reviewing a pre-op checklist;

reviewing a discharge checklist;

reviewing a chronic care checklist;

determining social determinants of health (SDOH); or

any combination thereof.

15 . A non-transitory memory device to store instructions executable by one or more processors to perform operations comprising:

initiating, by a conversational artificial intelligence, a conversation with a human;

receiving, by the conversational artificial intelligence, a human response from the human;

augmenting the human response to create an augmented response;

determining a context of the conversation;

providing the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel;

receiving a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules;

performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the patient to create a final text output; and

generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output.

16 . The non-transitory memory device of claim 15 , wherein augmenting the human response to create the augmented response comprises at least one of:

performing an intonation analysis of the human response including determining a volume, a pitch, and a rhythm of individual words in the human response;

performing noise cancellation by identifying speech content spoken by the human and reducing the volume of other content in the human response including other human speech; or

any combination thereof.

17 . The non-transitory memory device of claim 15 , wherein determining the context of the conversation comprises:

determining electronic medical records (EMR) associated with the human; and

determining a conversation history of the conversation between the conversational artificial intelligence and the human.

18 . The non-transitory memory device of claim 15 , wherein the plurality of ASR modules comprise at least:

a first ASR module implemented using a first artificial intelligence algorithm;

a second ASR module implemented using a second artificial intelligence algorithm; and

a third ASR module implemented using a third artificial intelligence algorithm;

wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.

19 . The non-transitory memory device of claim 15 , wherein the plurality of ASR modules comprise at least:

a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;

a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and

a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;

wherein the first type of accent, the type of second accent, and the third type of accent are different from each other.

20 . The non-transitory memory device of claim 15 , wherein the conversational artificial intelligence is engaged in a task that includes one or more of:

performing a preventative screening;

an intake-related task;

a scheduling-related task;

a pre-op related task;

a discharge-related task;

a chronic care related task; or

any combination thereof.

Assignments (1)

Continuity (5)

References Cited (31)