IP Library Granted Patent US 12,051,513
Granted Patent B2
US 12,051,513 · App. 18/347,382 · Granted Jul 30, 2024

Medical assessment based on voice

Inventors: Jangwon Kim (Los Angeles, CA); Namhee Kwon (Manhattan Beach, CA); Henry O'Connell (Spanish Fork, UT); Phillip Walstad (Provo, UT); Kevin Shengbin Yang (Boston, MA)
Assignee: CANARY SPEECH, LLC
G16H80/00A61B5/1123A61B5/4088A61B5/4803A61B5/7267G06N3/08G06N7/01G06N20/10G10L25/66G16H10/20G16H40/67G16H50/20G16H50/50G06F2111/10G10L15/02G10L15/063G10L15/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,051,513
App. No.
18/347,382
Granted
Jul 30, 2024
Kind
B2
Abstract

Apparatuses, systems, methods, and computer program products are disclosed for medical assessment based on voice. A query module is configured to audibly question a user from an electronic display screen and/or a speaker of a computing device with one or more open ended questions. A response module is configured to receive a conversational verbal response of a user from a microphone of a computing device in response to one or more open ended questions. A detection module is configured to provide a machine learning assessment for a user of a medical condition based on a machine learning analysis of a received conversational verbal response of the user.

Claims (81)

1. A system for training a mathematical model for detecting a medical condition, the system comprising at least one computer configured to:

obtain a training corpus comprising data items, wherein each data item is labelled with a diagnosis value and wherein the training corpus comprises speech data items and non-speech data items;

compute a plurality of features for each data item in the training corpus;

compute a feature selection score for each feature of the plurality of features, wherein:

the feature selection score for a feature indicates a usefulness of the feature for detecting the medical condition, and

the feature selection score is computed using, for each data item, a value of the feature and the diagnosis value corresponding to the data item;

select a subset of the plurality of features using the feature selection scores;

train the mathematical model for detecting the medical condition using the subset of the plurality of features for each data item of the training corpus;

deploy a computer program product or computer service for detecting the medical condition using the mathematical model;

present, by the computer program product or computer service, a prompt to a person;

receive, by the computer program product or computer service, a prompted data item corresponding to the person in response to the prompt;

compute a medical diagnosis score by processing the prompted data item using the mathematical model; and

display, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

2. The system of claim 1 , wherein the at least one computer is configured to:

obtain speech recognition results for one or more speech data items of the training corpus, wherein the speech recognition results for a speech data item comprise a transcription of the speech data item; and

compute a language feature for the one or more speech data items in the training corpus by processing the speech recognition results;

wherein the plurality of features comprise the language feature.

3. The system of claim 1 , wherein each data item of the training corpus corresponds to a prompt of a plurality of prompts, the plurality of prompts comprising the presented prompt, and wherein the at least one computer is configured to:

compute a medical diagnosis score for each speech data item of the training corpus by processing the data items with the mathematical model;

compute a prompt selection score for each prompt of the plurality of prompts using the medical diagnosis scores;

select a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the presented prompt;

deploy the computer program product or computer service for detecting the medical condition using the mathematical model and the subset of prompts;

receive a data item corresponding to speech of the person for each prompt of the subset of prompts; and

compute the medical diagnosis score for the person by processing the received data items using the mathematical model.

4. The system of claim 1 , wherein the at least one computer is configured to:

compute an acoustic feature for a first speech data item in the training corpus, wherein the acoustic feature is computed from the first speech data item and wherein computation of the acoustic feature does not use speech recognition results of the first speech data item;

wherein the plurality of features comprise the acoustic feature.

5. The system of claim 4 , wherein the plurality of features comprise a language feature computed from speech recognition results of the first speech data item.

6. The system of claim 1 , wherein the mathematical model comprises a neural network or a support vector machine.

7. The system of claim 1 , wherein the plurality of features comprises at least one of spectral features, prosodic features, or voice quality features.

8. A computer-implemented method for training a mathematical model for detecting a medical condition, the method comprising:

obtaining a training corpus comprising data items, wherein each data item is labelled with a diagnosis value and wherein the training corpus comprises speech data items and non-speech data items;

computing a plurality of features for each data item in the training corpus;

computing a feature selection score for each feature of the plurality of features, wherein:

the feature selection score for a feature indicates a usefulness of the feature for detecting the medical condition, and

the feature selection score is computed using, for each data item, a value of the feature and the diagnosis value corresponding to the data item;

selecting a subset of the plurality of features using the feature selection scores;

training the mathematical model for detecting the medical condition using the subset of the plurality of features for each speech data item of the training corpus;

deploying a computer program product or computer service for detecting the medical condition using the mathematical model;

presenting, by the computer program product or computer service, a prompt to a person;

receiving, by the computer program product or computer service, a prompted data item corresponding to the person in response to the prompt;

computing a medical diagnosis score by processing the prompted data item using the mathematical model; and

displaying, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

9. The computer-implemented method of claim 8 , wherein the medical condition is a concussion or Alzheimer's disease.

10. The computer-implemented method of claim 8 , wherein the plurality of features comprises one or more of a number of pause fillers over a period of time, a number of pause fillers over a number of words, word difficulty, or speaking rate.

11. The computer-implemented method of claim 8 , wherein computing a feature selection score for a feature comprises generating a pair of numbers for each data item of the training corpus, and wherein a first number of the pair corresponds to a feature value and a second number of the pair corresponds to a diagnosis value.

12. The computer-implemented method of claim 8 , comprising:

dividing the training corpus into a plurality of folds; and

computing a statistic for each feature and each fold of the plurality of folds.

13. The computer-implemented method of claim 12 , comprising:

computing a stability determination for each feature of the plurality of features using the statistics for each feature and each fold of the plurality of folds; and

selecting the subset of the plurality of features using the stability determinations.

14. The computer-implemented method of claim 8 , comprising:

selecting a plurality of prompts using the mathematical model; and

training a second mathematical model using the selected plurality of prompts and the data items of the training corpus.

15. One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:

obtaining a training corpus comprising data items, wherein each data item is labelled with a diagnosis value and wherein the training corpus comprises speech data items and non-speech data items;

obtaining a plurality of features for each data item in the training corpus;

computing a feature selection score for each feature of the plurality of features, wherein:

the feature selection score for a feature indicates a usefulness of the feature for detecting a medical condition, and

the feature selection score is computed using, for each data item, a value of the feature and the diagnosis value corresponding to the data item;

selecting a subset of the plurality of features using the feature selection scores;

training a mathematical model for detecting the medical condition using the subset of the plurality of features for each data item of the training corpus;

deploying a computer program product or computer service for detecting the medical condition using the mathematical model;

presenting, by the computer program product or computer service, a prompt to a person;

receiving, by the computer program product or computer service, a prompted data item corresponding to the person in response to the prompt;

computing a medical diagnosis score by processing the prompted data item using the mathematical model; and

displaying, by the computer program product or computer service, one or more of the medical diagnosis score or a medical diagnosis based on the medical diagnosis score.

16. The one or more non-transitory computer-readable media of claim 15 , wherein computing a first feature of the plurality of features comprises:

computing a value for each short-time segment of an audio signal to obtain a plurality of values; and

computing the first feature using the plurality of values.

17. The one or more non-transitory computer-readable media of claim 15 , wherein the feature selection score comprises an adjusted Rand index, adjusted mutual information, an absolute Pearson correlation, or an absolute Spearman correlation.

18. The one or more non-transitory computer-readable media of claim 15 , wherein the actions comprise:

computing a stability determination for each feature of the plurality of features; and

selecting the plurality of features using the stability determinations.

19. The one or more non-transitory computer-readable media of claim 15 , wherein each data item of the training corpus corresponds to a prompt of a plurality of prompts, the plurality of prompts comprising the presented prompt, and wherein the actions comprise:

computing a medical diagnosis score for each data item of the training corpus by processing the data items with the mathematical model;

computing a prompt selection score for each prompt of the plurality of prompts using the medical diagnosis scores;

selecting a subset of prompts from the plurality of prompts using the prompt selection scores, the subset of prompts comprising the presented prompt; and

deploying the computer program product or computer service for detecting the medical condition using the mathematical model and the subset of prompts.

20. The one or more non-transitory computer-readable media of claim 15 , wherein the plurality of features comprise a non-speech feature.

Assignments (2)
CHANGE OF NAME Recorded Jun 12, 2025
From: CANARY SPEECH, LLC
To: CANARY SPEECH, INC.
Reel/Frame 071400/0946 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2024
From: KIM, JANGWON; KWON, NAMHEE; O'CONNELL, HENRY; WALSTAD, PHILLIP; YANG, KEVIN SHENGBIN
To: CANARY SPEECH, LLC
Reel/Frame 067692/0182 →
Continuity (6)
Continuation 17827970 · May 30, 2022
Continuation 16422718 · May 24, 2019
Continuation 15973504 · May 7, 2018
Provisional Application 62614192 · Jan 5, 2018
Provisional Application 62502584 · May 5, 2017
Related Publication 20230352194A1 · Nov 2, 2023