Mood monitoring of bipolar disorder using speech analysis
A system that monitors and assesses the moods of subjects with neurological disorders, like bipolar disorder, by analyzing normal conversational speech to identify speech data that is then analyzed through an automated speech data classifier. The classifier may be based on a vector, separator, hyperplane, decision boundary, or other set of rules to classify one or more mood states of a subject. The system classifier is used to assess current mood state, predicted instability, and/or a change in future mood state, in particular for subjects with bipolar disorder.
1. A method of detecting a speech-identifiable condition of a subject, the method comprising:
recording speech data of the subject via a communication device input that receives the speech data for the subject while not receiving speech data for anyone else talking to the subject;
transmitting, by the communication device, the recorded speech data to a mood detection machine that includes a feature extraction module and a decision module;
performing, in the feature extraction module, a low-level feature extraction on the speech data over a plurality of short-time segments to develop low-level feature data;
performing, in the feature extraction module, a segment-level feature extraction on the low-level feature data over a window of time to develop segment-level feature data, where the window of time comprises the plurality of short time segments, and
wherein the low-level feature extraction combined with the segment-level feature extraction masks out words contained in the speech data such that the segment-level feature data is non-lexical data;
applying the segment-level feature data to the decision module that includes a database of one or more classifiers, each classifier from among the classifiers corresponding to a different classification of the speech-identifiable condition; and
determining, in the decision module, the classification of the speech-identifiable condition of the subject from the segment-level feature data.
2. The method of claim 1 , wherein the speech-identifiable condition is a mood state of the subject.
3. The method of claim 2 , wherein the subject has bipolar disorder and the mood state comprises one or more of depression, hypomania/mania, and euthymia.
4. The method of claim 3 , wherein the one or more classifiers comprises a classifier for identifying each of depression, hypomania/mania, and euthymia from the speech data.
5. The method of claim 1 , wherein the speech data is unstructured speech data.
6. The method of claim 1 , wherein the speech data is structured speech data.
7. The method of claim 1 , wherein the low-level feature data includes one or more of pitch, energy, spectrum, zero-crossing rate, maximum waveform amplitude value, and minimum waveform amplitude value.
8. The method of claim 1 , wherein the segment-level feature data includes one or more of mean, variance, jitter, and shimmer.
9. The method of claim 1 , wherein each of the plurality of short time segments is 25 ms or less.
10. The method of claim 9 , wherein the window of time is 3 s or greater.
11. The method of claim 10 , wherein the window of time is from 3 s to 30 s.
12. The method of claim 1 , wherein each of the one or more classification rules are support vector machine (SVMs).
13. The method of claim 12 , further comprising:
developing each of the one or more classifiers based on more than one of the segment-level data.
14. The method of claim 1 , wherein determining, in the decision module, the classification of the speech-identifiable condition of the subject from the segment-level feature data comprises determining an expected future change in the speech-identifiable condition of the subject.
15. The method of claim 1 , wherein the speech-identifiable condition includes one or more of psychiatrically diagnosed conditions, pain, depression, physical conditions, congenital heart disorders, coughing, lung related disorders, lung cancer, oncological disorders, Grave's disease, hearing impairment, neuromuscular disorders, and neurological disorders.
16. A mood detection machine to detect a speech-identifiable condition of a subject, comprising:
a processor;
a memory;
a feature extraction module stored on the memory and adapted to cause the processor to:
receive recorded speech data of the subject, the recorded speech data being transmitted by a communication device and recorded by an input of the communication device that receives the speech data for the subject while not receiving speech data for anyone else talking to the subject;
perform, on the recorded speech data, a low-level feature extraction over a plurality of short-time segments to develop low-level feature data, and
perform a segment-level feature extraction on the low-level feature data over a window of time to develop segment-level feature data, where the window of time comprises the plurality of short time segments contained therein,
wherein the low-level feature extraction combined with the segment-level feature extraction masks out words contained in the recorded speech data such that the segment-level feature data is non-lexical data; and
a decision module stored on the memory and adapted to cause the processor to:
access a database of one or more classifiers, each classifier from among the classifiers corresponding to a different classification of the speech-identifiable condition; and
determining the classification of the speech-identifiable condition of the subject from the segment-level feature data.
17. The mood detection machine of claim 16 , wherein the segment-level feature data is non-lexical data.
18. The mood detection machine of claim 16 , wherein the communication device is a cellular phone or a smart phone located remotely with respect to the mood detection machine, and
wherein the communication device input is a microphone associated with the communication device.
19. The mood detection machine of claim 16 , wherein the recorded speech data in unstructured speech data.
20. The method of claim 1 , wherein the communication device is a cellular phone or a smart phone located remotely with respect to the mood detection machine, and
wherein the communication device input is a microphone.