Sensor fusion to validate sound-producing behaviors

1. A method to measure sound-producing behaviors of a subject with a power- and bandwidth-limited electronic device that includes a processor, the method comprising:

measuring, by at least one sensor communicatively coupled to the processor, at least one parameter other than sound to generate a second data signal that represents the at least one parameter other than sound;

detecting a wake-up event from the second data signal;

providing power to a microphone communicatively coupled to the processor such that the microphone measures sound in a vicinity of the subject to generate an audio data signal that represents the sound;

detecting one or more sound-producing behaviors of the subject based on:

both the audio data signal and second data signal; or

information derived from both the audio data signal and the second data signal, the detecting comprising:

extracting, by the processor, a plurality of audio features from the audio data signal;

extracting, by the processor, a plurality of second features from the second data signal;

classifying each audio feature of a first subset of the plurality of audio features and each feature of a first subset of the plurality of second features as indicative of a sound-producing behavior; and

detecting the one or more sound-producing behaviors of the subject based on the classifying; and

measuring sound in the vicinity of the subject within a subsequent period of time to generate a subsequent audio data signal that represents the sound;

extracting, by the processor, a subsequent plurality of audio features from the subsequent audio data signal;

classifying each audio feature of a first subset of the subsequent plurality of audio features as indicative of a sound-producing behavior;

determining a quality score of the subsequent audio data signal; and

one of:

in response to determining that the quality score of the subsequent audio data signal exceeds a first threshold quality score, detecting one or more sound-producing behaviors of the subject that occur within the subsequent period of time based exclusively on the first subset of the subsequent plurality of audio features; or

in response to determining that the quality score of the subsequent audio data signal is less than a second threshold quality score that is lower than the first threshold quality score, discarding one or both of the subsequent audio data signal and the first subset of the subsequent plurality of audio features without detecting any sound-producing behaviors of the subject that occur within the subsequent period of time based thereon.

2. The method of claim 1 , wherein measuring, by the at least one sensor, the at least one parameter other than sound to generate the second data signal comprises at least one of:

measuring, by an accelerometer communicatively coupled to the processor, acceleration of a chest of the subject to generate an acceleration data signal that represents acceleration of the chest of the subject;

measuring, by a gyro sensor communicatively coupled to the processor, angular velocity of at least a portion of the subject to generate an angular velocity data signal that represents angular velocity of the at least the portion of the subject;

measuring, by a thermometer communicatively coupled to the processor, at least one of a skin temperature or a core body temperature of the subject to generate a temperature signal that represents at least one of the skin temperature or the core body temperature of the subject;

measuring, by an oxygen saturation sensor communicatively coupled to the processor, blood oxygenation of the subject to generate an oxygen saturation signal of the oxygen saturation level of the subject;

measuring, by a photoplethysmograph (PPG) sensor communicatively coupled to the processor, blood flow of the subject to generate a blood flow data signal that represents blood flow of the subject;

measuring, by an electrocardiograph (ECG) sensor communicatively coupled to the processor, electrical activity of a heart of the subject to generate an ECG data signal that represents electrical activity of the heart; or

measuring, by an electrodermal activity (EDA) sensor communicatively coupled to the processor, EDA of the subject to generate an EDA data signal that represents EDA of the subject.

3. The method of claim 1 , wherein the detecting further includes sub-classifying each audio feature of the first subset of the plurality of audio features and each feature of the first subset of the plurality of second features as indicative of a corresponding type of multiple different types of sound-producing behaviors.

4. The method of claim 3 , wherein the different types of sound-producing behaviors include at least one of sneezing, wheezing, shortness of breath, chewing, swallowing, masturbation, sex, coughing, vomiting, and shouting.

5. The method of claim 1 , further comprising reporting to a remote server information about sound-producing behaviors of the subject derived from one or both of the audio data signal or the second data signal.

6. The method of claim 5 , wherein the reporting to the remote server includes at least one of:

reporting to the remote server one or more of the plurality of audio features extracted from the audio data signal;

reporting to the remote server one or more of the plurality of second features extracted from the second data signal; or

reporting to the remote server one or more detected sound-producing behaviors and an occurrence time of each of the one or more detected sound-producing behaviors,

all without reporting to the remote server any of the audio data signal or any of the second data signal.

7. The method of claim 1 , further comprising:

receiving from the subject annotation input that confirms occurrence of one or more sound-producing behaviors; reporting, to a remote server, the annotation input, wherein the remote server is configured to receive annotation inputs and information about sound-producing behaviors from a plurality of subjects and to update an algorithm used in the classifying; and

receiving, from the remote server, the updated algorithm.