System and method of detecting falls of a subject using a wearable sensor

1. A method of performing physical activity monitoring and fall detection of a subject using a wearable sensor comprising at least one audio sensor and at least one of: an accelerometer, a gyroscope, and a magnetometer, the method comprising:

monitoring activities of the subject using the at least one of: the accelerometer, the gyroscope, and the magnetometer, and identifying at least one characteristic motion pattern based on the monitored activities;

detecting an apparent fall experienced by the subject based on the identified at least one characteristic motion pattern;

monitoring sounds provided by the at least one audio sensor following the detected apparent fall;

determining whether the detected apparent fall is an actual fall experienced by isolating sound signals produced by the subject and provided by the at least one audio sensor using a predetermined model tuned to an acoustic profile of the subject;

extracting segments corresponding to audible trauma-related cues from the isolated sound signals using a gated recurrent neural network with bi-directional long short-term memory (LSTM); and

determining that the detected apparent fall is an actual fall based on the extracted segments

communicating an indication of the actual fall to a monitoring system, enabling commencement of responsive action.

2. The method of claim 1 , wherein the audible trauma-related cues comprise one or more of abnormal breathing, sighing, moaning, groaning, grunting, crying and/or verbal expression of having fallen or being injured.

3. The method of claim 1 , further comprising:

when it is determined that the detected apparent fall is an actual fall, estimating an extent of pain sustained by the subject based on the audible trauma-related cues, and communicating an indication of the actual fall and the extent of pain to the monitoring system.

4. The method of claim 1 , wherein extracting segments corresponding to audible trauma-related cues from the isolated sound signals comprises:

extracting oral, linguistic, and paralinguistic cues from the isolated sound signals; and

estimating a level of pain of the subject based on the extracted oral, linguistic and paralinguistic cues, together with a determined heart rate of the subject,

wherein the detected apparent fall is determined to be an actual fall when the level of pain exceeds a predetermined threshold.

5. The method of claim 1 , further comprising:

determining at least one of a heart rate of the subject, a heart rate variability of the subject and a heart sound quality based on the monitored sounds provided by the at least one audio sensor, determining whether the detected apparent fall is an actual fall is based on the heart rate of the subject exceeding a predetermined heart rate baseline and/or the heart rate variability of the subject falling below a predetermined heart rate variability baseline.

6. The method of claim 5 , further comprising:

determining whether the subject is wearing the sensor based on at least one of the determined heart rate, the determined heart rate variability, and the determined heart sound quality.

7. The method of claim 6 , further comprising:

notifying the subject and/or the monitoring system when it is determined that the subject is not wearing the sensor to prompt the subject to wear the sensor.

8. The method of claim 6 , wherein the detected apparent fall is determined to be an actual fall when the determined heart rate increases and/or the determined heart rate variability decreases as compared to the predetermined baselines, respectively.

9. A sensor system, including a sensor wearable on a body of a subject, for monitoring physical activity of the subject, the system comprising:

an inertial measurement unit that monitors physical movements of the subject;

at least one audio sensor that detects ambient sounds around the subject;

a memory that stores executable instructions comprising a fall detection module, a post-fall sound acquisition module, a sound-driven fall detection confirmation module, a sound-driven pain level assessment module, and a sound segmentation module;

a processor that executes the instructions retrieved from the memory, wherein the instructions, when executed, cause the processor to:

detect an apparent fall experienced by the subject, based on the physical activities of the subject monitored by the inertial measurement unit, in accordance with the fall detection module;

acquire sounds provided by the at least one audio sensor following the detected apparent fall in accordance with the post-fall sound acquisition module;

isolate sound signals produced by the subject using a predetermined model tuned to an acoustic profile of the subject, and extract segments corresponding to audible trauma-related cues from the isolated sound signals using a gated recurrent neural network with bi-directional long short-term memory (LSTM), in accordance with the sound segmentation module;

determine that the detected apparent fall is an actual fall experienced by the subject based on the acquired sounds in accordance with the sound-driven fall detection confirmation module; and

estimate a level of pain of the subject based on the audible trauma-related cues produced by the subject and extracted from the acquired sounds in accordance with the sound-driven pain level assessment module; and

a communication interface that communicates an indication of the actual fall to a monitoring system over a wireless network, enabling commencement of responsive action.

10. The system of claim 9 , wherein the memory that stores executable instructions further comprises a heart sound detection module, and wherein the instructions, when executed, further cause the processor to:

determine at least one of a heart rate of the subject, a heart rate variability of the subject and a heart sound quality based on the acquired sounds in accordance with the heart sound detection module.

11. The system of claim 10 , wherein the memory that stores executable instructions further comprises a wear detection module, and wherein the instructions, when executed, further cause the processor to:

determine whether the sensor is being worn by the subject based on the physical activities of the subject monitored by the inertial measurement unit and the determined heart sound quality in accordance with the wear detection module.

12. A non-transitory machine-readable storage medium encoded with instructions for performing physical activity monitoring and fall detection of a subject using a wearable sensor comprising at least one audio sensor and at least one of an accelerometer, a gyroscope, and a magnetometer, wherein the instructions comprise:

instructions for monitoring activities of the subject using the at least one of: the accelerometer, the gyroscope, and the magnetometer, and identifying at least one characteristic motion pattern based on the monitored activities;

instructions for detecting an apparent fall experienced by the subject based on the identified at least one characteristic motion pattern;

instructions for monitoring sounds provided by the at least one audio sensor following the detected apparent fall;

instructions for determining whether the detected apparent fall is an actual fall experienced by isolating sound signals produced by the subject and provided by the at least one audio sensor using a predetermined model tuned to an acoustic profile of the subject;

instructions for extracting segments corresponding to audible trauma-related cues from the isolated sound signals using a gated recurrent neural network with bi-directional long short-term memory (LSTM); and

instructions for determining that the detected apparent fall is an actual fall based on the extracted segments;

instructions for communicating an indication of the actual fall to a monitoring system, enabling commencement of responsive action.

13. The non-transitory machine-readable storage medium of claim 12 , wherein the audible trauma-related cues comprise one or more of abnormal breathing, sighing, moaning, groaning, grunting, crying and/or verbal expression of having fallen or being injured.

14. The non-transitory machine-readable storage medium of claim 12 , further comprising instructions for when it is determined that the detected apparent fall is an actual fall, estimating an extent of pain sustained by the subject based on the audible trauma-related cues, and communicating an indication of the actual fall and the extent of pain to the monitoring system.

15. The non-transitory machine-readable storage medium of claim 12 , wherein instructions for extracting segments corresponding to audible trauma-related cues from the isolated sound signals comprises:

instructions for extracting oral, linguistic, and paralinguistic cues from the isolated sound signals; and

instructions for estimating a level of pain of the subject based on the extracted oral, linguistic and paralinguistic cues, together with a determined heart rate of the subject,

wherein the detected apparent fall is determined to be an actual fall when the level of pain exceeds a predetermined threshold.

16. The non-transitory machine-readable storage medium of claim 12 , further comprising instructions for determining at least one of a heart rate of the subject, a heart rate variability of the subject and a heart sound quality based on the monitored sounds provided by the at least one audio sensor, determining whether the detected apparent fall is an actual fall is based on the heart rate of the subject exceeding a predetermined heart rate baseline and/or the heart rate variability of the subject falling below a predetermined heart rate variability baseline.

17. The non-transitory machine-readable storage medium of claim 16 , further comprising instructions for determining whether the subject is wearing the sensor based on at least one of the determined heart rate, the determined heart rate variability, and the determined heart sound quality.

18. The non-transitory machine-readable storage medium of claim 17 , further comprising instructions for notifying the subject and/or the monitoring system when it is determined that the subject is not wearing the sensor to prompt the subject to wear the sensor.