Determining cardiovascular features using camera-based sensing
In one embodiment, a computer-readable non-transitory storage medium embodies software that is operable when executed to, in real time, capture a number of images of a user; and determine a time-series signal for the user based on the plurality of images. The signal includes one or more segments that are physiologically plausible and one or more segments that are physiologically implausible. The software is further operable to identify one or more of the physiologically plausible sub-segments based on one or more pre-defined signal characteristics; and calculate one or more heartrate measurements based on the physiologically plausible sub-segments.
1. One or more computer-readable non-transitory storage media embodying software that is operable when executed by a client system to, in real-time:
capture, by one or more cameras of the client system, a plurality of images of a user;
determine, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identify, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculate, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.
2. The media of claim 1 , wherein the plurality of images is captured by a single camera.
3. The media of claim 1 , wherein the software is further operable to:
determine a baseline signal for the user;
calculate a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
compare the SNR to a threshold SNR; and
identify sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.
4. The media of claim 1 , wherein the software is further operable to:
compare the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identify sub-segments of the signal that are consistent with the respiratory profile.
5. The media of claim 1 , wherein the software is further operable to:
perform a Fourier transform of the time-series signal; and
identify sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.
6. The media of claim 1 , wherein the software is further operable to identify sub-segments for which a corresponding systolic portion of the signal is less than a corresponding diastolic portion.
7. The media of claim 1 , wherein the software is further operable to:
extract red, green, and blue channel components of the time-series signal;
compare the red, green, and blue channel components with respective components of a pre-determined RGB profile, wherein the pre-determined RGB profile comprises a pulsatile relationship, total power relationship, or co-variation relationship between the red, green, and blue channel components; and
identify sub-segments of the signal that are consistent with the pre-determined RGB profile.
8. A method executed by a client system comprising, in real-time:
capturing, by one or more cameras of the client system, a plurality of images of a user;
determining, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identifying, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculating, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.
9. The method of claim 8 , wherein the plurality of images is captured by a single camera.
10. The method of claim 8 , further comprising:
determining a baseline signal for the user;
calculating a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
comparing the SNR to a threshold SNR; and
identifying sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.
11. The method of claim 8 , further comprising:
comparing the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identifying sub-segments of the signal that are consistent with the respiratory profile.
12. The method of claim 8 , further comprising:
performing a Fourier transform of the time-series signal; and
identifying sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.
13. The method of claim 8 , further comprising identifying sub-segments for which a corresponding systolic portion of the signal is less than a corresponding diastolic portion.
14. The method of claim 8 , further comprising:
extracting red, green, and blue channel components of the time-series signal;
comparing the red, green, and blue channel components with respective components of a pre-determined RGB profile, wherein the pre-determined RGB profile comprises a pulsatile relationship, total power relationship, or co-variation relationship between the red, green, and blue channel components; and
identifying sub-segments of the signal that are consistent with the pre-determined RGB profile.
15. A system comprising:
one or more processors; and
a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions by a client system to, in real-time:
capture, by one or more cameras of the client system, a plurality of images of a user;
determine, by the client system, a time-series signal for the user based on the plurality of images, wherein the signal comprises one or more physiologically plausible sub-segments and one or more physiologically implausible sub-segments;
identify, by the client system, one or more of the physiologically plausible sub-segments and one or more of the physiologically implausible sub-segments based on one or more pre-defined, physiological signal characteristics; and
calculate, by the client system, an average heartrate or heartrate range for the user based on the one or more physiologically plausible sub-segments.
16. The system of claim 15 , wherein the processors are further operable to:
determine a baseline signal for the user;
calculate a signal-to-noise ratio (SNR) of the time-series signal based on the baseline signal;
compare the SNR to a threshold SNR; and
identify sub-segments of the signal with a calculated SNR that is higher than the threshold SNR.
17. The system of claim 15 , wherein the processors are further operable to:
compare the time-series signal to a pre-determined respiratory profile, wherein the pre-determined respiratory profile comprises characteristics of a pulse volume measurement relative to R wave-to-R wave (RR) intervals during exhalation or during inhalation; and
identify sub-segments of the signal that are consistent with the respiratory profile.
18. The system of claim 15 , wherein the processors are further operable to:
perform a Fourier transform of the time-series signal; and
identify sub-segments of the time-series signal based on a measured RR interval being consistent with a dominant frequency of the Fourier transform.
19. The system of claim 15 , wherein the processors are further operable to identify sub-segments for which a corresponding systolic portion of the signal is less than a corresponding diastolic portion.
20. The system of claim 15 , wherein the processors are further operable to:
extract red, green, and blue channel components of the time-series signal;
compare the red, green, and blue channel components with respective components of a pre-determined RGB profile, wherein the pre-determined RGB profile comprises a pulsatile relationship, total power relationship, or co-variation relationship between the red, green, and blue channel components; and
identify sub-segments of the signal that are consistent with the pre-determined RGB profile.