System and method for camera-based heart rate tracking
A system and method for camera-based heart rate tracking. The method includes: determining bit values from a set of bitplanes in a captured image sequence that represent the HC changes; determining a facial blood flow data signal for each of a plurality of predetermined regions of interest (ROIs) of the subject captured by the images based on the HC changes; applying a band-pass filter of a passband approximating the heart rate to each of the blood flow data signals; applying a Hilbert transform to each of the blood flow data signals; adjusting the blood flow data signals from revolving phase-angles into linear phase segments; determining an instantaneous heart rate for each the blood flow data signals; applying a weighting to each of the instantaneous heart rates; and averaging the weighted instantaneous heart rates.
1. A method for camera-based heart rate tracking of a human subject, the method comprising:
receiving a captured image sequence of light re-emitted from the skin of the human subject;
determining, using a machine learning model trained with a hemoglobin concentration (HC) changes training set, bit values from a set of bitplanes in the captured image sequence that represent the HC changes of the subject, the set of bitplanes being those that are determined to approximately maximize a signal-to-noise ratio (SNR), the HC changes training set comprising bit values from each bitplane of images captured from a set of subjects for which heart rate is known;
determining a facial blood flow data signal for each of a plurality of predetermined regions of interest (ROIs) of the subject captured by the images based on the HC changes;
applying a band-pass filter of a passband approximating the heart rate to each of the blood flow data signals;
applying a Hilbert transform to each of the blood flow data signals;
adjusting the blood flow data signals from revolving phase-angles into linear phase segments;
determining an instantaneous heart rate for each the blood flow data signals;
applying a weighting to each of the instantaneous heart rates;
averaging the weighted instantaneous heart rates; and
outputting the average heart rate.
2. The method of claim 1 , wherein the ROIs are captured from the face of the subject.
3. The method of claim 1 , wherein the ROIs are captured from the wrist or the neck of the subject.
4. The method of claim 1 , wherein the ROIs are non-overlapping.
5. The method of claim 1 , wherein determining a set of bitplanes that maximize SNR comprises:
performing pixelwise image subtraction and addition of bitplane vectors to maximize signal differences in all ROIs over a predetermined time period;
identifying bit values from bitplanes that increase the signal differentiation and bit values from bitplanes that decrease the signal differentiation or do not contribute to signal differentiation; and
discarding the bit values from the bitplanes that decrease the signal differentiation or do not contribute to signal differentiation.
6. The method of claim 1 , wherein the machine learning model comprises a Long Short Term Memory (LSTM) neural network or a non-linear Support Vector Machine.
7. The method of claim 1 , wherein the passband is in a range of approximately 0.6 hertz to 1.2 hertz, where 60 heartbeats-per-minute is equivalent to 1 hertz.
8. The method of claim 1 , wherein determining the instantaneous heart rate for each the blood flow data signals comprises applying a differential filter to the linear phase segments to convert the phase-angle data into frequency units representing a count value, the count value for each of the ROIs represents the instantaneous heart rate.
9. The method of claim 1 , further comprising linearizing and differentiating the revolving phase-angles on a phase continuum scale to determine the instantaneous heart rate.
10. The method of claim 1 , wherein the weighting is integrated over an interval in the range of approximately one second to ten seconds.
11. The method of claim 9 , wherein the weighting is integrated over an interval of approximately five seconds.
12. A system for camera-based heart rate tracking of a human subject, the system comprising one or more processors and a data storage device, the one or more processors configured to execute:
a TOI module to receive a captured image sequence of light re-emitted from the skin of a human subject, the TOI module determines, using a machine learning model trained with a hemoglobin concentration (HC) changes training set, bit values from a set of bitplanes in the captured image sequence that represent the HC changes of the subject, the set of bitplanes being those that are determined to approximately maximize a signal-to-noise ratio (SNR), the HC changes training set comprising bit values from each bitplane of images captured from a set of subjects for which heart rate is known, the TOI module determines a facial blood flow data signal for each of a plurality of predetermined regions of interest (ROIs) of the subject captured by the images based on the HC changes;
a filtering module to apply a band-pass filter of a passband approximating the heart rate to each of the blood flow data signals;
a Hilbert transform module to apply a Hilbert transform to each of the blood flow data signals;
an adjustment module to adjust the blood flow data signals from revolving phase-angles into linear phase segments;
a derivative module to determine an instantaneous heart rate for each the blood flow data signals;
a weighting module to apply a weighting to each of the instantaneous heart rates;
a summation module to average the weighted instantaneous heart rates; and
an output module to output the average heart rate.
13. The system of claim 12 , wherein the ROIs are captured from the face of the subject.
14. The system of claim 12 , wherein the ROIs are non-overlapping.
15. The system of claim 12 , wherein the TOI module determines a set of bitplanes that maximize SNR by:
performing pixelwise image subtraction and addition of bitplane vectors to maximize signal differences in all ROIs over a predetermined time period;
identifying bit values from bitplanes that increase the signal differentiation and bit values from bitplanes that decrease the signal differentiation or do not contribute to signal differentiation; and
discarding the bit values from the bitplanes that decrease the signal differentiation or do not contribute to signal differentiation.
16. The system of claim 12 , wherein the passband is in a range of approximately 0.6 hertz to 1.2 hertz, where 60 heartbeats-per-minute is equivalent to 1 hertz.
17. The system of claim 12 , wherein the derivative module determines the instantaneous heart rate for each the blood flow data signals by applying a differential filter to the linear phase segments to convert the phase-angle data into frequency units representing a count value, the count value for each of the ROIs represents the instantaneous heart rate.
18. The system of claim 12 , wherein the derivative module linearizes and differentiates the revolving phase-angles on a phase continuum scale to determine the instantaneous heart rate.
19. The system of claim 12 , wherein the weighting applied by the weighting module is integrated over an interval in the range of approximately one second to ten seconds.
20. The system of claim 12 , wherein the weighting applied by the weighting module is integrated over an interval of approximately five seconds.