Continuous cardiac pulse rate estimation from multi-channel source video data with mid-point stitching
View Patent ↗What is disclosed is a novel system and method for extracting photoplethysmographic (PPG) signals (i.e., cardiac signals) on a continuous basis from signals generated from video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment. In one embodiment, a time-series signal is received. The time-series signal is generated from video images captured of a region of exposed skin where a PPG signal of a subject of interest can be registered. The time-series signal is then divided into batches for processing, with successive batches having at least a 95% overlap with a previous batch. Each of the batches of time-series signals is processed to obtain a PPG signal from each batch. A mid-point of each of these PPG-signals is stitched together to obtain a continuous PPG signal for the subject. The continuous PPG signal for the subject can then viewed on a display device.
1. A method for extracting photoplethysmographic (PPG) signals (i.e., cardiac signals) on a continuous basis from signals generated from batches of video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment, the method comprising:
receiving a time-series signal generated from video images captured of a region of exposed skin where a photoplethysmographic (PPG) signal of a subject of interest can be registered;
dividing said time-series signal into batches for processing, with successive batches having at least a 95% overlap with a previous batch;
processing each of said batches of time-series signals to obtain a PPG signal from each batch; and
stitching together said PPG signal from each batch to form a continuous PPG signal, said stitching comprising using mid-point stitching.
2. The method of claim 1 , wherein said video images are captured using a video imaging device capturing any combination of: NIR images, RGB images, RGB with NIR images, multispectral images, and hyperspectral video images.
3. The method of claim 1 , wherein, in advance of obtaining said time-series signal, pre-processing said video to compensate for any of: a motion induced blur, an imaging blur, and slow illuminant variation.
4. The method of claim 1 , further comprising:
using said continuous PPG signal for detecting peak-to-peak pulse points in said continuous cardiac signal;
analyzing said pulse points to obtain peak-to-peak pulse dynamics; and
determining an occurrence of a cardiac arrhythmia based on said pulse dynamics.
5. The method of claim 1 , wherein said PPG signal is filtered using a moving average comprising:
y
(
n
)
=
1
N
∑
1
N
x
(
n
-
i
)
where N is the number of frames in a moving window of said video, x is an unfiltered PPG signal, y is a filtered PPG signal, n is a current frame and i is an index designating a moving frame.
6. The method of claim 1 , wherein said video is captured over two imaging channels, further comprising using a constrained source separation algorithm with a reference signal that has a frequency range which approximates a frequency range of said subject's cardiac pulse.
7. The method of claim 4 , wherein said peak-to-peak pulse points are detected in said continuous PPG signal using an adaptive threshold technique with successive thresholds being based on variations detected in previous magnitudes of said pulse peaks.
8. The method of claim 7 , further comprising using a Poincare diagram of said peak-to-peak pulse dynamics, said Poincare diagram showing a relationship between consecutive beats.
9. The method of claim 7 , further comprising normalizing said peak-to-peak pulse points to a frequency of 70 bpm to reduce pulse variations.
10. The method of claim 7 , further comprising determining whether a time interval between consecutive peaks in said signal is outside an acceptable limit for said subject.
11. The method of claim 7 , further comprising comparing said peak-to-peak pulse dynamics across different patients.
12. The method of claim 7 , further comprising communicating said peak-to-peak pulse dynamics to a display device.
13. The method of claim 1 , wherein said time-series signal comprises one of: stored values, and values generated from a streaming video.
14. A system for extracting photoplethysmographic (PPG) signals (i.e., cardiac signals) on a continuous basis from signals generated from batches of video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment, the system comprising:
a memory; and
a processor in communication with a memory, said processor executing machine readable instructions for performing:
receiving a time-series signal generated from video images captured of a region of exposed skin where a photoplethysmographic (PPG) signal of a subject of interest can be registered;
dividing said time-series signal into batches for processing, with successive batches having at least a 95% overlap with a previous batch;
processing each of said batches of time-series signals to obtain a PPG signal from each batch; and
stitching together said PPG signal from each batch to form a continuous PPG signal, said stitching comprising using mid-point stitching.
15. The system of claim 14 , wherein said video images are captured using a video imaging device capturing any combination of: NIR images, RGB images, RGB with NIR images, multispectral images, and hyperspectral video images.
16. The system of claim 14 , wherein, in advance of obtaining said time-series signal, pre-processing said video to compensate for any of: a motion induced blur, an imaging blur, and slow illuminant variation.
17. The system of claim 14 , further comprising:
using said continuous PPG signal for detecting peak-to-peak pulse points in said continuous cardiac signal;
analyzing said pulse points to obtain peak-to-peak pulse dynamics; and
determining an occurrence of a cardiac arrhythmia based on said pulse dynamics.
18. The system of claim 14 , wherein said PPG signal is filtered using a moving average comprising:
y
(
n
)
=
1
N
∑
1
N
x
(
n
-
i
)
where N is the number of frames in a moving window of said video, x is an unfiltered PPG signal, y is a filtered PPG signal, n is a current frame and i is an index designating a moving frame.
19. The system of claim 14 , wherein said video is captured over two imaging channels, further comprising using a constrained source separation algorithm with a reference signal that has a frequency range which approximates a frequency range of said subject's cardiac pulse.
20. The system of claim 17 , wherein said peak-to-peak pulse points are detected in said continuous PPG signal using an adaptive threshold technique with successive thresholds being based on variations detected in previous magnitudes of said pulse peaks.
21. The system of claim 20 , further comprising using a Poincare diagram of said peak-to-peak pulse dynamics, said Poincare diagram showing a relationship between consecutive beats.
22. The system of claim 20 , further comprising determining whether a time interval between consecutive peaks in said signal is outside an acceptable limit for said subject.
23. The system of claim 20 , further comprising comparing said peak-to-peak pulse dynamics across different patients.
24. The system of claim 20 , further comprising normalizing said peak-to-peak pulse points to a frequency of 70 bpm to reduce pulse variations.
25. The system of claim 14 , wherein said time-series signal comprises one of: stored values, and values generated from a streaming video.