Pulse detection from head motions in video
Heart rates and beat lengths of a subject can be extracted from videos of the subject by measuring subtle head motion caused by the Newtonian reaction to the influx of blood at each beat. Embodiments track features on the video images of the subject's head and perform principal component analysis (PCA) to decompose the feature location-time series into a set of component motions. The method or system then selects a component that best corresponds to heartbeats based on its temporal frequency spectrum. Finally, the motion projected to this component is analyzed and peaks of the location-time series are identified, which correspond to heartbeats. Pulse rate measurements or heart rate measurements of the subject are output.
1. A computer-implemented method of monitoring vital signs comprising:
in a digital processor:
receiving an input video of an upper body portion of a subject, the input video being formed of a sequence of frames, each frame: (i) providing a respective image of the upper body portion, (ii) having spatial locations corresponding to feature points in the image, and (iii) representing a data point in a position-time series of the sequence of frames, each location being a pixel representing visible light spectrum;
forming a location-time series returning the spatial locations of each feature point across plural frames of the input video; and
from the formed location-time series and the position-time series, detecting a pulse signal and extracting pulse rate measurements, said extracting including output of a pulse rate or a heart rate of the subject shown in the video.
2. The method of claim 1 , wherein the upper body portion is a head of the subject, and the input video captures movement of the head.
3. The method of claim 2 , wherein for each frame, the data point represents position of the head in the position-time series.
4. The method of claim 2 , wherein detecting the pulse signal includes:
isolating from the formed location-time series motion of the head corresponding to pulse, said isolating resulting in a component of the location-time series of the feature points that correspond to pulse; and
projecting the resulting component onto a one-dimensional signal in a manner that allows extraction of individual beat boundaries.
5. The method of claim 2 , wherein each frame represents a multidimensional position of the head; and
wherein detecting the pulse signal: (i) utilizes principal component analysis on the formed location-time series and finds a set of main dimensions along which the position of the head varies; and (ii) selects a dimension on which to project the position-time series to obtain the pulse signal.
6. The method of claim 2 , wherein the detecting includes:
temporally filtering the formed location-time series;
identifying component of motion of the head corresponding to pulse using principal component analysis on the temporally filtered location-time series; and
identifying individual beats of the pulse signal by extracting peaks of the temporally filtered location-time series.
7. The method of claim 6 , wherein temporally filtering the formed location-time series removes frequencies that are outside a predetermined range of pulse rates.
8. The method of claim 6 , wherein performing principal component analysis includes decomposing the filtered location-time series into a set of independent source signals.
9. The method of claim 1 , wherein detecting the pulse signal includes: (a) extracting peaks of the formed location-time series, and (b) from the extracted peaks, identifying individual beats and calculating beat duration.
10. The method of claim 1 , wherein the detecting and extracting further results in output of heart beat sequences of the subject.
11. A system monitoring vital signs comprising:
a computer processor coupled to receive an input video of an upper body portion of a subject, the input video being formed of a sequence of frames, each frame: (i) providing a respective image of the upper body portion, (ii) having spatial locations corresponding to feature points in the image, and (iii) representing a data point in a position-time series of the sequence of frames, each location being a pixel representing visible light spectrum;
a tracking module executable by the processor and configured to form a location-time series returning the spatial locations of each feature point across plural frames of the input video; and
an analyzer implemented by the processor and configured to responsively detect a pulse signal and extract pulse rate measurements from the formed location-time series and the position-time series, the analyzer outputting a pulse rate or a heart rate of the subject in the input video.
12. The system of claim 11 , wherein the upper body portion is a head of the subject, and the input video captures movement of the head.
13. The system of claim 12 , wherein for each frame, the data point represents position of the head in the position-time series.
14. The system of claim 12 , wherein the analyzer detecting the pulse signal includes the analyzer being configured to:
isolate from the formed location-time series motion of the head corresponding to pulse, said isolating resulting in a component of the location-time series of the feature points that correspond to pulse; and
project the resulting component onto a one-dimensional signal in a manner that allows extraction of individual beat boundaries.
15. The system of claim 12 , wherein each frame represents a multi-dimensional position of the head; and
wherein the analyzer: (i) utilizes principal component analysis on the formed location-time series to find a set of main dimensions along which the position of the head varies; and (ii) selects a dimension on which to project the position-time series to obtain the pulse signal.
16. The system of claim 12 , wherein the analyzer is further configured to:
temporally filter the formed location-time series;
identify component of motion of the head corresponding to pulse using principal component analysis on the temporally filtered location-time series; and
identify individual beats of the pulse signal by extracting peaks of the temporally filtered location-time series.
17. The system of claim 16 , wherein the analyzer is further configured to filter the formed location-time series temporally by removing frequencies that are outside a predetermined range of pulse rates.
18. The system of claim 16 , wherein the analyzer is further configured to perform principal component analysis by decomposing the filtered location-time series into a set of independent source signals.
19. The system of claim 11 , wherein the analyzer detecting and extracting further results in output of heartbeat sequences of the subject.
20. A non-transitory computer readable medium comprising:
a memory area having program code stored thereon; and
the program code including instructions which when loaded and executed by a processor cause the processor to:
receive an input video of at least a head of a subject, the input video being formed of a sequence of frames, each frame: (i) providing a respective image of the subject, (ii) having spatial locations corresponding to feature points in the image, and (iii) representing a data point in a position-time series of the sequence of frames, each location being a pixel representing visible light spectrum;
form a location-time series returning the spatial locations of each feature point across plural frames of the input video; and
from the formed location-time series and the position-time series, detect a pulse signal and extract pulse rate measurements, said extracting including output of a pulse rate or a heart rate of the subject shown in the video.