Generating a breathing alert
For generating a breathing alert, a method captures a video stream of a subject. The method further generates a vector time series that includes a vector for each image frame of the video stream. The method estimates of breathing signal from the vector time series. The method further determines one of large-scale motion and a breathing event of the subject based on the breathing signal. The method generates an alert if no breathing event is identified and no large-scale motion of the subject is identified within an event time interval.
1. A method comprising:
receiving, by a processing apparatus, a video stream of a subject;
generating, by the processing apparatus, a vector time series that comprises a vector for each image frame of the video stream;
estimating, by the processing apparatus, a breathing signal from the vector time series;
determining, by the processing apparatus, one of a large-scale motion and a breathing event of the subject based on the breathing signal; and
generating, by the processing apparatus, an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval,
wherein estimating the breathing signal comprises:
decomposing a matrix X of a windowed video subsequence of the video stream into a sparse matrix S representing moving objects and a low rank matrix L representing non-moving objects;
reconstructing a video stream of the moving objects from the sparse matrix S, wherein each pixel of the reconstructed video stream of the moving objects comprises one of a scalar time series and a vector time series; and
measuring breathing events in the reconstructed video stream.
2. The method of claim 1 , further comprising capturing the video stream using a bandpass filter in the range of 0.8-2.5 micrometers and with a Charge Coupled Device (CCD) tuned to 1.5 micrometers.
3. The method of claim 1 , wherein generating the vector time series comprises dividing each image frame into regions and summing pixel values in each region as a pixel sum.
4. The method of claim 1 , wherein generating the vector time series comprises dividing each image frame into regions and removing a signal mean in each region.
5. The method of claim 1 , the method further comprising:
generating a breathing report based at least in part on breathing events determined based on the breathing signal, wherein the breathing report comprises one or more of: a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, an inter-breath interval histogram, and apnea event data; and
generating a motion report based at least in part on large-scale motions determined based on the breathing signal, wherein the motion report comprises one or more of: a motion frequency, a motion magnitude, a motion duration, a sleep length, a sleep quality, and sleep intervals.
6. The method of claim 1 , wherein estimating the breathing signal comprises applying an adaptive subspace algorithm to each vector of the vector time series.
7. The method of claim 1 , wherein estimating the breathing signal further comprises:
defining a cursor t that indexes the image frames I of the video stream;
defining a window length N;
extracting the windowed video subsequence of the video stream comprising N image frames I(t−N+1), I(t−N+2), I(t−N+3), . . . , I(t−2), I(t−1), I(t); and
organizing the windowed video subsequence into the matrix, X, having M rows and N columns, where M is the product of a height, a width and a number of channels in the image frames of the video stream.
8. The method of claim 7 , the method further comprising:
calculating a normalized auto-correlation for each pixel of the reconstructed video stream of moving objects; and
identifying a breathing event in response to an auto-correlation peak for a given pixel of the reconstructed video stream of moving objects persisting over a specified number of consecutive time periods.
9. The method of claim 7 , wherein the matrix X is decomposed by iteratively approximating a decomposition algorithm by:
initializing S=0;
calculating L=mean (X−S), wherein X=L+S+N and N is additive noise;
calculating
S{i,j}={X{i,j}−L{i,j }, if . . . X{i,j}−L{i,j}·<T 0,otherwise},
wherein S{i,j} denotes the element of the matrix S on the ith row and the jth column, and T is a sparsity threshold; and
repeating the calculating steps until a stopping condition is satisfied.
10. The method of claim 1 , the method further comprises:
encoding a breath rate from a plurality of breathing events as a heat map;
overlaying the heat map on the video stream for display to a user.
11. The method of claim 1 , the method further comprises:
encoding large-scale motion of a subject as a heat map;
overlaying the heat map on the video stream for display to a user.
12. An apparatus comprising:
one or more memory devices; and
one or more processing devices, in communication with the one or more memory devices, and configured to:
receive a video stream of a subject via a camera;
generate a vector time series that comprises a vector for each image frame of the video stream, estimate a breathing signal from the vector time series, and determine one of a large-scale motion and a breathing event of the subject based on the breathing signal; and
generate an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval,
wherein estimating the breathing signal comprises:
decomposing a matrix X of a windowed video subsequence of the video stream into a sparse matrix S representing moving objects and a low rank matrix L representing non-moving objects;
reconstructing a video stream of the moving objects from the sparse matrix S, wherein each pixel of the reconstructed video stream of the moving objects comprises one of a scalar time series and a vector time series; and
measuring breathing events in the reconstructed video stream.
13. The apparatus of claim 12 , further comprising a camera that comprises a bandpass filter in the range of 0.8-2.5 micrometers and a Charge Coupled Device (CCD) tuned to 1.5 micrometers.
14. The apparatus of claim 12 , wherein generating the vector time series comprises dividing each image frame into regions and summing pixel values in each region as a pixel sum.
15. The apparatus of claim 12 , wherein generating the vector time series comprises dividing each image frame into regions and removing a signal mean in each region.
16. The apparatus of claim 12 , wherein the one or more processing devices are further configured to:
generate a breathing report based at least in part on breathing events determined based on the breathing signal, wherein the breathing report comprises one or more of: a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, an inter-breath interval histogram, and apnea event data; and
generate a motion report based at least in part on large-scale motions determined based on the breathing signal, wherein the motion report comprises one or more of: a motion frequency, a motion magnitude, a motion duration, a sleep length, a sleep quality, and sleep intervals.
17. The apparatus of claim 12 , wherein estimating the breathing signal comprises applying an adaptive subspace algorithm to each vector of the vector time series.
18. The apparatus of claim 12 , wherein estimating the breathing signal further comprises:
defining a cursor t that indexes the image frames I of the video stream;
defining a window length N;
extracting the windowed video subsequence of the video stream comprising N image frames I(t−N+1), I(t−N+2), I(t−N+3), . . . , I(t−2), I(t−1), I(t);
organizing the windowed video subsequence into the matrix, X, having M rows and N columns, where M is the product of a height, a width and a number of channels in the image frames of the video stream.
19. The apparatus of claim 17 , wherein the one or more processing devices are further configured to:
calculate a normalized auto-correlation for each pixel of the reconstructed video stream of moving objects; and
identify a breathing event in response to an auto-correlation peak for a given pixel of the reconstructed video stream of moving objects persisting over a specified number of consecutive time periods.
20. The apparatus of claim 17 , wherein the matrix X is decomposed by iteratively approximating a decomposition algorithm by:
initializing S=0;
calculating L=mean (X−S), wherein X=L+S+N and N is additive noise;
calculating
S{i,j}={X{i,j}−L{i,j }, if . . . X{i,j}−L{i,j}·<T 0,otherwise},
wherein S{i,j} denotes the element of the matrix S on the ith row and the jth column, and T is a sparsity threshold; and
repeating the calculating steps until a stopping condition is satisfied.
21. The method of claim 1 , further comprising:
calculating breathing data comprising one or more of: a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, and an inter-breath interval histogram; and
identifying a sleep apnea event based on the breathing data.