Respiratory function estimation from a 2D monocular video
View Patent ↗What is disclosed is a system and method for processing a video acquired using a 2D monocular video camera system to assess respiratory function of a subject of interest. In various embodiments hereof, respiration-related video signals are obtained from a temporal sequence of 3D surface maps that have been reconstructed based on an amount of distortion detected in a pattern placed over the subject's thoracic region (chest area) during video acquisition relative to known spatial characteristics of an undistorted reference pattern. Volume data and frequency information are obtained from the processed video signals to estimate chest volume and respiration rate. Other respiratory function estimations of the subject in the video can also be derived. The obtained estimations are communicated to a medical professional for assessment. The teachings hereof find their uses in settings where it is desirable to assess patient respiratory function in a non-contact, remote sensing environment.
1. A method for processing a video acquired using a 2D monocular video acquisition system to assess respiratory function of a subject of interest in a remote sensing environment, the method comprising:
receiving a video of a target region of a subject of interest being monitored for respiratory function assessment, said video having been acquired using a 2D monocular video acquisition system, said target region having a detectable pattern thereon, said pattern being one of: reflective and emissive in a given wavelength, said wavelength being detectable by sensor elements in a detector array of said camera during acquisition of said video;
processing said video to construct a temporal sequence of 3D surface maps of said patterned target region, said 3D surface maps being reconstructed by characterizing spatial attributes of a spatial distortion of said pattern over time relative to spatial characteristics of an undistorted reference pattern;
estimating time-series data from said temporal sequence of 3D surface maps, comprising:
for each image of said received video:
comparing said characterized spatial attributes of said pattern to known spatial attributes of said undistorted reference pattern to determine an amount of spatial distortion that has occurred locally at different locations on a surface of said patterned target region;
calculating a depth map from said determined amount of spatial distortion;
estimating a volume from said depth map; and
concatenating said estimated volumes to obtain said time-series data; and
estimating a respiratory function for said subject from said time-series data.
2. The method of claim 1 , wherein said target region comprises one of: said subject's anterior thoracic region, a region of said subject's dorsal body, and a side view containing said subject's thoracic region.
3. The method of claim 1 , wherein said undistorted reference pattern is virtual.
4. The method of claim 1 , wherein said respiratory function comprises any of: tidal chest volume, minute ventilation, respiration rate, inspiratory reserve volume, expiratory reserve volume, residual volume, vital capacity, inspiratory capacity, and functional residual capacity.
5. The method of claim 1 , further comprising:
de-trending said time-series data to remove low frequency variations;
performing a Fast Fourier Transform on said de-trended time-series data; and
performing peak detection to identify frequency components of said respiratory function over time.
6. The method of claim 1 , wherein said detectable pattern comprises any of: salient textural characteristics on said target region, salient textural characteristics of a clothing worn over said target region, a physical characteristic inherent to a surface of said target region, a physical characteristic imprinted on a surface of said target region, and a plurality of emissive sources on said target region at different locations.
7. The method of claim 1 , wherein said detectable pattern comprises any of: a pattern of marks on said target region, and a pattern of marks on clothing worn over said target region, said marks being any of: reflective and emissive.
8. The method of claim 1 , further comprising using said estimated respiratory function to monitor for an occurrence of any of: PUHD-I, PUHD-II, SIDS, Respiratory Failure, and Pulmonary Disease.
9. A system for processing a video acquired using a 2D monocular video camera system to assess respiratory function of a subject of interest in a remote sensing environment, the system comprising:
a memory and a storage device;
a processor in communication with said memory and storage device, and said 2D monocular video camera, said processor executing machine readable instructions for performing:
receiving a video of a target region of a subject of interest being monitored for respiratory function assessment, said video having been acquired using a 2D monocular video system, said target region having a detectable pattern thereon, said pattern being one of: reflective and emissive in a given wavelength, said wavelength being detectable by sensor elements in a detector array of said camera during acquisition of said video;
processing said video to construct a temporal sequence of 3D surface maps of said patterned target region, said 3D surface maps being reconstructed by characterizing spatial attributes of a spatial distortion of said pattern over time relative to spatial characteristics of an undistorted reference pattern;
estimating time-series data from said temporal sequence of 3D surface maps, comprising:
for each image of said received video:
comparing said characterized spatial attributes of said pattern to known spatial attributes of said undistorted reference pattern to determine an amount of spatial distortion that has occurred locally at different locations on a surface of said patterned target region;
calculating a depth map from said determined amount of spatial distortion;
estimating a volume from said depth map; and
concatenating said estimated volumes to obtain said time-series data;
estimating a respiratory function for said subject from said time-series data; and
communicating said estimated respiratory function to said storage device.
10. The system of claim 9 , wherein said target region comprises one of: said subject's anterior thoracic region, a region of said subject's dorsal body, and a side view containing said subject's thoracic region.
11. The system of claim 9 , wherein said undistorted reference pattern is virtual.
12. The system of claim 9 , wherein said respiratory function comprises any of: tidal chest volume, minute ventilation, respiration rate, inspiratory reserve volume, expiratory reserve volume, residual volume, vital capacity, inspiratory capacity, and functional residual capacity.
13. The system of claim 9 , further comprising:
de-trending said time-series data to remove low frequency variations;
performing a Fast Fourier Transform on said de-trended time-series data; and
performing peak detection to identify frequency components of said respiratory function over time.
14. The system of claim 9 , wherein said detectable pattern comprises any of: salient textural characteristics on said target region, salient textural characteristics of a clothing worn over said target region, a physical characteristic inherent to a surface of said target region, a physical characteristic imprinted on a surface of said target region, and a plurality of emissive sources on said target region at different locations.
15. The system of claim 9 , wherein said detectable pattern comprises any of: a pattern of marks on said target region, and a pattern of marks on clothing worn over said target region, said marks being any of: reflective and emissive.
16. The system of claim 9 , further comprising using said estimated respiratory function to monitor for an occurrence of any of: PUHD-I, PUHD-II, SIDS, Respiratory Failure, and Pulmonary Disease.
17. A method for processing an image acquired using a 2D monocular video acquisition system to assess a respiratory function of a subject of interest in a remote sensing environment, the method comprising:
receiving an image of a target region of a subject of interest being monitored for respiratory function assessment, said image having been acquired using a 2D monocular video acquisition system, said target region having a detectable pattern thereon, said pattern being one of: reflective and emissive in a given wavelength, said wavelength being detectable by sensor elements in a detector array of said camera during acquisition of said video;
processing said image to obtain a 3D surface map of said patterned target region, said 3D surface map being reconstructed by characterizing spatial attributes of a spatial distortion of said pattern relative to spatial characteristics of an undistorted reference pattern;
estimating time-series data from said 3D surface map, comprising:
for each image of said received video:
comparing said characterized spatial attributes of said pattern to known spatial attributes of said undistorted reference pattern to determine an amount of spatial distortion that has occurred locally at different locations on a surface of said patterned target region;
calculating a depth map from said determined amount of spatial distortion;
estimating a volume from said depth map; and
concatenating said estimated volumes to obtain said time-series data; and
assessing a respiratory function for said subject from said time-series data.
18. The method of claim 17 , wherein said target region comprises one of: said subject's anterior thoracic region, a region of said subject's dorsal body, and a side view containing said subject's thoracic region.
19. The method of claim 17 , wherein said undistorted reference pattern is virtual.
20. The method of claim 17 , wherein said detectable pattern comprises any of: salient textural characteristics on said target region, salient textural characteristics of a clothing worn over said target region, a physical characteristic inherent to a surface of said target region, a physical characteristic imprinted on a surface of said target region, and a plurality of emissive sources on said target region at different locations.
21. The method of claim 17 , wherein said detectable pattern comprises any of: a pattern of marks on said target region, and a pattern of marks on clothing worn over said target region, said marks being any of: reflective and emissive.