IP Library › Granted Patent US 12,555,225
Granted Patent B2
US 12,555,225 · App. 17/878,590 · Granted Feb 17, 2026

Method and apparatus for monitoring of a human or animal subject

Inventors: Nicholas Dunkley Hutchinson (Oxford, GB); Simon Mark Chave Jones (Oxford, GB)
Assignee: OXEHEALTH LIMITED
G06T7/0012A61B5/0077A61B5/02416A61B5/0816A61B5/1128A61B5/746G06T7/20G06T7/70A61B2503/40G06T2207/10016G06T2207/20081G06T2207/30048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,555,225
App. No.
17/878,590
Granted
Feb 17, 2026
Kind
B2
Abstract

A method and apparatus for monitoring a human or animal subject in a room using video imaging of the subject and analysis of the video image to detect and quantify movement of the subject and to derive an estimate of vital signs such as heart rate or breathing rate. The method includes techniques for de-correlating global intensity variations such as sunlight changes, compensating for noise, eliminating areas not of interest in the image, and quickly and automatically finding regions of interest for detecting subject movement and estimating vital signs. A logic machine is used for interpreting detected movement of the subject, and an artificial neural network is used to calculate a confidence measure for the vital signs estimates from signal quality indices. The confidence measure may be used with a normal density filter to output estimates of the vital signs.

Claims (27)

1 . A method of monitoring a human or animal subject comprising the steps of:

capturing a video image of the subject consisting of a time series of image frames each image frame comprising a pixel array of image intensity values; and

analysing the video image to determine automatically a vital sign estimate of one or more vital signs of the subject,

wherein the step of determining automatically a vital sign estimate of one or more vital signs of the subject comprises:

analysing the video image to detect a plurality of signals each comprising the variation in image intensity values with time at each of a respective plurality of positions in the video image;

determining a first plurality of signal quality indices of the signals and retaining only those signals whose signal quality indices are above a predetermined threshold;

analysing the retained signals in a multi-dimensional component analysis to obtain components thereof and retaining a predetermined number of the strongest components;

determining a second plurality of signal quality indices of the retained components;

determining from said second plurality of signal quality indices a confidence value for each of said retained components;

selecting amongst the retained components using the confidence values, wherein the selecting comprises weighting said retained components by the respective confidence values, and updating a prior estimate of said one or more vital signs by said weighted components;

determining the frequency of each of the selected components; and

outputting the vital sign estimate based on said determined frequencies.

2 . A method according to claim 1 wherein each of said plurality of signals comprises the variation in intensity at a plurality of pixels in a local neighbourhood in each of said image frames whose image intensity values are combined together to form one of said plurality of signals.

3 . A method according to claim 1 wherein said positions are positions in the video image at which subject movement has been detected.

4 . A method according to claim 1 further comprising the step of frequency filtering each of said plurality of signals to exclude those outside predetermined expected physiological range for said one or more vital signs.

5 . A method according to claim 1 further comprising the step of calculating the logarithm of said image intensity values to form a logarithm image and said step of analysing the video image is performed on said logarithm image.

6 . A method according to claim 1 wherein said first plurality of signal quality indices comprise measures of periodicity, for example one or more of peak consistency and amplitude consistency.

7 . A method according to claim 1 wherein said step of multi-dimensional component analysis comprises decomposing the retained signals into their components, for example by one of principal component analysis or independent component analysis.

8 . A method according to claim 1 wherein said second plurality of signal quality indices comprise measures of one or more of: periodicity, spatial distribution within the video image, uniformity and variability of the component, amount of movement associated with the component.

9 . A method according to claim 1 further comprising the step of down-weighting said prior estimate of said one or more vital signs by a predetermined amount before updating said prior estimate with said weighted components.

10 . A method according to claim 9 further comprising the step of detecting the amount of subject movement in the image and varying said predetermined amount in dependence upon the detected amount of subject movement.

11 . A method according to claim 9 wherein the amount of subject movement is detected by determining the amount of variation in image intensity with time at each of said respective plurality of positions in the video image.

12 . A method according to claim 1 wherein said step of determining from said second plurality of signal quality indices a confidence value for each of said retained components is performed using a machine learning technique.

13 . A system for monitoring a human or animal subject in accordance with the method of claim 1 , comprising:

a video camera adapted to capture a video image of the subject;

a display;

a video image processing unit adapted to process the video image in accordance with the method of claim 1 .

Priority Claims (1)
GB 1900032 · Jan 2, 2019 · national
Continuity (2)
Continuation 16732769 · Jan 2, 2020
Related Publication 20230028571A1 · Jan 26, 2023
References Cited (162)
US 8855384B2 · Kyal et al. · 2014 [cited by applicant]
US 8965090B1 · Khachaturian et al. · 2015 [cited by applicant]
US 9036877B2 · Kyal et al. · 2015 [cited by applicant]
US 10034979B2 · Bechtel et al. · 2018 [cited by applicant]
US 10292662B2 · Kirenko · 2019 [cited by applicant]
US 11403754B2 · Hutchinson · 2022 [cited by examiner]
US 20020106709A1 · Potts et al. · 2002 [cited by applicant]
US 20020180870A1 · Chen · 2002 [cited by applicant]
US 20030138149A1 · Iizuka et al. · 2003 [cited by applicant]
US 20030228032A1 · Rui et al. · 2003 [cited by applicant]
US 20050197590A1 · Osorio et al. · 2005 [cited by applicant]
US 20060058618A1 · Nishiura · 2006 [cited by applicant]
US 20070156060A1 · Cervantes · 2007 [cited by applicant]
US 20070195931A1 · Ohishi · 2007 [cited by applicant]
US 20080292151A1 · Kurtz et al. · 2008 [cited by applicant]
US 20090216499A1 · Tobola et al. · 2009 [cited by applicant]
US 20100049064A1 · Bodmer et al. · 2010 [cited by applicant]
US 20100074475A1 · Chouno · 2010 [cited by applicant]
US 20100298656A1 · McCombie et al. · 2010 [cited by applicant]
US 20110046498A1 · Klap et al. · 2011 [cited by applicant]
US 20110150274A1 · Patwardhan et al. · 2011 [cited by applicant]
US 20110251493A1 · Poh et al. · 2011 [cited by applicant]
US 20110311143A1 · Cennini et al. · 2011 [cited by applicant]
US 20120141000A1 · Jeanne et al. · 2012 [cited by applicant]
US 20120213405A1 · Sasaki · 2012 [cited by applicant]
US 20120242819A1 · Schamp · 2012 [cited by applicant]
US 20130138009A1 · Nierenberg et al. · 2013 [cited by applicant]
US 20130324875A1 · Mestha et al. · 2013 [cited by applicant]
US 20140003690A1 · Razeto et al. · 2014 [cited by applicant]
US 20140023235A1 · Cennini et al. · 2014 [cited by applicant]
US 20140037163A1 · Kirenko et al. · 2014 [cited by applicant]
US 20140037166A1 · De Haan et al. · 2014 [cited by applicant]
US 20140236036A1 · de Haan et al. · 2014 [cited by applicant]
US 20140276099A1 · Kirenko et al. · 2014 [cited by applicant]
US 20140276104A1 · Tao et al. · 2014 [cited by applicant]
US 20140334697A1 · Kersten et al. · 2014 [cited by applicant]
US 20140371599A1 · Wu et al. · 2014 [cited by applicant]
US 20140371635A1 · Shinar et al. · 2014 [cited by applicant]
US 20140378842A1 · Xu et al. · 2014 [cited by applicant]
US 20150005646A1 · Balakrishnan et al. · 2015 [cited by applicant]
US 20150063708A1 · Sripadarao et al. · 2015 [cited by applicant]
US 20150104088A1 · Kirenko et al. · 2015 [cited by applicant]
US 20150148687A1 · Kitajima et al. · 2015 [cited by applicant]
US 20150208987A1 · Shan et al. · 2015 [cited by applicant]
US 20150221069A1 · Shaburova et al. · 2015 [cited by applicant]
US 20150250391A1 · Kyal et al. · 2015 [cited by applicant]
US 20150363361A1 · Kniazev · 2015 [cited by applicant]
US 20160106340A1 · Mestha et al. · 2016 [cited by applicant]
US 20160125260A1 · Huang et al. · 2016 [cited by applicant]
US 20160132732A1 · Gunther et al. · 2016 [cited by applicant]
US 20160210747A1 · Hay et al. · 2016 [cited by applicant]
US 20160220128A1 · Den Brinker et al. · 2016 [cited by applicant]
US 20160253820A1 · Jeanne et al. · 2016 [cited by applicant]
US 20160310067A1 · Heinrich et al. · 2016 [cited by applicant]
US 20170007185A1 · Lin et al. · 2017 [cited by applicant]
US 20170042432A1 · Adib et al. · 2017 [cited by applicant]
US 20170224256A1 · Kirenko · 2017 [cited by applicant]
US 20170238805A1 · Addison et al. · 2017 [cited by applicant]
US 20170238842A1 · Jacquel et al. · 2017 [cited by applicant]
US 20180085010A1 · Jones et al. · 2018 [cited by applicant]
US 20180279885A1 · Bulut · 2018 [cited by applicant]
US 20190000391A1 · De Haan et al. · 2019 [cited by applicant]
US 20190029543A1 · Hutchinson · 2019 [cited by examiner]
US 20190267040A1 · Ikeda et al. · 2019 [cited by applicant]
EP 0615245A2 · 1994 [cited by applicant]
EP 0919184A1 · 1999 [cited by applicant]
EP 1571594A2 · 2005 [cited by applicant]
EP 2767233A1 · 2014 [cited by applicant]
EP 2976998A1 · 2016 [cited by applicant]
EP 2988274A2 · 2016 [cited by applicant]
EP 3073905A1 · 2016 [cited by applicant]
EP 3207862A1 · 2017 [cited by applicant]
JP 2011130996A · 2011 [cited by applicant]
KR 100812996B1 · 2008 [cited by applicant]
WO WO2010100593A1 · 2010 [cited by applicant]
WO WO2010115939A2 · 2010 [cited by applicant]
WO WO2011021128A2 · 2011 [cited by applicant]
WO WO2013027027A2 · 2013 [cited by applicant]
WO WO2014125250A1 · 2014 [cited by applicant]
WO WO2014131850A1 · 2014 [cited by applicant]
WO WO2014140994A1 · 2014 [cited by applicant]
WO WO201504915A1 · 2015 [cited by applicant]
WO WO2015049150A1 · 2015 [cited by applicant]
WO WO2015055709A1 · 2015 [cited by applicant]
WO WO2015078735A1 · 2015 [cited by applicant]
WO WO2015091582A1 · 2015 [cited by applicant]
WO WO2015172735A1 · 2015 [cited by applicant]
WO WO2016092290A1 · 2016 [cited by applicant]
WO WO2016094749A1 · 2016 [cited by applicant]
WO WO2016159151A1 · 2016 [cited by applicant]
WO WO2017125743A1 · 2017 [cited by applicant]
WO WO2017125744A1 · 2017 [cited by applicant]
WO WO2017125763A1 · 2017 [cited by applicant]
WO 2018087528A1 · 2018 [cited by applicant]
Amelard Robert et al. “Illumination-compensated non-contact imaging photoplethysmography via dual-mode temporally coded illumination”. Progress in Biomedical Optics and Imaging, SPIE—International Society for Optical En… [cited by applicant]
An Efficient Approach for Object Detection and Tracking of Objects in a Video with Variable Background, Kumar S. Ray and Soma Chakraborty, May 11, 2017. [cited by applicant]
Andreas Stein et al, Functionalization of Porous Carbon Materials with Designed Pore Architecture, Advanced Materials, 2008, 20, 1-29. [cited by applicant]
Arindam Sikdar et al, “Computer-Vision-Guided Human Pulse Rate Estimation: A Review”, IEEE Reviews in Biomedical Engineering, vol. 9, Sep. 16, 2016 (Sep. 16, 2016), pp. 91-105. [cited by applicant]
Armentano, et al, Sensors for sub-ppm No Gas Detection based on Carbon Nanotube Thin Films, Applied Physics Letters, Feb. 2003. [cited by applicant]
Blocker Timon et al, “An online PPGI approach for camera based heart rate monitoring using beat-to-beat detection”, 2017 IEEE Sensors Applications Symposium (SAS), IEEE, Mar. 13, 2017. [cited by applicant]
British Search Report regarding Appliction No. 1900034.8 dated Jun. 13, 2019. [cited by applicant]
Carbon-atom wires produced by nanosecond pulsed laser deposition in a baground gas, Casari, et al, Dept. of Energy, Via Ponzio 34/3 Milan, Italy (2016 190-195. [cited by applicant]
D. Vick, et al. Production of porous carbon thin films by pulsed laser deposition, Dept. of Electrical and Computer Engineering, Univ. of Alberta, Canada, Feb. 6, 1999. [cited by applicant]
Distance PPG: Robust Non-Contact Vital Signs Monitoring Using a Camera by Mayank Kumar et al; Apr. 6, 2015; Biomedical Optics Express 1565, May 1, 2015, vol. 6 No. 5. [cited by applicant]
Distance PPG: Robust Noncontact Vital Signs Monitoring Using a Camera, Kumar, 2015. [cited by applicant]
Eduard Llobet, Gas Sensors Using Carbon Nanomaterials: A Review, Sensors and Actuators B 179 (2013) 32-45. [cited by applicant]
European Search Report regarding Application No. 19220090.5-115 dated Feb. 24, 2020. [cited by applicant]
European Search Report regarding Application No. EP 19 15 8085 dated Jul. 10, 2019. [cited by applicant]
Extended EP Search Report regarding Application No. 19220090.5 dated Feb. 24, 2020. [cited by applicant]
Extended European Search Report regarding applicaiton No. 18168310.3-1115 dated Oct. 1, 2018. [cited by applicant]
Foreground detection, May 5, 2020, https://en.wikipedia.org/w/index.php?title=Foreground_detection&oldid=854014910. [cited by applicant]
Henley, et al, Dynamics of Confirmed Plumes During Short and Ultrashort Pulsed Laser Ablation of Graphite, Physical Review B 72, 205413, (2005). [cited by applicant]
Hisato AOTA et al, “Extracting objects by clustering of full pixel trajectories”, Signal Processing And Multimedia Applications (SIGMAP), Proceedings of the 2010 International Conference On, IEEE, Jul. 26, 2010 (Jul. 26… [cited by applicant]
Inagaki, et al, Carbon foam: preparation and application, Carbon (2015). [cited by applicant]
International Preliminary Report on Patentability and Written Opinion regarding Applicaiton No. PCT/GB2017/052779 dated Mar. 19, 2019. [cited by applicant]
International Search Report and Written Opinion for PCT/GB2017/052779, dated Nov. 10, 2017; ISA/EP. [cited by applicant]
International Search Report and Written Opinion for PCT/GB2017/053343, mailed Jan. 4, 2018; ISA/EP. [cited by applicant]
International Search Report for PCT/GB2017/050126, ISA/EP, Rijswijk, NL, mailed Apr. 20, 2017. [cited by applicant]
International Search Report for PCT/GB2017/050127, ISA/EP, Rijswijk, NL, mailed Mar. 28, 2017. [cited by applicant]
International Search Report for PCT/GB2017/050128, ISA/EP, Rijswijk, NL, mailed Apr. 13, 2017. [cited by applicant]
International Search Report for PCT/GB2017/050162, ISA/EP, Rijswijk, NL, mailed Jul. 6, 2017. [cited by applicant]
Kononenko et al, Pulsed laser deposition of hard carbon coatings at atmospheric pressure, Quantium Electronics 33(3) 189-191 (2003). [cited by applicant]
Konstantinos Avgerinakis et al, “Activity detection and recognition of daily living events”, Proceedings of the 1ST ACM International Workshop on Multimedia Indexing and Information Retrieval for Healthcare, MIIRH '13, … [cited by applicant]
Kumar-DistancePPG: Robust non-contact vital signs monitoring using a camera, Optical Society of America (2015). [cited by applicant]
Making porous conductive carbon films with unbalanced magnetron sputtering, Kim, et al, Japanese Journal of Applied Physics 54, (2015). [cited by applicant]
Mayank Kumar et al, DistancePPG: Robust non-contact vital signs monitoring using a camera, Biomedical Optics Express, 2015, pp. 1565-1588. [cited by applicant]
Melios, et al, Detection of Ultra-Low Concentration NO in Complex Envirnment Using Epitaxial Graphene Sensors, American Chemical Society,. [cited by applicant]
Nakajima, Kazuki, Yoshiaki Matsumoto, and Toshiyo Tamura. “Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed.” Physiological Measurement 22.3 (2001). [cited by applicant]
Nathalie M. El Nabbout et al, “Automatically Detecting and Tracking People Walking through a Transparent Door with Vision”, Computer and Robot Vision, 2008. CRV '08. Canadian Conference on, IEEE, Piscataway, NJ, USA, Ma… [cited by applicant]
Pisani, Francesco et al. “Real-time automated detection of clonic seizures in newborns.” Clinical Neurophysiology, (2014) http://dx.doi.org/10.1016/j.clinph.2013.12.119. [cited by applicant]
Pisani-Real-time Automated Detection of Clonic Seizures in Newborns, Clinical Neurophysiology 125 (2014) 1533-1540. [cited by applicant]
Production of porous carbon thin films by pulsed laser deposition, D. Vick, et al, Dept. of Electrical and Computer Entineering, Univ. of Alberta, Canada, Feb. 6, 1999. [cited by applicant]
Qiang ZHU et al, “Learning a Sparse, Corner-Based Representation for Corner-Based Representation for Time-varying Background Modeling” , Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on Beijing, … [cited by applicant]
R. Janmohamed et al, Study of Porous Carbon Thin Films produced by Pulsed Laser Deposition, Dept. of Electrical and Computer Engineering, Univ. of Alberta, Edmonton, Canada, Applied Surface Science (2007) 7964-7968. [cited by applicant]
Real Time Automated Detection of Clonic Seizures In Newbors, Pisani, 2014. [cited by applicant]
Remote Plethysmorgraphic Imaging Using Ambient Light, Verkruysse, Dec. 22, 2008. [cited by applicant]
Rode, et al, Formation of Cluster-Assembled Carbon Nano-Foam by High-Repetition Rate Laser Ablation, Applied Phsics A Materials Science & Processing, 135-144 (2000). [cited by applicant]
Rode, et al, Structural Analysis of a Carbon Foam Formed by High Pulse-Rate Laser Ablation, Appl. Phys. A 69 (Suppl.) S755-S758 (1999). [cited by applicant]
Search Report for GB Application No. 1615899.0, dated Feb. 28, 2017. [cited by applicant]
Search Report for GB Application No. 1618828.6, dated Mar. 31, 2017. [cited by applicant]
Search Report for Priority Application GB1601140.5, UK IPO, Newport, South Wales, mailed Jul. 21, 2016. [cited by applicant]
Search Report of UKIPO regarding Application No. GB1900033.0 dated Jun. 13, 2019. [cited by applicant]
Search Report regarding United Kingdom Patent Application No. GB1706449.4, dated Oct. 25, 2017. [cited by applicant]
Search Report under Section 17(5) for priority application GB1601142.1, UKIPO, Newport, South Wales, mailed Jun. 28, 2016. [cited by applicant]
Shandong WU et al, “A hierarchical motion trajectory signature descriptor”, 2008 IEEE International Conference on Robotics and Automation. The Half-Day Workshop on: Towards Autonomous Agriculture of Tomorrow, IEEE—Pisca… [cited by applicant]
Tarassenko et al, “Non-contact video-based vital sign monitoring using ambient light and auto-regressive models”, 2014 Physiol. Meas. 35 807, pp. 807-831. [cited by applicant]
Thareja, et al, Pulsed Laser Deposition of Carbon Films, Department of Physics and Centre for Laser Technology, Indian Institute of Tech., Kanpur 208 016 (UP) India. [cited by applicant]
Tongchi Zhou et al, “A study of relative motion point trajectories for action recognition”, 2015 International Conference on Wireless Communications & Signal Processing (WCSP), IEEE, Oct. 15, 2015 (Oct. 15, 2015), pp. 1… [cited by applicant]
UK IPO Search Report for GB priority application 1601217.1, Newport, South Wales, mailed Jul. 25, 2016. [cited by applicant]
UK IPO Search Report under Section 17(5) for priority application GB1061143.9, mailed Mar. 30, 2016. [cited by applicant]
Verkruysse et al., Remote Plethysmographic Imaging Using Ambient Light, Optices Express, 16 (26), Dec. 22, 2008 pp. 21434-21445. [cited by applicant]
Verkruysse-Remote Plethysmographic Imaging using Ambient Light, Optics Express (Dec. 22, 2008) vol. 16, No. 26. [cited by applicant]
Wim Verkruysse et al., “Remote Plethysmographic imaging using ambient light”, Optics Express, 16(26), Dec. 22, 2008, pp. 21434-21445. [cited by applicant]
Written Opinion of the ISA for PCT/GB2017/050126, ISA/EP, Rijswijk, NL, mailed Apr. 20, 2017. [cited by applicant]
Written Opinion of the ISA for PCT/GB2017/050127, ISA/EP, Rijswijk, NL, mailed Mar. 28, 2017. [cited by applicant]
Written Opinion of the ISA for PCT/GB2017/050128, ISA/EP, Rijswijk, NL, mailed Apr. 13, 2017. [cited by applicant]
Written Opinion of the ISA for PCT/GB2017/050162, ISA/EP, Rijswijk, NL, mailed Jul. 6, 2017. [cited by applicant]
Wu et al, Eulerian Video Magnification for Revealing Subtle Changes in the World, 2012. [cited by applicant]
Yap, et al, Nanostructured Diamond-Like Carbon Films Grown by Off-Axis Pulsed Laser Deposition, Hindawi Pub. Corp. vol. 2015, Article ID 731306, Sep. 4, 2015. [cited by applicant]
Yavari, et al, High Sensitivity Gas Detection Using a Macroscopic Three-Dimensional Graphene Foam Network, Scientific Reports, Sep. 30, 2011. [cited by applicant]
Yu Sun et al, “Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging”, IEEE Transactions on Biomedical Engineering, IEEE Service Center, Piscataway, NJ, USA, vol. 63, No. 3, Mar. 1, 2016 (Mar… [cited by applicant]
Search Report of UKIPO regarding Application No. GB1900032.2 dated Jun. 13, 2019. [cited by applicant]