IP Library Granted Patent US 10,342,464
Granted Patent B2
US 10,342,464 · App. 14/837,169 · Granted Jul 9, 2019

3D camera system for infant monitoring

Inventor: Erik A. Niemeyer (Rio Rancho, NM)
Assignee: Intel Corporation
A61B5/113A61B5/0077A61B5/0826A61B5/0022A61B5/746A61B2503/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,342,464
App. No.
14/837,169
Granted
Jul 9, 2019
Kind
B2
Abstract

Systems and methods employing a depth-sensing camera to detect abdomen rise and fall during an infant sleep period, or lack thereof due to respiratory arrest. Visual processing of image data collected by one or more 3D digital camera is performed to detect the infant and to measure a distance between an infant's abdominal cavity and a camera baseline over time. An alarm may be triggered at by the system, locally and/or at remote devices, such as a mobile phone, tablet, laptop, etc. The monitoring system may also detect situations when an infant rolls from a back-sleeping to a belly-sleeping position.

Claims (67)

1. A computerized respiratory monitoring device, comprising:

an input port to receive a sequence of image frames from camera image sensor data collected over a time interval; and

one or more processors coupled to the input port, the processors to:

process a sequence of image frames from camera image sensor data collected over a time interval;

determine a depth map associated with each of the frames;

determine changes in the depth map over the frames;

determine a respiratory cycle over the time interval based on the depth map changes;

store an indication of the respiratory cycle to an electronic memory; and

generate an alert if a frequency or an amplitude of the respiratory cycle fails to satisfy one or more predetermined criteria, wherein to determine the respiratory cycle, the processor is to:

determine transitions between exhalation and inhalation based on a maximum and a minimum of filtered depth values associated with a region of interest (ROI) within the images over time;

determine a frequency of the respiratory cycle based on the transitions between exhalation and inhalation; and

determine a magnitude of the respiratory cycle based on a difference between consecutive maximum and minimum filtered depth values.

2. The device of claim 1 , wherein the processor is further to:

compare the respiratory frequency against a predetermined minimum respiration rate threshold; and

compare the respiratory magnitude against a predetermined minimum respiration tidal volume threshold.

3. The device of claim 1 , wherein a processor to determine the respiratory cycle is to:

spatially filter depth values for each of the frames; or

temporally filter depth values over a plurality of frames; and

store the filtered depth values in association with a timestamp.

4. The device of claim 1 , wherein:

a processor to determine changes in the depth map is to:

select the ROI within the frames based on a position of facial features detected within the frames, in response to succeeding to detect the facial features; and

track depth values limited to within the ROI over the frames; and

a processor is to generate a second alert in response to failing to detect the facial features subsequent to the indication of the respiratory cycle being stored to an electronic memory.

5. The device of claim 1 , further comprising:

at least one of a plurality of digital cameras, a time of flight digital camera, or a structured light illuminator to generate the camera image sensor data.

6. A mobile monitoring platform, comprising:

a mount;

a support arm attached to the mount, the support arm configured to suspending one or more objects; and the computerized respiratory monitoring device recited in claim 1 .

7. One or more non-transitory computer-readable storage media, with instructions stored thereon, which when executed by a processor, cause the processor to perform a method comprising: processing a sequence of image frames from camera image sensor data collected over a time interval; determining a depth map associated with each of the frames; determining changes in the depth map over the frames; determining a respiratory cycle over the time interval based on the depth map changes, wherein determining the respiratory cycle further comprises: spatially filtering depth values associated with a region of interest (ROI) for each of the frames; temporally filtering depth values over a plurality of frames; storing the filtered depth values in association with a timestamp, determining transitions between exhalation and inhalation based on a maximum and minimum of the filtered depth values over time; determining a frequency of the respiratory cycle based on the transitions between exhalation and inhalation, and determining a magnitude of the respiratory cycle based on a difference between consecutive maximum and minimum depth values; and generating an alert if the a frequency or an amplitude of the respiratory cycle fails to satisfy one or more predetermined criteria.

8. The media of claim 7 ,

wherein determining changes in the depth map further comprises:

selecting the ROI within the frames based on a position of facial features detected within the frames in response to succeeding to detect the facial features; and

tracking depth values limited to within the ROI over the frames; and

wherein the processor is further to generate a second alert in response to failing to detect the facial features subsequent to the indication of the respiratory cycle being stored to an electronic memory.

9. The media of claim 8 , wherein the facial features exclude eyes.

10. A computer implemented method for respiratory monitoring, comprising:

processing a sequence of image frames from camera image sensor data collected over a time interval;

determining a depth map associated with each of the frames;

determining changes in the depth map over the frames;

determining a respiratory cycle over the time interval based on the depth map changes; and

generating an alert if a frequency or an amplitude of the respiratory cycle fails to satisfy one or more predetermined criteria, wherein determining the respiratory cycle over the time interval based on the depth map changes further comprises:

determining transitions between exhalation and inhalation based on a maximum and a minimum of filtered depth values over time;

determining a frequency of the respiratory cycle based on the transitions between exhalation and inhalation; and

determining a magnitude of the respiratory cycle based on a difference between consecutive maximum and minimum filtered depth values.

11. The method of claim 10 ,

wherein determining changes in the depth map further comprises:

selecting the ROI within the frames based on a position of facial features detected within the frames, in response to succeeding to detect the facial features;

tracking depth values limited to within the ROI over the frames; and

the method further comprises generating a second alert in response to failing to detect the facial features subsequent to determining the respiratory cycle.

12. The method of claim 11 , wherein the facial features exclude eyes.

13. The method of claim 10 , wherein determining the respiratory cycle over the time interval based on the depth map changes further comprises at least one of:

spatially filtering depth values for each of the frames;

temporally filtering depth values over a plurality of frames; and

storing the filtered depth values in association with a timestamp.

14. The method of claim 10 , wherein generating the alert further comprises:

comparing the frequency against a predetermined minimum respiration rate threshold; and

comparing the magnitude against a predetermined minimum respiration tidal volume threshold.

15. The method of claim 14 , wherein generating the alert further comprises:

triggering a first alarm locally, or remotely over a communication network, in response to the frequency failing to satisfy the predetermined minimum respiration rate threshold for a predetermined period of time; and

triggering a second alarm locally, or remotely over a communication network, in response to the magnitude failing to satisfy the predetermined minimum respiration tidal volume threshold for a predetermined period of time.

16. The method of claim 10 , further comprising:

collecting the image frames with at least one of:

a plurality of digital cameras;

a time of flight digital camera; or

a structured light illuminator.

17. The method of claim 16 , further comprising illuminating, with a near infrared (NIR) source, at least a portion of a field of view associated with the camera image sensor.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2016
From: NIEMEYER, ERIK A.
To: INTEL CORPORATION
Reel/Frame 039644/0518 →
Continuity (1)
Related Publication 20170055877A1 · Mar 2, 2017
Cited By (2)
US 12,220,220 US 12,714,622