IP Library Granted Patent US 11,998,366
Granted Patent B2
US 11,998,366 · App. 17/862,266 · Granted Jun 4, 2024

Generating a breathing alert

Inventors: Jacob Gunther (North Logan, UT); Nathan E. Ruben (Smithfield, UT)
Assignee: Nutrits, Ltd.
A61B5/743A61B5/0816A61B5/107A61B5/4815A61B5/4818A61B5/7264A61B5/741A61B5/746A61B5/747G06T7/0012G06T7/13G06T7/143G06T7/194G06T7/44G06T2207/10016G06T2207/20076G06T2207/30088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,998,366
App. No.
17/862,266
Granted
Jun 4, 2024
Kind
B2
Abstract

For generating a breathing alert is disclosed, a method receives a video stream of a subject. The method further estimates a breathing signal from the video stream. The method determines one of a large-scale motion and/or a breathing event of the subject based on the breathing signal. The method generates an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval.

Claims (83)

1. A method comprising:

training, by a processing apparatus, a breathing model on training video streams of a plurality of subjects;

receiving, by the processing apparatus, a first video stream of a subject;

estimating, by the processing apparatus, a breathing signal from the first video stream using the breathing model;

determining, by the processing apparatus, one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;

generating, by the processing apparatus, an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;

presenting, by the processing apparatus, the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, and inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;

detecting, by a processing apparatus, a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and

communicating, by the processing apparatus, a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

2. The method of claim 1 , the method further comprising:

training, by a processing apparatus, a crying model on a plurality of subjects crying;

receiving, by the processing apparatus, an audio stream; and

identifying, by the processing apparatus, the subject crying from the audio stream using the crying model.

3. The method of claim 1 , the method further comprising:

training, by a processing apparatus, a smiling model on a plurality of subjects smiling; and

identifying, by the processing apparatus, the subject smiling from the first video stream using the smiling model.

4. The method of claim 1 , the method further comprising:

measuring, by a processing apparatus, ambient light; and

in response to the ambient light being less than a light threshold, activating, by the processing apparatus, an infrared light, wherein the first video stream captures the subject in infrared.

5. The method of claim 1 , the method further comprising:

detecting, by the processing apparatus, a face from the first video stream;

scaling, by the processing apparatus, a face region comprising the face to a specified pixel size;

calculating, by the processing apparatus, a skin histogram on a color space of face region;

selecting, by the processing apparatus, vectors of pixels based on the skin histogram;

iteratively generating, by the processing apparatus, a skin mask from the vectors; and

iteratively tracking, by the processing apparatus, the face.

6. An apparatus comprising:

a processing apparatus;

a memory storing code executable by the processing apparatus to perform:

training a breathing model on training video streams of a plurality of subjects;

receiving a first video stream of a subject;

estimating a breathing signal from the first video stream using the breathing model;

determining one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;

generating an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;

presenting the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, and inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;

detecting a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and

communicating a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

7. The apparatus of claim 6 , the processing apparatus further:

training a crying model on a plurality of subjects crying;

receiving an audio stream; and

identifying the subject crying from the audio stream using the crying model.

8. The apparatus of claim 6 , the processing apparatus further:

training a smiling model on a plurality of subjects smiling; and

identifying the subject smiling from the first video stream using the smiling model.

9. The apparatus of claim 6 , the processing apparatus further:

measuring ambient light; and

in response to the ambient light being less than a light threshold, activating an infrared light, wherein the first video stream captures the subject in infrared.

10. The apparatus of claim 6 , the processing apparatus further:

detecting a face from the first video stream;

scaling a face region comprising the face to a specified pixel size;

calculating a skin histogram on a color space of face region;

selecting vectors of pixels based on the skin histogram;

iteratively generating a skin mask from the vectors; and

iteratively tracking the face.

11. A system comprising:

a camera;

a processing apparatus;

a memory storing code executable by the processing apparatus to perform:

training a breathing model on training video streams of a plurality of subjects;

receiving a first video stream from the camera of a subject;

estimating a breathing signal from the first video stream using the breathing model;

determining one of a large-scale motion and/or a breathing event of the subject based on the breathing signal using the breathing model;

generating an alert if both no breathing event is identified and no large-scale motion of the subject is identified within an event time interval;

presenting the first video stream, a breathing report, and/or an audio stream, wherein the breathing report comprises a numerical representation, a heat map, a graph, and/or a streaming sinusoidal graph of a breath rate, a maximum inter-breath interval, a minimum inter-breath interval, inter-breath interval statistics, an inter-breath interval histogram, motion frequency, motion magnitude, motion duration, sleep length, sleep quality, sleep intervals, a sleep position, a sleep status, and/or apnea event data;

detecting a sleep position and/or a sleep status of the subject from the first video stream, wherein the sleep position comprises thrashing, a stomach orientation, a back orientation, and/or a side orientation and the sleep status comprises asleep and awake; and

communicating a notification in response to a specified sleep position, a specified sleep status, a change of the sleep position, and/or a change of sleep status.

12. The system of claim 11 , the processing apparatus further:

training a crying model on a plurality of subjects crying;

receiving an audio stream; and

identifying the subject crying from the audio stream using the crying model.

13. The system of claim 11 , the processing apparatus further:

training a smiling model on a plurality of subjects smiling; and

identifying the subject smiling from the first video stream using the smiling model.

14. The system of claim 11 , the processing apparatus further:

measuring ambient light; and

in response to the ambient light being less than a light threshold, activating an infrared light, wherein the first video stream captures the subject in infrared.

15. The system of claim 11 , the processing apparatus further:

detecting a face from the first video stream;

scaling a face region comprising the face to a specified pixel size;

calculating a skin histogram on a color space of face region;

selecting vectors of pixels based on the skin histogram;

iteratively generating a skin mask from the vectors; and

iteratively tracking the face.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2026
From: NUTRITS LTD
To: NANOBEBE LTD
Reel/Frame 074694/0700 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2024
From: GUNTHER, JACOB; RUBEN, NATHAN E.
To: NUTRITS LTD.
Reel/Frame 067229/0094 →
SECURITY INTEREST Recorded May 22, 2023
From: NUTRITS LTD.
To: ARBEL FUND 2 L.P; ARBEL FUND 2B L.P
Reel/Frame 063718/0062 →
Continuity (4)
Continuation 16828479 · Mar 24, 2020
Continuation In Part 15697220 · Sep 6, 2017
Provisional Application 62383769 · Sep 6, 2016
Related Publication 20220361824A1 · Nov 17, 2022