IP Library Granted Patent US 10,229,491
Granted Patent B1
US 10,229,491 · App. 15/210,478 · Granted Mar 12, 2019

Medical environment monitoring system

Inventors: Benjamin D. Rush (Lincoln, NE); Joshua M. Brown-Kramer (Lincoln, NE); Lucas A. Sabalka (Lincoln, NE); Brian S. Gansemer (Lincoln, NE); Clay T. Upton (Lincoln, NE); Paul B. Bauer (Lincoln, NE); Andrew R. Unterseher (Lincoln, NE); Benjamin J. Origas (Lincoln, NE); Douglas W. Durham (Lincoln, NE)
Assignee: Ocuvera LLC
G06T7/0012G06K9/52G06K9/6267G06T7/004G06T7/0051G06T7/60G06T2207/30004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,229,491
App. No.
15/210,478
Granted
Mar 12, 2019
Kind
B1
Abstract

A system and a method are described for monitoring a medical care environment. In one or more implementations, a method includes identifying a plurality of potential locations within a field of view of a camera as representing a bed and/or a seating platform. The method also includes identifying a first subset of pixels from the plurality of potential locations as representing a bed and/or a seating platform. The method also includes identifying a second subset of pixels within the field of view of the camera as representing an object (e.g., a subject, such as a patient, medical personnel; bed; chair; patient tray; medical equipment; etc.) proximal to the bed and/or seating platform. The method also includes determining an orientation of the object with respect to the bed and/or seating platform. In implementations, the method further includes issuing an electronic communication alert based upon the orientation of the object.

Claims (63)

1. A system for monitoring a medical care environment, the system comprising:

an imaging device having an imaging plane, the imaging device configured to receive a plurality of depth frame images comprising a plurality of pixels, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform;

at least one computing device in communication with the imaging device, the at least one computing device including:

a memory configured to store one or more modules;

a processor coupled to the memory, the processor configured by a fusion engine and the one or more modules to cause the processor to:

generate a depth mask representing the physical space based upon the plurality of depth frame images

identify, via one or more object location algorithms, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identify, via one or more object identification algorithms, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identify one or more second pixels of the plurality of pixels as representing a portion of a body of a subject proximate to the at least one of the bed or the seating platform;

process, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the portion of the body with respect to the image plane, the first possible orientation signal representing a position of the portion of the body with respect to the at least one of the bed or the seating platform;

process, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the portion of the body with respect to the image plane, the second possible orientation signal representing the position of the portion of the body with respect to the at least one of the bed or the seating platform;

determine a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determine a transitional state of the subject based upon the probability distribution; and

issue an electronic communication alert based upon the determined transitional state of the subject.

2. The system as recited in claim 1 , wherein the processor configured by a fusion engine and the one or more modules to cause the processor identify, via one or more object location algorithms, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

3. The system as recited in claim 1 , wherein the processor configured by a fusion engine and the one or more modules to cause the processor to identify, via one or more object identification algorithms, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations comprises modeling, via one or more gradient ascent algorithms, the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations and selecting the at least one of the plurality of potential bed locations or the plurality of potential seating platform with the highest fitness as representing the at least one of the bed or the seating platform.

4. The system as recited in claim 1 , wherein the processor is further configured to execute the one or more modules to cause the processor to receive feedback about a validity of the electronic communication alert.

5. The system as recited in claim 1 , wherein the processor is further configured to:

receive feedback about an accuracy of at least one of the depth mask, the one or more first pixels, or the one or more second pixels; and

dynamically adjust at least one of the depth mask, the one or more object identification algorithms, the one or more object location algorithms, the one or more first pixels, or the one or more second pixels.

6. The system as recited in claim 1 , wherein the processor is further configured to execute the one or more modules to cause the processor to process the depth frame image to detect at least one of a presence or an absence of a gesture; and provide an indication associated with the at least one of the presence or the absence of the gesture.

7. The system as recited in claim 1 , wherein the processor is further configured to determine an electronic communication alert sensitivity level associated with the subject based upon at least one characteristic of the subject.

8. The system as recited in claim 1 , wherein the at least one computing device comprises a computer cluster, the computer cluster comprising a plurality of cluster nodes configured to access a shared memory.

9. A method for monitoring a subject within a medical care environment, the method comprising:

generating, via a processor configured by a fusion engine, a depth mask representing the medical care environment based upon a plurality of depth frame images received from an imaging device, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform, the plurality of depth frame images including a plurality of pixels;

identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identifying, via the processor, one or more second pixels of the plurality of pixels as representing a portion of a body of a subject proximate to the at least one of the bed or the seating platform;

processing, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the portion of the body with respect to the image plane, the first possible orientation signal representing a position of the portion of the body with respect to the at least one of the bed or the seating platform;

processing, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the portion of the body with respect to the image plane, the second possible orientation signal representing the position of the portion of the body with respect to the at least one of the bed or the seating platform;

determining a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determining a transitional state of the subject based upon the probability distribution; and

issuing an electronic communication alert based upon the determined transitional state of the subject.

10. The method as recited in claim 9 , wherein identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

11. The method as recited in claim 9 , wherein identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations comprises modeling, via one or more gradient ascent algorithms, the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations and selecting the at least one of the plurality of potential bed locations or the plurality of potential seating platform with the highest fitness as representing the at least one of the bed or the seating platform.

12. The method as recited in claim 9 , further comprising:

tracking the one or more second pixels through the plurality of depth frame images;

determining an activity parameter of the subject based upon the tracked one or more second pixels; and issuing an activity electronic communication alert indicating the activity parameter when the activity parameter is outside of an activity threshold.

13. The method as recited in claim 9 , further comprising receiving feedback about a validity of the electronic communication alert.

14. The method as recited in claim 9 , further comprising determining an electronic communication alert sensitivity level associated with the subject based upon at least one characteristic of the subject.

15. A method for monitoring a medical care environment, the method comprising:

receiving a plurality of depth frame images comprising a plurality of pixels the plurality of depth frame images captured by an imaging device having an image plane, the plurality of depth frame images representing a three-dimensional environment corresponding to a physical space, the physical space including at least one of a bed or a seating platform;

generating, via a processor configured by a fusion engine, a depth mask representing the physical space based upon a plurality of depth frame images received from an imaging device;

identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask;

identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels as representing the at least one of the bed or the seating platform based upon the at least one of the plurality of potential bed locations or the plurality of potential seating platform locations;

identifying, via the processor, one or more second pixels of the plurality of pixels as representing a body part of a subject proximate to the at least one of the bed or the seating platform;

processing, via a first processing technique, the one or more second pixels to generate a first possible orientation signal of the body part with respect to the image plane, the first possible orientation signal representing a position of the body part with respect to the at least one of the bed or the seating platform;

processing, via a second processing technique via the fusion engine, the one or more second pixels to generate a second possible orientation signal of the body part with respect to the image plane, the second possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform;

determining, via the processor, a probability distribution based upon the first possible orientation signal and the second possible orientation signal via the fusion engine;

determining a transitional state of the subject based upon the probability distribution; and

issuing an electronic communication alert based upon the determined transitional state of the subject.

16. The method as recited in claim 15 , further comprising:

receiving feedback about an accuracy of at least one of the depth mask, the one or more first pixels, or the one or more second pixels; and

dynamically adjusting at least one of the generating, via a processor configured by a fusion engine, a depth mask representing the physical space, the identifying, via one or more object identification algorithms via the processor, one or more first pixels of the plurality of pixels, or the identifying, via the processor, one or more second pixels of the plurality of pixels in response to the feedback received.

17. The method as recited in claim 15 , wherein identifying, via one or more object location algorithms via the processor, at least one of a plurality of potential bed locations or a plurality of potential seating platform locations based upon the depth mask comprises generating a Boolean array from the depth mask and identifying maximal rectangles that represent at least one of a plurality of potential bed locations or a plurality of potential seating platform locations.

18. The method as recited in claim 15 , further comprising:

processing, via a third processing technique via the fusion engine, the one or more second pixels to generate a third possible orientation signal of the body part with respect to the image plane, the third possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform;

processing, via a fourth processing technique via the fusion engine, the one or more second pixels to generate a fourth possible orientation signal of the body part with respect to the image plane, the fourth possible orientation signal representing the position of the body part with respect to the at least one of the bed or the seating platform; and

determine the probability distribution based upon the first possible orientation signal, the second possible orientation signal, the third possible orientation signal, and the fourth possible orientation signal via the fusion engine.

19. The method as recited in claim 15 , further comprising identifying one or more third pixels of the plurality of pixels as representing at least one of a second subject or an object relative to the subject.

20. The method as recited in claim 19 , further comprising:

receiving feedback about an accuracy of the one or more third pixels; and

dynamically adjusting the identifying one or more third pixels of the plurality of pixels as representing at least one of a second subject or an object relative to the subject.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2016
From: NG HEALTH VENTURES, LLC
To: OCUVERA LLC
Reel/Frame 040323/0352 →
SECURITY INTEREST Recorded Nov 15, 2016
From: OCUVERA LLC
To: NG HEALTH VENTURES, LLC D/B/A OCUVERA
Reel/Frame 040324/0518 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2016
From: NEBRASKA GLOBAL INVESTMENT COMPANY, LLC
To: NG HEALTH VENTURES, LLC
Reel/Frame 040270/0450 →
Continuity (5)
Continuation In Part 15061323 · Mar 4, 2016
Continuation In Part 14055139 · Oct 16, 2013
Provisional Application 61880273 · Sep 20, 2013
Provisional Application 61826669 · May 23, 2013
Provisional Application 61714252 · Oct 16, 2012
Cited By (1)
US 12,354,273