Bed occupancy detection using geometric features from point clouds
Embodiments described herein provide systems and techniques for detecting hospital bed occupancy based on three-dimensional (3D) point clouds of the hospital bed extracted from depth images. In one aspect, a process for determining if a bed inside an operating room (OR) is occupied by a patient is disclosed. This process begins by receiving a 3D point cloud of a bed object within a depth image captured inside the OR. The process then segments the 3D point cloud of the bed object into a plurality of segments in both a length direction and a width direction of the bed object. Next, the process extracts a set of geometric features from the plurality of segments. The process subsequently applies a binary classifier to the set of geometric features to classify the bed object as either being occupied by a patient or not being occupied by a patient.
1 . A computer-implemented method for determining if a bed inside an operating room (OR) is occupied by a patient, the method comprising:
segmenting a three-dimensional (3D) point cloud of a bed object within a depth image captured in an OR into a plurality of segments in both a length direction and a width direction of the bed object, by i) segmenting the 3D point cloud along a length direction of the bed object into a first plurality of segments being three equal-sized segments, and ii) segmenting the 3D point cloud along a width direction of the bed object into a second plurality of segments; and then
extracting a set of geometric features from the plurality of segments; and then
applying a binary classifier to the set of geometric features to classify the bed object as either being occupied by a patient or not being occupied by a patient.
2 . The computer-implemented method of claim 1 , wherein the depth image is captured by a depth camera installed in the OR, and wherein the method further comprises extracting the 3D point cloud from the depth image by projecting each 2D pixel (u, v) and the corresponding distance value d(u, v) in the depth image into a 3D point (x, y, z) in a 3D-coordinate system aligned with the depth camera.
3 . The computer-implemented method of claim 1 , wherein the second plurality of segments are two equal-sized segments.
4 . The computer-implemented method of claim 3 , wherein extracting the set of geometric features from the plurality of segments includes:
computing a set of standard deviations of height values using the z-component of a subset of the 3D point cloud associated with each segment of the first plurality of equal-sized segments along the length direction; and
computing a set of standard deviations of height values using the z-component of a subset of the 3D point cloud associated with each segment of the second plurality of equal-sized segments along the width direction.
5 . The computer-implemented method of claim 4 wherein the patient is lying flat on the bed object, and wherein computing the set of standard deviations of height values using the z-component associated with each segment of the first plurality of segments along the length direction captures unique height distributions in each section of the patient's body in the length direction.
6 . The computer-implemented method of claim 4 wherein the patient is lying flat on the bed object, and wherein computing the set of standard deviations of height values using the z-component associated with each segment of the second plurality of segments along the width direction captures a geometrical symmetry along the width direction.
7 . The computer-implemented method of claim 1 , wherein prior to segmenting the 3D point cloud of the bed object, the method further comprises preprocessing the 3D point cloud by:
computing an average height value of the z-component of the 3D point cloud; and
removing a subset of 3D points in the 3D point cloud with the z-component values smaller than the computed average height value.
8 . The computer-implemented method of claim 7 , wherein extracting the set of geometric features further includes:
computing an overall standard deviation of height values using the z-component of the remaining 3D points in the 3D point cloud after preprocessing the 3D point cloud; and
combining the overall standard deviation as an additional geometric feature with the set of geometric features extracted from the plurality of segments.
9 . The computer-implemented method of claim 1 , wherein applying the binary classifier to the set of extracted geometric features to classify the bed object includes applying a decision tree classifier to the set of extracted geometric features, which outputs either a positive classification indicating the bed object is occupied by a patient or a negative classification indicating the bed object is not occupied by a patient.
10 . The computer-implemented method of claim 9 , wherein applying the binary classifier to the set of extracted geometric features to classify the bed object as the negative classification includes identifying and classifying a set of false positives associated with the bed object as the negative classification.
11 . The computer-implemented method of claim 10 , wherein the set of false positives includes scenarios when the bed object is occupied by various non-human objects including:
one or more boxes;
one or more OR equipments; and
one or more blankets.
12 . The computer-implemented method of claim 1 , wherein prior to applying the binary classifier to classify the bed object, the method further comprises:
constructing a training dataset from a set of depth-camera videos capturing OR workflow, wherein the training dataset includes a first class of labeled samples of occupied bed scenarios and a second class of samples of non-occupied bed scenarios; and
training the binary classifier using the training dataset.
13 . The computer-implemented method of claim 12 , wherein the non-occupied bed scenarios include both empty bed scenarios and non-human-object occupied bed scenarios.
14 . The computer-implemented method of claim 12 , wherein the training dataset includes training samples extracted from depth images of a training bed object captured from multiple viewing angles that include at least a top-view of the training bed object.
15 . The computer-implemented method of claim 1 , wherein the bed object includes:
a hospital bed used for transporting a patient into and out of the OR; and
a surgical table used to operate a patient during a surgical procedure in the OR.
16 . The computer-implemented method of claim 1 , wherein applying the binary classifier to the set of geometric features extracted from the 3D point cloud does not require color images of the OR.
17 . The computer-implemented method of claim 1 , wherein prior to segmenting the 3D point cloud of the bed object, the method further comprises receiving the depth image among a sequence of depth images captured in the OR.
18 . An apparatus for determining if a bed inside an operating room (OR) is occupied by a patient, the apparatus comprising:
one or more processors;
a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
segment a three-dimensional (3D) point cloud of a bed object within a depth image captured in an OR into a plurality of segments in both a length direction and a width direction of the bed object, by i) segmenting the 3D point cloud along a length direction of the bed object into a first plurality of segments, and ii) segmenting the 3D point cloud along a width direction of the bed object into a second plurality of segments being two equal-sized segments;
extract a set of geometric features from the plurality of segments; and
apply a binary classifier to the set of geometric features to classify the bed object as either being occupied by a patient or not being occupied by a patient.
19 . The apparatus of claim 18 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to preprocess the 3D point cloud prior to segmenting the 3D point cloud by:
computing an average height value of the z-component of the 3D point cloud; and
removing a subset of 3D points in the 3D point cloud with the z-component values smaller than the computed average height value.
20 . The apparatus of claim 18 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to receive the depth image among a sequence of depth images captured in the OR prior to segmenting the 3D point cloud of the bed object.
21 . A system for determining if a bed inside an operating room (OR) is occupied by a patient, the system comprising:
one or more depth cameras installed in the OR;
one or more processors coupled to the one or more depth cameras; and
a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the system to:
segment a three-dimensional (3D) point cloud of a bed object within a depth image captured in an OR into a plurality of segments in both a length direction and a width direction of the bed object;
extract a set of geometric features from the plurality of segments; and
apply a binary classifier to the set of geometric features to classify the bed object as either being occupied by a patient or not being occupied by a patient, wherein the binary classifier comprises a machine learning model trained using a training dataset constructed from a set of depth-camera videos that capture operating room workflow, and the training dataset includes a first class of labeled samples of occupied bed scenarios and a second class of labeled samples of non-occupied bed scenarios, wherein the non-occupied bed scenarios include both empty bed scenarios and non-human-object occupied bed scenarios.
22 . The system of claim 21 , wherein the one or more depth cameras include at least two depth cameras which are installed in the OR to capture depth images of the bed object from multiple viewing angles that include at least a top-view angle.
23 . The system of claim 21 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the system to receive the depth image among one or more sequences of depth images captured by the one or more depth cameras prior to segmenting the 3D point cloud of the bed object.