Bedding system with a CNN based machine vision process
A bedding system uses a convolutional neural network (CNN)-based machine vision to makes adjustments for comfort and/or support. The machine vision process identities a body position by using a trained CNN that receives a pressure image and identifies a body position. The body position may be determined by classifying the pressure image into a predetermined body position classification. The machine vision process includes at least one trained CNN that determines joint locations. The machine vision tracks pressure accumulated at joints over time.
1. A method for adjusting a bedding system, comprising:
receiving a two-dimensional pressure image of a sleeper on a bedding system while the sleeper is sleeping on the bedding system; and
applying a machine vision process to analyze the pressure image, comprising:
selecting, by a first machine learned model, a position classification for a body position of the sleeper, the position classification selected from a set of predetermined position classifications, wherein the first machine learned model comprises a convolutional neural network, the convolutional neural network trained using training data comprising pressure images tagged with body position labels, and wherein the convolutional neural network comprises at least three convolution layers each of which is coupled to a pooling layer,
determining, by a second machine learned model, one or more joint locations of the sleeper, the one or more joints selected from a set of predetermined joints, and
determining pressure values at the one or more joints using the pressure image and the one or more joint locations.
2. The method of claim 1 , further comprising:
adjusting a comfort and/or support of the bedding system based at least in part on the position classification.
3. The method of claim 1 , wherein applying the machine vision process further comprises:
tracking the pressure values at the one or more joints over time.
4. The method of claim 1 , further comprising:
selecting the second machine learned model from a set of machine learned models for determining joint locations based on the position classification, each of the set of machine learned models corresponding to a predetermined position classification.
5. The method of claim 1 , wherein the set of predetermined position classifications includes a leftside-sleeping classification, a rightside-sleeping classification, a prone-sleeping classification, and a supine-sleeping classification.
6. A method for adjusting a bedding system, comprising:
receiving a two-dimensional pressure image of a sleeper on a bedding system while the sleeper is sleeping on the bedding system; and
applying a machine vision process to analyze the pressure image, comprising:
pre-processing the pressure image before providing the pressure image to a first machine learned model, wherein the pre-processing comprises center-aligning the pressure image by aligning a center of a body on the pressure image to a center of the pressure image
selecting, by the first machine learned model, a position classification for a body position of the sleeper, the position classification selected from a set of predetermined position classifications,
determining, by a second machine learned model, one or more joint locations of the sleeper, the one or more joints selected from a set of predetermined joints, and
determining pressure values at the one or more joints using the pressure image and the one or more joint locations.
7. The method of claim 6 , wherein the pre-processing further comprises normalizing the center-aligned pressure image.
8. The method of claim 6 , wherein the pre-processing further comprises resizing the pressure image to a predetermined size responsive to determining that the size of the pressure image is less than a predetermined threshold.
9. The method of claim 1 , wherein a joint location comprises at least one of a 2-dimensional coordinate and a 3-dimensional coordinate.
10. The method of claim 1 , wherein the second machine learned model comprises a convolutional neural network.
11. The method of claim 10 , further comprising training the convolutional neural network using training data comprising training pressure images and a ground truth table including identified joint locations of one or more joints for the training pressure images.
12. The method of claim 11 , wherein the identified joint locations are identified by manual labeling, color detection, or motion detection.
13. The method of claim 11 , wherein training the convolutional neural network further comprising training the convolutional neural network according to an objective function to minimize errors between actual joint locations and the determined joint locations.
14. The method of claim 13 , wherein the objective function defines relationships between joints.
15. The method of claim 10 , wherein the second machine learned model further comprises at least one of a kinematic model and an auto-encoder coupled to the convolutional neural network, the kinematic model or the auto-encoder defining relationships among joints and to refine the joint locations determined by the convolutional neural network.
16. A method for adjusting a bedding system, comprising:
receiving a two-dimensional pressure image of a sleeper on a bedding system while the sleeper is sleeping on the bedding system; and
applying a machine vision process to analyze the pressure image, comprising:
selecting, by a first machine learned model, a position classification for a body position of the sleeper, the position classification selected from a set of predetermined position classifications,
determining, by a second machine learned model, one or more joint locations of the sleeper, the one or more joints selected from a set of predetermined joints, and
determining pressure values at the one or more joints using the pressure image and the one or more joint locations, wherein determining pressure values at the one or more joints using the pressure image and the one or more joint locations comprises:
clustering locations of the pressure image into multiple clusters; and
for each joint location, identifying a corresponding cluster having a center closest to the joint location and averaging the pressure measures in the corresponding cluster to calculate the pressure value at the joint location.
17. The method of claim 16 , wherein clustering locations into multiple cluster comprises:
generating a pressure graph including nodes connected by edges using the pressure image, each node corresponding to one of the locations and each edge connecting one of the nodes to one of the closest neighboring nodes to the node; and
partitioning the nodes into multiple clusters, each cluster including a set of the nodes.