Sensing device for medical facilities
A medical system may utilize a modular and extensible sensing device to derive a two-dimensional (2D) or three-dimensional (3D) human model for a patient in real-time based on images of the patient captured by a sensor such as a digital camera. The 2D or 3D human model may be visually presented on one or more devices of the medical system and used to facilitate a healthcare service provided to the patient. In examples, the 2D or 3D human model may be used to improve the speed, accuracy and consistency of patient positioning for a medical procedure. In examples, the 2D or 3D human model may be used to enable unified analysis of the patient's medical conditions by linking different scan images of the patient through the 2D or 3D human model. In examples, the 2D or 3D human model may be used to facilitate surgical navigation, patient monitoring, process automation, and/or the like.
1. A portable sensing device, comprising:
a housing configured to be installed on a device or a structure in a medical environment;
one or more digital cameras configured to be removably installed inside the housing and capture at least one photo of a patient in the medical environment;
a first processor and a second processor, wherein:
the first processor and the second processor are configured to be removably installed inside the housing;
the first processor is configured to determine a first three-dimensional (3D) human model associated with the patient based on an artificial neural network (ANN), the at least one photo of the patient captured by the one or more digital cameras, and a first set of requirements associated with a first imaging modality;
the second processor is configured to determine a second 3D human model associated with the patient based on the ANN, the at least one photo of the patient captured by the one or more digital cameras, and a second set of requirements associated with a second imaging modality; and
the first 3D human model and the second 3D human model are determined based on a body shape and a plurality of joint angles of the patient estimated by the ANN; and
one or more communication circuits configured to transmit information associated with the first 3D human model or the second 3D human model to a receiving device.
2. The portable sensing device of claim 1 , wherein the first imaging modality is associated with computer tomography (CT) imaging, and wherein the second imaging modality is associated with magnetic resonance imaging (MRI).
3. The portable sensing device of claim 1 , wherein the first imaging modality is associated with X-ray imaging, and wherein the second imaging modality is associated with computer tomography (CT) imaging or magnetic resonance imaging (MRI).
4. The portable sensing device of claim 1 , wherein the housing includes a first slot and a second slot, the first slot configured to host the first processor, the second slot configured to host the second processor.
5. The portable sensing device of claim 1 , wherein the one or more digital cameras include a first subset of digital cameras configured to capture a first photo of the patient in accordance with the first set of requirements associated with the first imaging modality, and wherein the one or more digital cameras further include a second subset of digital cameras configured to capture a second photo of the patient in accordance with the second set of requirements associated with the second imaging modality.
6. The portable sensing device of claim 1 , further comprising a depth sensor configured to capture a depth image of the patient, and wherein the ANN is trained to estimate the body shape and the plurality of joint angles of the patient further based on the depth image of the patient.
7. The portable sensing device of claim 1 , wherein the housing is configured to be attached to a medical scanner located in the medical environment.
8. The portable sensing device of claim 1 , wherein at least one of the first 3D human model or the second 3D human model includes a 3D mesh of the patient.
9. The portable sensing device of claim 1 , wherein at least one of the first 3D human model or the second 3D human model indicates a position or a pose of the patient before or during a medical procedure.
10. The portable sensing device of claim 9 , wherein at least one of the first processor or the second processor is further configured to determine relative positions of the patient and a medical device associated with the medical procedure, and wherein the information transmitted to the receiving device includes an indication of the relative positions of the patient and the medical device.
11. The portable sensing device of claim 1 , wherein the ANN is trained to extract a plurality of features from the at least one photo of the patient and to estimate the body shape and the plurality of joint angles of the patient based on the extracted features.
12. The portable sensing device of claim 1 , wherein the one or more communication circuits are further configured to receive a command associated with determining the first 3D human model or the second 3D human model from an external device, and wherein the first processor and the second processor are further configured to determine the first 3D human model or the second 3D human model based on the command.
13. A system, comprising:
a portable sensing device and a processing device, wherein:
the portable sensing device comprises one or more digital cameras, a first processor, and a second processor that are configured to be removably installed inside a housing of the portable sensing device;
the one or more digital cameras are configured to capture at least one photo of a patient in a medical environment;
the first processor is configured to determine a first three-dimensional (3D) model associated with the patient based on an artificial neural network (ANN), the at least one photo of the patient captured by the one or more digital cameras, and a first set of requirements associated with a first imaging modality;
the second processor is configured to determine a second 3D human model associated with the patient based on the ANN, the at least one photo of the patient captured by the one or more digital cameras, and a second set of requirements associated with a second imaging modality;
the first 3D human model and the second 3D human model are determined based on a body shape and a plurality of joint angles of the patient estimated by the ANN; and
the processing device is configured to receive information associated with the first 3D human model or the second 3D human model from the portable sensing device, and generate, based on the received information, a command for guiding or alerting the patient or another person, or for controlling a medical device in the medical environment.
14. The system of claim 13 , wherein the first imaging modality is associated with X-ray imaging, and wherein the second imaging modality is associated with computer tomography (CT) imaging or magnetic resonance imaging (MRI).
15. The system of claim 13 , wherein the first imaging modality is associated with computer tomography (CT) imaging, and wherein the second imaging modality is associated with magnetic resonance imaging (MRI).
16. The system of claim 13 , wherein the housing includes a first slot and a second slot, the first slot configured to host the first processor, the second slot configured to host the second processor.
17. The system of claim 13 , wherein the portable sensing device further comprises a depth sensor configured to capture a depth image of the patient, and wherein the ANN is trained to estimate the body shape and the plurality of joint angles of the patient further based on the depth image of the patient.
18. The system of claim 13 , wherein at least one of the first 3D human model or the second 3D human model includes a three-dimensional (3D) mesh of the patient.
19. The system of claim 13 , wherein the processing device is further configured to determine relative positions of the patient and the medical device in response to receiving the information from the portable sensing device, and wherein the command for guiding or alerting the patient or the other person, or for controlling the medical device is generated based on the relative positions of the patient and the medical device.