Body representations
For training a machine learning system for representing a patient body a plurality of stored medical imaging data sets each representing at least a part of a respective patient are obtained. A first one of the plurality of stored medical imaging data sets represents a different part of the patient body than a second one of the plurality of stored medical imaging data sets. A plurality of landmarks in the stored medical imaging data sets are estimated, and each of the stored medical imaging data sets are aligned to a predefined pose using the plurality of landmarks. A plurality of points in the aligned medical imaging data sets are sampled, and the machine learning system is trained based on at least the plurality of points. The learned parameters of the machine learning system are then stored and used in a method for inferring a body representation.
1. A computer-implemented method for training a machine learning system for representing a patient body, the computer-implemented method comprising:
obtaining a plurality of stored medical imaging data sets each representing at least a part of a respective patient, wherein a first one of the plurality of stored medical imaging data sets represents a different part of the patient body than a second one of the plurality of stored medical imaging data sets;
estimating a plurality of landmarks in the stored medical imaging data sets;
aligning each of the stored medical imaging data set to a predefined pose using the plurality of landmarks, the aligning resulting in a plurality of aligned medical imaging data sets;
sampling a plurality of points in the aligned medical imaging data sets;
machine training the machine learning system based on at least the plurality of points to configure parameters of the machine learning system; and
storing at least the configured parameters of the machine learning system.
2. The computer-implemented method according to claim 1 , wherein the stored medical imaging data set is at least one of: computed tomography data; magnetic resonance imaging data; x-ray imaging data; or ultrasound imaging data.
3. The computer-implemented method according to claim 1 , wherein sampling the plurality of points comprises for each of the plurality of points, determining a distance to a closest surface represented in the medical imaging data set.
4. The computer-implemented method according to claim 3 , wherein training the machine learning system comprises adjusting at least one body representation, based on at least one of the plurality of points and corresponding distance to the closest surface.
5. The computer-implemented method according to claim 4 , wherein the at least one body representation is an n-dimensional vector.
6. The computer-implemented method according to claim 1 , wherein estimating the plurality of landmarks comprises applying at least one landmark detection algorithm to the stored medical imaging data set, the landmark detection algorithm being a machine learning algorithm trained to identify one or more landmarks of a body.
7. The computer-implemented method according to claim 1 , wherein aligning the stored medical imaging data set, comprises:
identifying at least one first medical data landmark in a first medical imaging data set;
identifying at least one second medical data landmark in a second medical imaging data set, wherein the first medical data landmark and the second medical data landmark represent corresponding landmarks; and
aligning the first medical imaging data set and the second medical imaging data set based on the first medical data landmark and the second medical data landmark.
8. The computer-implemented method according to claim 1 , wherein the machine learning system comprises at least one neural network.
9. A computer-implemented method of inferring a body representation, the computer-implemented method comprising:
receiving image data captured using an imaging modality representing at least a part of a patient;
estimating a plurality of landmarks in the image data;
aligning the image data to a predefined pose using the plurality of landmarks, the aligning resulting in aligned image data;
sampling a plurality of points in the aligned image data;
analyzing the plurality of points using a trained machine learning system, the trained machine learning system being configured to represent a patient body;
inferring the body representation based on the analysis; and
outputting the inferred body representation.
10. The computer-implemented method according to claim 9 , wherein the image data is captured by an imaging apparatus, the image data being at least one of:
computed tomography data; magnetic resonance imaging data; x-ray imaging data;
ultrasound data;
point cloud data; or depth data.
11. The computer-implemented method according to claim 9 , wherein sampling the plurality of points comprises, for each of the plurality of points, determining a distance to a closest surface represented in the image data, thereby determining a plurality of distances.
12. The computer-implemented method according to claim 11 , wherein inferring the body representation comprises iteratively providing a body representation and the plurality of points to a trained machine learning system, and comparing the respective outputs of the machine learning system with the plurality of distances associated with the plurality of points.
13. The computer-implemented method according to claim 12 , further comprising adjusting the randomized body representation based on the comparison between the respective outputs and the plurality of distances associated with the plurality of points.
14. The computer-implemented method according to claim 13 , wherein aligning the image data comprises:
identifying at least one first data landmark in the received image data;
identifying at least one second data landmark data associated with the predefined pose, wherein the first data landmark and the second data landmark represent corresponding landmarks; and
aligning the received image data and the predefined pose based on the first data landmark and the second data landmark.
15. The computer-implemented method according to claim 9 , wherein the body representation is an n-dimensional vector.
16. The computer-implemented method according to claim 9 , wherein estimating the plurality of landmarks comprises applying at least one landmark detection algorithm to the medical imaging data, the landmark detection algorithm being a machine learning algorithm trained to identify one or more landmarks of a body.
17. The computer-implemented method according to claim 9 , wherein aligning the image data comprises:
identifying at least one first data landmark in the received image data;
identifying at least one second data landmark data associated with the predefined pose, wherein the first data landmark and the second data landmark represent corresponding landmarks; and
aligning the received image data and the predefined pose based on the first data landmark and the second data landmark.
18. A system for inferring a body representation, the system comprising:
an imaging apparatus configured to capture image data;
at least one processor configured to:
estimate a plurality of landmarks in the image data;
align the image data to a predefined pose using the plurality of landmarks, the alignment resulting in aligned image data;
sample a plurality of points in the aligned image data;
analyze the plurality of points using a trained machine learning system, the trained machine learning system being configured to represent a patient's entire body;
infer the body representation based on the analysis; and
output the inferred body representation.
19. The system according to claim 18 , wherein the imaging apparatus comprises:
a computed tomography system;
a magnetic resonance imaging system;
an x-ray imaging system; or
an ultrasound system.