Whole-body anatomical digital twin from partial medical images
For generating and/or machine training to generate a whole-body representation of a patient, one or more partial-body scans or images of the patient are extrapolated to the whole-body representation of both interior and exterior anatomy. One or more machine-learned models (e.g., per-organ-group implicit generative shape models) fill the whole-body representation based on the partial information from imaging.
1 . A method for generating a whole-body representation of a patient, the method comprising:
acquiring at least one medical image representing only part of a patient, the at least one medical image representing one or more first organs, a part of one or more second organs, and skin of the patient, the at least one medical image not representing one or more third organs; and
generating a representation of a whole body of the patient from the at least one medial image representing only part of the patient, the representation of the whole body representing interior and exterior anatomy of the patient, the representation generated, at least in part, by a machine-learned model, wherein generating the representation of the whole body comprises generating the representation as including the skin, the first organs, the second organs, and the third organs.
2 . The method of claim 1 wherein acquiring comprises acquiring just the at least one medical image representing only part of the patient and wherein generating comprises generating from just the at least one medical image.
3 . The method of claim 1 wherein generating comprises generating where a part of the representation of the whole body of the patient is not represented in any information used to generate the representation of the whole body, the at least one medical image not representing the part of the representation of the whole body.
4 . The method of claim 1 wherein generating comprises generating with the machine-learned model comprising a per-organ group implicit generative shape model.
5 . The method of claim 1 wherein generating comprises generating with the machine-learned model comprising an autodecoder.
6 . The method of claim 1 wherein generating comprises optimizing a latent vector by sampling in three-dimensional space in a trained manifold.
7 . The method of claim 1 wherein generating comprises generating a shape of a same organ of the second organs multiple times as part of recursive operation and forming a representation for the same organ from the shapes of the multiple times based on recursion depth of the recursive operation.
8 . The method of claim 1 wherein generating comprises generating signed distance functions for the interior anatomy by the machine-learned model.
9 . The method of claim 1 wherein generating comprises generating by the machine-learned model outputting the representation of the whole body as a single output.
10 . The method of claim 1 wherein generating comprises reconstructing the skin as the exterior anatomy of the whole body and enforcing consistency with the skin for generating of the interior anatomy not represented in the at least one medical image.
11 . A method for generating a whole-body representation of a patient, the method comprising:
acquiring at least one medical image representing only part of a patient; and
generating a representation of a whole body of the patient from the at least one medical image representing only part of the patient, the representation of the whole body representing interior and exterior anatomy of the patient, the representation generated, at least in part, by a machine-learned model, wherein generating comprises segmenting anatomy of the part of the patient represented in the at least one medical image, the segmented anatomy used in optimization with the machine-learned model, the machine-learned model outputting information used to form part of the representation of the whole body not represented in the input.
12 . The method of claim 11 wherein segmenting comprises segmenting the anatomy as a first organ, part of a second organ, and skin, and wherein the machine-learned model outputs the information for the entire second organ in response to input of the first organ, part of the second organ, and the skin.
13 . A method for generating a whole-body representation of a patient, the method comprising:
acquiring at least one medical image representing only part of a patient; and
generating a representation of a whole body of the patient from the at least one medical image representing only part of the patient, the representation of the whole body representing interior and exterior anatomy of the patient, the representation generated, at least in part, by a machine-learned model, wherein generating comprises estimating shapes of surrounding organs to first shapes represented in the at least one medical image, the estimating occurring recursively using the machine-learned model for a first organ group and a second machine-learned model for another organ group, the shapes of the surrounding organs and the first shapes included in the representation of the whole body of the patient.
14 . The method of claim 13 wherein the machine-learned model and the second machine-learned model are in a hierarchy of models for the whole body.
15 . A medical imaging system comprising:
a medical imager configured to scan only part of a patient;
an image processor configured to form a whole-body avatar from the scan of only part of the patient, the whole-body avatar formed by a first machine-learned implicit generative shape model optimization of a latent vector to segmentations from the scan.
16 . The medical imaging system of claim 15 wherein the whole-body avatar is formed by multiple machine-learned implicit generative shape models including the first machine-learned implicit generative shape model, different ones of the multiple machine-learned implicit generative shape models forming different organs of the whole-body avatar in a hierarchy of related organs.