Internal body marker prediction from surface data in medical imaging
Machine learning is used to train a network to predict the location of an internal body marker from surface data. A depth image or other image of the surface of the patient is used to determine the locations of anatomical landmarks. The training may use a loss function that includes a term to limit failure to predict a landmark and/or off-centering of the landmark. The landmarks may then be used to configure medical scanning and/or for diagnosis.
1. A method for internal body marker prediction from surface data in a medical imaging system, the method comprising:
capturing, with a sensor, an outer surface of a patient, the surface data being from the capturing of the outer surface of the patient;
generating, by an image processor, an image by a machine-learned network in response to input of the surface data to the machine-learned network, the image representing a location of the internal body marker of the patient; and
displaying, by a display device, the image,
wherein generating comprises generating with the machine-learned network having been trained with a joint loss function including an error and a penalty for centering and/or missing, and
wherein the penalty comprises a loss regularizer, a landmark coefficient, a square root of a difference between argmax of the internal body marker and a ground truth of the internal body marker, and an absolute value of a difference between the maximum of the internal body marker and a ground truth of the internal body marker.
2. The method of claim 1 wherein capturing comprises capturing with the sensor being a depth sensor.
3. The method of claim 1 wherein capturing comprises capturing with the sensor being a camera where the surface data based on optical measurements.
4. The method of claim 1 wherein capturing further comprises fitting a statistical shape model to the output of the sensor for the outer surface, the surface data comprising (a) heights of the outer surface from a table and (b) thicknesses of the patient from the fit statistical shape model.
5. The method of claim 1 wherein capturing further comprises down sampling such that each pixel of the surface data is 1 cm or greater.
6. The method of claim 1 wherein generating comprises generating with the machine-learned network comprising a fully convolutional network with an encoder and a decoder.
7. The method of claim 6 wherein the encoder has fewer convolutional layers than the decoder and wherein the fully convolutional network has fewer than twelve convolutional layers.
8. The method of claim 1 wherein generating further comprises locating a plurality of other body markers and normalizing the surface data based on the other body markers, and wherein generating comprises generating the image with the internal body marker in response to input of the normalized surface data.
9. The method of claim 1 wherein the machine-learned network was trained with sequential iteration of the error and the joint loss function, the error comprising a mean square error.
10. The method of claim 9 wherein the sequential iteration starts using the mean square error as the loss function and ends where the mean square error is not decreasing.
11. The method of claim 1 wherein generating comprises generating the image to include the internal body marker and at least ten other body markers.
12. The method of claim 1 wherein displaying comprises displaying with the internal body marker labeled.