Creation method of trained model, image generation method, and image processing device
In a creation method of a trained model, a reconstructed image ( 60 ) obtained by reconstructing three-dimensional X-ray image data ( 80 ) is generated. A projection image ( 61 ) is generated from a three-dimensional model of an image element ( 50 ) by a simulation. The projection image is superimposed on the reconstructed image to generate a superimposed image ( 67 ). A trained model ( 40 ) is created by performing machine learning using the superimposed image, and the reconstructed image or the projection image.
1. A creation method of a trained model, the method comprising:
generating a reconstructed image obtained by reconstructing three-dimensional X-ray image data into a two-dimensional projective image;
generating a two-dimensional projection image from a three-dimensional model of an image element, which is an extraction target, by a simulation;
superimposing the projection image of the image element on the reconstructed image to generate a superimposed image; and
creating a trained model that performs processing of extracting a plurality of the image element included in an input image and generating a plurality of extraction images which the plurality of the image element are reflected therein, by performing machine learning using the superimposed image as teacher input data and the reconstructed image or the projection image as teacher output data.
2. The creation method of a trained model according to claim 1 ,
wherein a plurality of the superimposed images are created for each of a plurality of image elements different from each other, and
the plurality of image elements include
a first element, which is a biological tissue, and a second element, which is a non-biological tissue, or
at least a plurality of image elements of a bone, a blood vessel, a device introduced into a body, clothing, a noise, and a scattered ray component of X-rays.
3. The creation method of a trained model according to claim 1 ,
wherein the image element includes a device having a linear shape or a tubular shape, and
the projection image of the image element is generated by simulating a shape of a three-dimensional model of the device with a curve generated based on a random coordinate value.
4. The creation method of a trained model according to claim 1 ,
wherein the image element includes a blood vessel, and
the projection image of the image element is generated by a simulation that randomly changes a shape of a three-dimensional model of the blood vessel.
5. The creation method of a trained model according to claim 1 ,
wherein the image element includes a scattered ray component of X-rays, and
the projection image of the image element is generated by a Monte Carlo simulation that models an imaging environment of the input image.
6. The creation method of a trained model according to claim 5 ,
wherein a plurality of the projection images of the image element are generated
by changing a projection angle over a projection angle range capable of being imaged by an X-ray imaging device in an imaging environment model, or
by changing an energy spectrum of a virtual radiation source in an imaging environment model.
7. The creation method of a trained model according to claim 1 ,
wherein the machine learning includes inputting the teacher input data and the teacher output data created for each image element to one learning model, and
the trained model is configured to extract the plurality of image elements from the input image without duplication, and output the extracted plurality of image elements and a residual image element remaining after extraction, respectively.
8. An image generation method comprising:
separately generating a plurality of extraction images by a plurality of image elements from an X-ray image using a trained model in which processing of extracting an image element from an input image has been learned; and
generating a processed image in which image processing is performed on each image element included in the X-ray image, by weighting each of the plurality of extraction images and adding or subtracting the weighted plurality of extraction images to or from the X-ray image.
9. The image generation method according to claim 8 ,
wherein the image processing includes enhancement processing or removal processing.
10. The image generation method according to claim 8 ,
wherein the plurality of image elements include
a first element, which is a biological tissue, and a second element, which is a non-biological tissue, or
at least a plurality of image elements of a bone, a blood vessel, a device introduced into a body, clothing, a noise, and a scattered ray component of X-rays.
11. The image generation method according to claim 8 ,
wherein the image processing is performed separately on a part or all of the plurality of extraction images, and
the processed image is generated by the plurality of extraction images after the image processing, and the X-ray image.
12. The image generation method according to claim 8 ,
wherein the trained model is configured to extract the plurality of image elements from the input image without duplication, and output the extracted plurality of image elements and a residual image element remaining after extraction, respectively.
13. The image generation method according to claim 8 ,
wherein the trained model is created in advance through machine learning using a reconstructed image obtained by reconstructing three-dimensional image data into a two-dimensional projective image, and a projection image created from a three-dimensional model of the image element by a simulation.
14. An image processing device comprising:
an image acquisition unit that acquires an X-ray image;
an extraction processing unit that separately generates a plurality of extraction images by a plurality of image elements from the X-ray image using a trained model in which processing of extracting an image element from an input image has been learned; and
an image generation unit that generates a processed image in which image processing is performed on each image element included in the X-ray image, by weighting each of the plurality of extraction images and adding or subtracting the weighted plurality of extraction images to or from the X-ray image.
15. A creation method of a trained model, the method comprising:
generating a reconstructed image obtained by reconstructing three-dimensional X-ray image data into a two-dimensional projective image;
generating a two-dimensional projection image by simulating a shape of a three-dimensional model of an image element of a linear or tubular shape curved device, which is an extraction target, with a curve;
superimposing the projection image of the image element on the reconstructed image to generate a superimposed image; and
creating a trained model used in a process of extracting the image element included in an input image, by performing machine learning using the superimposed image as teacher input data and the reconstructed image or the projection image as teacher output data.