Learning device, learning method, operation program of learning device, teacher data generation device, machine learning model, and medical imaging apparatus
Provided is a learning device that trains a machine learning model. The learning device includes a processor and a memory, in which the processor generates a plurality of pseudo images, which are pseudo images of a human body generated based on a human body model composed of three-dimensional computer graphics data and imitate an examination subject in a state of being positioned with respect to a medical imaging apparatus, for each combination of an imaging part and an imaging direction, and trains the machine learning model using a plurality of teacher data composed of the generated pseudo image and correct answer data of the combination.
1 . A learning device that trains a machine learning model that uses, as input, a camera image obtained by imaging an examination subject in a state of being positioned with respect to a medical imaging apparatus with an optical camera, and derives an imaging part and an imaging direction of the examination subject shown in the camera image, the learning device comprising:
a processor; and
a memory connected to or built in the processor,
wherein the processor
generates a plurality of pseudo images, which are pseudo images of a human body generated based on a human body model composed of three-dimensional computer graphics data and imitate the examination subject in a state of being positioned with respect to the medical imaging apparatus, for each imaging technique that is information specified for an examination and that is defined by a combination of the imaging part and the imaging direction, and
trains the machine learning model using a plurality of teacher data composed of the generated pseudo image and correct answer data of the imaging technique.
2 . The learning device according to claim 1 ,
wherein the three-dimensional computer graphics data is attached with a modeling parameter for changing at least one of a posture or appearance of the human body model.
3 . The learning device according to claim 2 ,
wherein the modeling parameter includes at least one of physique information indicating a physique of the human body model, gender, posture information, a skin color, a hair color, a hairstyle, or clothes.
4 . The learning device according to claim 1 ,
wherein the processor is able to generate the pseudo image by rendering the human body model from a set viewpoint, and the viewpoint is changeable by a rendering parameter.
5 . The learning device according to claim 4 ,
wherein viewpoint information for setting the viewpoint includes a focal length of a virtual camera, which is virtually installed at the viewpoint, and an imaging distance, which is a distance from the virtual camera to the human body model.
6 . The learning device according to claim 1 ,
wherein, in a case in which the teacher data composed of the pseudo image and the correct answer data is defined as first teacher data,
the processor trains the machine learning model using second teacher data composed of the camera image captured by the optical camera and the correct answer data in addition to the first teacher data.
7 . The learning device according to claim 1 ,
wherein a machine learning model for physique output that uses the camera image as input and derives physique information indicating a physique of the examination subject is further trained using a plurality of teacher data for physique output composed of the pseudo image and correct answer data of physique information indicating a physique of the human body model.
8 . The learning device according to claim 1 ,
wherein the medical imaging apparatus includes at least one of a radiography apparatus or an ultrasound imaging apparatus.
9 . A learning method of training, via a computer, a machine learning model that uses, as input, a camera image obtained by imaging an examination subject in a state of being positioned with respect to a medical imaging apparatus with an optical camera, and derives an imaging part and an imaging direction of the examination subject shown in the camera image, wherein a combination of the imaging part and the imaging direction is an imaging technique that is information specified for an examination, the learning method comprising:
generating a plurality of pseudo images, which are pseudo images of a human body generated based on a human body model composed of three-dimensional computer graphics data and imitate the examination subject in a state of being positioned with respect to the medical imaging apparatus, for each imaging technique; and
training the machine learning model using a plurality of teacher data composed of the generated pseudo image and correct answer data of the imaging technique.
10 . A non-transitory computer-readable storage medium storing an operation program of a learning device, the operation program causing a computer to function as a learning device that trains a machine learning model that uses, as input, a camera image obtained by imaging an examination subject in a state of being positioned with respect to a medical imaging apparatus with an optical camera, and derives an imaging part and an imaging direction of the examination subject shown in the camera image, wherein a combination of the imaging part and the imaging direction is an imaging technique that is information specified for an examination, the operation program causing the computer to function as the learning device that
generates a plurality of pseudo images, which are pseudo images of a human body generated based on a human body model composed of three-dimensional computer graphics data and imitate the examination subject in a state of being positioned with respect to the medical imaging apparatus, for each imaging technique, and
trains the machine learning model using a plurality of teacher data composed of the generated pseudo image and correct answer data of the imaging technique.
11 . A teacher data generation device that generates teacher data used to train a machine learning model that uses, as input, a camera image obtained by imaging an examination subject in a state of being positioned with respect to a medical imaging apparatus with an optical camera, and derives an imaging part and an imaging direction of the examination subject shown in the camera image, the teacher data generation device comprising:
a processor; and
a memory connected to or built in the processor,
wherein a combination of the imaging part and the imaging direction is an imaging technique that is information specified for an examination, and wherein the processor
uses three-dimensional computer graphics data that configures a human body model for generating a pseudo image of a human body and is attached with a parameter for changing at least one of a posture or appearance of the human body model,
generates a plurality of the pseudo images in which at least one of the posture or the appearance of the human body model is different by changing the parameter, for each imaging technique, and
generates a plurality of teacher data composed of the generated plurality of pseudo images and correct answer data of the imaging technique.
12 . A non-transitory computer-readable storage medium storing a machine learning model that uses, as input, a camera image obtained by imaging an examination subject in a state of being positioned with respect to a medical imaging apparatus with an optical camera, and derives an imaging part and an imaging direction of the examination subject shown in the camera image, wherein a combination of the imaging part and the imaging direction is an imaging technique that is information specified for an examination,
wherein the machine learning model is trained using a plurality of teacher data that are composed of a pseudo image, which is a pseudo image of a human body generated based on a human body model composed of three-dimensional computer graphics data and imitates the examination subject in a state of being positioned with respect to the medical imaging apparatus, and correct answer data of the imaging technique, the plurality of teacher data being generated for each imaging technique.
13 . A medical imaging apparatus comprising:
the non-transitory computer-readable storage medium storing the machine learning model according to claim 12 .