Machine learning model, program, ultrasound diagnostic apparatus, ultrasound diagnostic system, image processing apparatus, and training apparatus
Techniques for efficiently generating time-varying image data for use in training a machine learning model are disclosed. An aspect of the present disclosure relates to a machine learning model trained using training data that includes at least one piece of training time-varying image data of second time-varying image data and third time-varying image data, the second time-varying image data being obtained by standardizing first time-varying image data in a time direction, the first time-varying image data being based on a reception signal for image generation received by an ultrasound probe, third time-varying image data being based on the second time-varying image data, and, and ground truth data including a detection target corresponding to the at least one piece of training time-varying image data.
1 . A method of training a machine learning model comprising the steps of:
obtaining first time-varying image data based on a reception signal for image generation received by an ultrasound probe and label data as ground truth data including a detection target;
generating second time-varying image data by standardizing first time-varying image data in a time direction based on one or both of a sweep speed and a heart rate per predetermined time when the first time-varying image data is generated, or third time-varying image data by deforming and/or further standardizing the second time varying image data in the time direction; and
using the second time-varying image data or the third time-varying image data as training time-varying image data to train the machine learning model.
2 . The method of training the machine learning model according to claim 1 , wherein the detection target is information on a feature that is affected by a time change in a time-varying image.
3 . The method of training the machine learning model according to claim 1 , wherein the detection target is at least one piece of information selected from information on a position of at least one point in a time-varying image, information on a width, information on a shape of a waveform, information on timing, information on a time segment, and information on a trace.
4 . The method of training the machine learning model according to claim 1 , wherein the third time-varying image data is data obtained by standardizing the second time-varying image data in the time direction.
5 . The method of training the machine learning model according to claim 1 , wherein the third time-varying image data is data obtained by deforming the second time-varying image data in the time direction.
6 . The method of training the machine learning model according to claim 1 , wherein the second time-varying image data is data obtained by standardizing the first time-varying image data in the time direction based on information that affects a time-varying image in the time direction when the first time-varying image data is generated.
7 . The method of training the machine learning model according to claim 1 , wherein the training time-varying image data is time-varying image data cut out to have a predetermined image width or time-varying image data pasted on background image data having a predetermined image width.
8 . The method of training the machine learning model according to claim 1 , wherein:
the first time-varying image data is a Doppler image; and
the detection target is one of the following
i) a segment in the time direction of a spectral waveform that is a target of velocity time integral (VTI) measurement or a region of the spectral waveform,
ii) positions or timing on a time axis of peak systolic velocity (PSV) and end diastolic velocity (EDV),
iii) a boundary of a segment in the time direction of a spectral waveform that is a target of blood flow volume measurement, and
iv) positions or timing on the time axis of the blood flow velocities of an E wave and an A wave in E/A measurement.
9 . The method of training the machine learning model according to claim 1 , wherein
the first time-varying image data is an M-mode image; and
the detection target is one of the following
i) diastolic and systolic positions of an annulus in tricuspid annular plane systolic excursion (TAPSE) measurement or mitral annular plane systolic excursion (MAPSE) measurement, diastolic and systolic timing on a time axis, or a trace of the annulus, and
ii) a blood vessel diameter of an inferior vena cava, timing of expiration and inspiration, or traces of an anterior wall and a posterior wall of the inferior vena cava, in M-mode inferior vena cava (IVC) diameter measurement.
10 . The method of training the machine learning model according to claim 1 , wherein the step of using the second time-varying image data or the third time-varying image data comprises using ground truth data including a detection target corresponding to the at least one piece of training time-varying image data.
11 . A non-transitory computer-readable storage medium storing a program for causing a hardware processor of a computer to, by using the machine learning model trained according to the method of claim 1 , implement an output function of outputting, as an inference result, the detection target from second prediction target time-varying image data obtained by standardizing first prediction target time-varying image data in a time direction based on one or both of a sweep speed and a heart rate per predetermined time when the first prediction time-varying image data is generated, the first prediction target time-varying image data being based on the reception signal for image generation received by an ultrasound probe.
12 . The non-transitory computer-readable storage medium according to claim 11 , wherein the detection target output as the inference result is information on a feature affected by a time change in a time-varying image.
13 . The non-transitory computer-readable storage medium according to claim 11 , wherein the detection target output as the inference result is at least one piece of information selected from information on a position of at least one point in a time-varying image, information on a width, information on a shape of a waveform, information on timing, information on a time segment, and information on a trace.
14 . The non-transitory computer-readable storage medium according to claim 11 , wherein the second prediction target time-varying image data is data obtained by standardizing the first prediction target time-varying image data in the time direction based on information that affects a time-varying image in the time direction when the first prediction target time-varying image data is generated.
15 . The non-transitory computer-readable storage medium according to claim 11 , wherein the second prediction target time-varying image data is time-varying image data cut out to have a predetermined image width or time-varying image data pasted on background image data having a predetermined image width.
16 . The non-transitory computer-readable storage medium according to claim 11 , wherein:
the first prediction target time-varying image data is a Doppler image; and
the detection target output as the inference result is one of the following
i) a segment in the time direction of a spectral waveform that is a target of velocity time integral (VTI) measurement or a region of the spectral waveform,
ii) positions or timing on a time axis of peak systolic velocity (PSV) and end diastolic velocity (EDV),
iii) a boundary of a segment in the time direction of a spectral waveform that is a target of blood flow volume measurement, and
iv) positions or timing on the time axis of the blood flow velocities of an E wave and an A wave in E/A measurement.
17 . The non-transitory computer-readable storage medium according to claim 11 , wherein:
the first prediction target time-varying image data is an M-mode image; and
the detection target is one of the following
i) diastolic and systolic positions of an annulus in tricuspid annular plane systolic excursion (TAPSE) measurement or mitral annular plane systolic excursion (MAPSE) measurement, diastolic and systolic timing on a time axis, or a trace of the annulus, and
ii) a blood vessel diameter of an inferior vena cava, timing of expiration and inspiration, or traces of an anterior wall and a posterior wall of the inferior vena cava, in M-mode inferior vena cava (IVC) diameter measurement.
18 . An ultrasound diagnostic apparatus, comprising:
an ultrasound probe that transmits and receives an ultrasonic wave to and from a subject; and
a hardware processor that, by using the machine learning model trained according to the method of claim 1 , outputs the detection target as an inference result from the second prediction target time-varying image data, the second prediction target time-varying image data being obtained by standardizing first prediction target time-varying image data in a time direction based on one or both of a sweep speed and a heart rate per predetermined time when the first prediction time-varying image data is generated, the first prediction target time-varying image data being based on a reception signal received by the ultrasound probe.
19 . An ultrasound diagnostic system, comprising:
a database that stores the machine learning model trained according to the method of claim 1 ;
an ultrasound probe that transmits and receives an ultrasonic wave to and from a subject; and
a hardware processor that, by using the machine learning model, outputs the detection target as an inference result from the second prediction target time-varying image data, the second prediction target time-varying image data being obtained by standardizing first prediction target time-varying image data in a time direction based on one or both of a sweep speed and a heart rate per predetermined time when the first prediction time-varying image data is generated, the first prediction target time-varying image data being based on a reception signal received by the ultrasound probe.
20 . An image processing apparatus comprising:
a hardware processor that, by using the machine learning model trained according to the method of claim 1 , outputs the detection target as an inference result from the second prediction target time-varying image data, the second prediction target time-varying image data being obtained by standardizing first prediction target time-varying image data in a time direction based on one or both of a sweep speed and a heart rate per predetermined time when the first prediction time-varying image data is generated, the first prediction target time-varying image data being based on a reception signal received by an ultrasound probe.
21 . A training apparatus for training a machine learning model by using training data that comprises the following data:
at least one piece of training time-varying image data of second time-varying image data and third time-varying image data, the second time-varying image data being obtained by standardizing first time-varying image data in a time direction, the first time-varying image data being based on a reception signal for image generation received by an ultrasound probe, and the third time-varying image data by deforming and/or further standardizing the second time-varying image data in the time direction, and
ground truth data including a detection target corresponding to the at least one piece of training time-varying image data.