Method and apparatus for acquiring labeled data, training method and apparatus, and medical equipment
Provided is a method for acquiring labeled data, including acquiring a 3D-3D registration sample, wherein the 3D-3D registration sample includes a sample 3D image, a reference 3D image and 3D-3D registration data, and the 3D-3D registration data includes correspondence data between the sample 3D image and the reference 3D image; generating at least one dual-2D-image based on the sample 3D image, wherein each dual-2D-image is a combination of 2D projection images of the sample 3D image in two different directions; and generating 2D-3D registration data corresponding to each dual-2D-image based on the 3D-3D registration data, wherein each 2D-3D registration data includes correspondence data between the dual-2D-image and the reference 3D image.
1. A method for acquiring labeled data, comprising:
acquiring a 3D-3D registration sample, wherein the 3D-3D registration sample comprises a sample 3D image, a reference 3D image and 3D-3D registration data, and the 3D-3D registration data comprises correspondence data between the sample 3D image and the reference 3D image;
generating at least one dual-2D-image based on the sample 3D image, wherein each dual-2D-image is a combination of 2D projection images of the sample 3D image in two different directions; and
generating 2D-3D registration data corresponding to each dual-2D-image based on the 3D-3D registration data, wherein each 2D-3D registration data comprises correspondence data between the dual-2D-image and the reference 3D image;
wherein acquiring the 3D-3D registration sample comprises:
acquiring the reference 3D image and at least one set of sample 2D images, wherein each set of sample 2D images comprises a number of 2D images acquired by imaging a same object in a number of different directions respectively;
reconstructing each set of sample 2D images into one sample 3D image respectively; and
acquiring the correspondence data between each sample 3D image and the reference 3D image by a 3D-3D image registration; and
wherein generating at least one dual-2D-image based on the sample 3D image comprises combining, in each set of sample 2D images, every two 2D images having a predetermined angular interval into the dual-2D-image.
2. The method according to claim 1 , wherein: the 3D-3D registration data comprises coordinates of a number of positioning points which are positioned in the sample 3D image, and the 2D-3D registration data comprises coordinates of the number of positioning points in each 2D projection image of the dual-2D-image; and generating the 2D-3D registration data corresponding to each set of the dual-2D-images based on the 3D-3D registration data comprises:
converting the coordinates of the number of positioning points in the sample 3D image into the coordinates of the number of positioning points in the each 2D projection image of the dual-2D-image, based on a projection relationship between any dual-2D-image and the sample 3D image.
3. The method according to claim 1 , wherein two 2D projection images of each dual-2D-image have a same image form as that of a computed tomography image, an X-ray fluoroscopy image, a nuclear magnetic resonance image, an ultrasonic image or a fluorescence fluoroscopy image.
4. A training method, comprising:
acquiring a number of labeled data by the method as defined in claim 1 , wherein each of the labeled data comprises one dual-2D-image and 2D-3D registration data corresponding to the dual-2D-image; and
training a machine learning model with the number of labeled data, to determine model parameters for 2D-3D image registration,
wherein input of the machine learning model comprises the dual-2D-image, output of the machine learning model comprises the 2D-3D registration data, and the number of labeled data are used as at least one of training data or test data during training.
5. The training method according to claim 4 , wherein training the machine learning model with the number of labeled data to determine the model parameters for the 2D-3D image registration comprises:
optimizing the model parameters acquired based on the training data with the test data.
6. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program comprising a program instruction, and the program instruction, when being executed by a processor, causes the processor to execute the method as defined in claim 4 .
7. Medical equipment comprising a processor and a memory, wherein a program instruction is stored in the memory, and the processor is configured to call the program instruction in the memory to execute the method as defined in claim 4 .
8. A training apparatus, comprising a processor and a memory storing a program instruction which is executable by the processor, wherein the processor is configured to:
acquire a number of labeled data by the method as defined in claim 1 , wherein each of the labeled data comprises one dual-2D-image and 2D-3D registration data corresponding to the dual-2D-image; and
train a machine learning model with the number of labeled data, to determine model parameters for 2D-3D image registration,
wherein input of the machine learning model comprises the dual-2D-image, output of the machine learning model comprises the 2D-3D registration data, and the number of labeled data are used as at least one of training data or test data during training.
9. The training apparatus according to claim 8 , wherein the processor is further configured to optimize the model parameters acquired based on the training data with the test data.
10. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program comprising a program instruction, and the program instruction, when being executed by a processor, causes the processor to execute the method as defined in claim 1 .
11. An apparatus for acquiring labeled data, comprising a processor and a memory storing a program instruction which is executable by the processor, wherein the processor is configured to:
acquire a 3D-3D registration sample, wherein the 3D-3D registration sample comprises a sample 3D image, a reference 3D image and 3D-3D registration data, and the 3D-3D registration data comprises correspondence data between the sample 3D image and the reference 3D image;
generate at least one dual-2D-image based on the sample 3D image, wherein each dual-2D-image is a combination of 2D projection images of the sample 3D image in two different directions; and
generate 2D-3D registration data corresponding to each dual-2D-image based on the 3D-3D registration data, wherein each 2D-3D registration data comprises correspondence data between the dual-2D-image and the reference 3D image;
wherein the processor is further configured to:
acquire the reference 3D image and at least one set of sample 2D images, wherein each set of sample 2D images comprises a number of 2D images acquired by imaging a same object in a number of different directions respectively;
reconstruct each set of sample 2D images into one sample 3D image respectively;
acquire the correspondence data between each sample 3D image and the reference 3D image by a 3D-3D image registration; and
combine, in each set of sample 2D images, every two 2D images having a predetermined angular interval into the dual-2D-image.
12. The apparatus according to claim 11 , wherein: the 3D-3D registration data comprises coordinates of a number of positioning points which are positioned in the sample 3D image, and the 2D-3D registration data comprises coordinates of the number of positioning points in each 2D projection image of the dual-2D-image; and the processor is further configured to:
convert the coordinates of the number of positioning points in the sample 3D image into the coordinates of the number of positioning points in each 2D projection image of the dual-2D-image, based on a projection relationship between any dual-2D-image and the sample 3D image.
13. The apparatus according to claim 11 , wherein two 2D projection images of each dual-2D-image have a same image form as that of a computed tomography image, an X-ray fluoroscopy image, a nuclear magnetic resonance image, an ultrasonic image or a fluorescence fluoroscopy image.
14. Medical equipment comprising a processor and a memory, wherein a program instruction is stored in the memory, and the processor is configured to call the program instruction in the memory to execute a method for acquiring labeled data comprising:
acquiring a 3D-3D registration sample, wherein the 3D-3D registration sample comprises a sample 3D image, a reference 3D image and 3D-3D registration data, and the 3D-3D registration data comprises correspondence data between the sample 3D image and the reference 3D image;
generating at least one dual-2D-image based on the sample 3D image, wherein each dual-2D-image is a combination of 2D projection images of the sample 3D image in two different directions; and
generating 2D-3D registration data corresponding to each dual-2D-image based on the 3D-3D registration data, wherein each 2D-3D registration data comprises correspondence data between the dual-2D-image and the reference 3D image;
wherein acquiring the 3D-3D registration sample comprises:
acquiring the reference 3D image and at least one set of sample 2D images, wherein each set of sample 2D images comprises a number of 2D images acquired by imaging a same object in a number of different directions respectively;
reconstructing each set of sample 2D images into one sample 3D image respectively; and
acquiring the correspondence data between each sample 3D image and the reference 3D image by a 3D-3D image registration; and
wherein generating at least one dual-2D-image based on the sample 3D image comprises combining, in each set of sample 2D images, every two 2D images having a predetermined angular interval into the dual-2D-image.
15. The medical equipment according to claim 14 , wherein: the 3D-3D registration data comprises coordinates of a number of positioning points which are positioned in the sample 3D image, and the 2D-3D registration data comprises coordinates of the number of positioning points in each 2D projection image of the dual-2D-image; and generating the 2D-3D registration data corresponding to each set of the dual-2D-images based on the 3D-3D registration data comprises:
converting the coordinates of the number of positioning points in the sample 3D image into the coordinates of the number of positioning points in each 2D projection image of the dual-2D-image, based on a projection relationship between any dual-2D-image and the sample 3D image.