Image registration device, image registration method, and ultrasonic diagnosis apparatus having image registration device
There is provided an image registration device and an image registration method. The device includes: a feature extractor configured to extract, from a first image, a first feature group and to extract, from a second image, a second feature group; a feature converter configured to convert, using a converted neural network in which a correlation between features is learned, the extracted second feature group to correspond to the extracted first feature group, to obtain a converted group; and a register configured to register the first image and the second image based on the converted group and the extracted first feature group.
1. An apparatus comprising:
a first artificial neural network:
a second artificial neural network; and
at least one processor configured to:
perform unsupervised training on the first artificial neural network based on a plurality of images obtained by a medical device having a first modality, to thereby provide a trained first artificial neural network,
perform unsupervised training on the second artificial neural network based on a plurality of images obtained by a medical device having a second modality different from the first modality, to thereby provide a trained second artificial neural network,
use the trained first artificial neural network to extract a feature form a first medical image obtained by a medical device having the first modality,
use the trained second artificial neural network to extract a feature from a second medical image obtained by a medical device having the second modality,
perform processing so that the feature extracted from the first medical image and the feature extracted from the second medical image are in a same feature space, and
register the first medical image and the second medical image having the feature extracted from the first medical image and the feature extracted from the second medical image in the same feature space.
2. The apparatus of claim 1 , wherein each of the first artificial neural network and the second artificial neural network includes a plurality of layers, each layer of the plurality of layers including units, and the units of adjacent layers of the plurality of layers being connnected to each other according to a method of a Boltzmann machine.
3. The apparatus of claim 2 , wherein
in the performing of the unsupervised training on the first artificial neural network, a connection strength or a connection type of the units of the adjacent layers of the first artificial neural network is determined, and
in the peforming of the unsupervised training on the second artificial neural network, a connection strength or a connection type of the units of the adjacent layers of the second artificial neural network is determined.
4. The apparatus of claim 3 , wherein
in the performing of the unsupervised training on the first artificial neural network, the connection strength of the units of the adjacent layers of the first artificial neural network is increased, and
in the performing of the unsupervised training on the second artificial neural network, the connection strength of the units of the adjacent layers of the second artificial neural network is increased.
5. The apparatus of claim 1 , wherein
in the performing of the unsupervised training on the first artificial neural network, the first artificial neural network is extended or decreased, and
in the performing of the unsupervised training on the second artificial neural network, the second artificial neural network is extended or decreased.
6. The apparatus of claim 5 , wherein
in the performing of the unsupervised training on the first artificial neural network, a backpropagation algorithm is used, and
in the performing of the unsupervised training on the second artificial neural network, a backpropagation algorithm is used.
7. The apparatus of claim 5 , wherein
in the performing of the unsupervised training on the first artificial neural network, an error between a patch image input to the first artificial neural network and a reconstructed image generated on the basis of a feature vector corresponding to the patch image is minimized, and
in the performing of the unsupervised training on the second artificial neural network, an error between a patch image input to the second artificial neural network and a reconstructed image generated on the basis of a feature vector corresponding to the patch image is minimized.
8. The apparatus of claim 1 , wherein the first artificial neural network and the second artificial neural network have different structures from each other.
9. The apparatus of claim 1 , wherein the unsupervised training performed on the first artificial neural network and the unsupervised training performed on the second artificial neural network are a same method.
10. An apparatus comprising:
at least one memory storing instructions; and
at least one processor that executes the instructions to:
use a first artificial neural network trained by unsupervissed training based on a plurality of images obtained by a medical device having a first modality, to extract a feature from a first medical image obtained by a medical device having the first modality,
use a second artificial neural network trained by unsupervised training based on a plurality of images obtained by a medical device having a second modality different from the first modality, to extract a feature form a second medical image obtained by a medical device having the second modality,
perform processing so that the feature extracted from the first medical image and the feature extracted from the second medical image are in a same feature space, and
register the first medical image and the second medical image having the feature extracted from the first medical image and the feature extracted from the second medical image in the same feature space.
11. An apparatus comprising:
at least one memory storing instructions; and
at least one processor that executes the instructions to:
use a first artificial neural network trained to extract features from medical images obtained by a medical device having a first modality, to extract a feature from a first medical image obtained by a medical device having the first modality,
use a second artificial neural network trained to extract features from medical images obtained by a medical device having a second modality different from the first modality, to extract a feature from a second medical image obtained by a medical device having the second modality,
perform processing so that the feature extracted from the first medical image and the feature extracted from the second medical image are in a same feature space, and
register the first medical image and the second medical image having the feature extracted from the first medical image and the feature extracted from the second medical image in the same feature space.