Deep residual inception encoder-decoder network for amyloid PET harmonization
View Patent ↗Using deep learning, imaging harmonization among images produced with differing PET tracers is achieved. A method may include providing an original PET image of the brain using an original PET tracer, providing a target PET tracer, and converting, using a deep learning neural network, the original PET image into a target PET image simulating the image that would be obtained had the target PET tracer been used.
1. An image harmonization method for a brain, the method comprising:
providing an original PET image of the brain using an original PET tracer;
providing a target PET tracer; and
converting, using a deep learning neural network, the original PET image into a target PET image simulating an image that would be obtained had the target PET tracer been used,
wherein the deep learning neural network comprises a U-Net-like architecture,
wherein the deep learning neural network comprises at least one residual block,
wherein the at least one residual block comprises one or more encoding blocks and one or more decoding blocks, and
wherein each encoding block receives a first input matrix, and each encoding block comprises:
a first convolutional path comprising:
a first convolutional layer comprising a first kernel and a first filter bank, wherein the first convolutional layer receives the first input matrix and generates a first output matrix, and
a second convolutional layer comprising a second kernel and a second filter bank, wherein the second convolutional layer receives the first output matrix as an input and generates a second output matrix; and
a second convolutional path comprising a third kernel, wherein the second convolutional path comprises a third filter bank and generates a third output matrix,
wherein each encoding block (i) sums together the second output matrix and the third output matrix resulting in a first summed matrix, (ii) down-samples the first summed matrix, and (iii) generates a fourth output matrix.
2. The method of claim 1 , wherein:
the first kernel comprises a 3 by 3 kernel;
the second kernel comprises a 3 by 3 kernel;
the third kernel comprises a 1 by 1 kernel;
the first, second, and third filter banks each comprise a 32-channel filter bank;
the first input matrix comprises a 2D slice of the original PET image and comprises a 256 by 256 matrix;
the first output matrix comprises a 256 by 256 by 32 matrix;
the second output matrix comprises a 256 by 256 by 32 matrix;
the third output matrix comprises a 256 by 256 by 32 matrix;
the first summed matrix comprises a 256 by 256 by 32 matrix;
the first summed matrix is down-sampled using a rectified linear unit (ReLu) function; and
the fourth output matrix comprises a 128 by 128 by 32 matrix.
3. The method of claim 1 , wherein the at least one residual block comprises five encoding blocks and four decoding blocks.
4. The method of claim 1 , wherein the original PET tracer comprises one of the [C11]-Pittsburgh compound (PIB), florbetapir (FBP), florbetaben (FBB), flutemetamol (FTE), or NAV4694 (NAV).
5. The method of claim 1 , wherein the target PET tracer comprises one of the [C11]-Pittsburgh compound (PIB), florbetapir (FBP), florbetaben (FBB), flutemetamol (FTE), or NAV4694 (NAV).
6. The method of claim 1 , wherein the original PET image comprises a two-dimensional image.
7. The method of claim 1 , wherein the target PET image comprises a two-dimensional image.
8. The method of claim 6 , wherein the original PET image has a size of 256 pixels by 256 pixels.
9. The method of claim 7 , wherein the target PET image has a size of 256 pixels by 256 pixels.
10. The method of claim 1 , wherein the original PET image comprises a three-dimensional image.
11. The method of claim 1 , wherein the target PET image comprises a three-dimensional image.
12. The method of claim 10 , wherein the original PET image is converted into a series of original two-dimensional images, each original two-dimensional image being converted into a target two-dimensional image, and the target two-dimensional images are recombined to form the target PET image.
13. The method of claim 12 , wherein the series of original two-dimensional images and the target two-dimensional images each comprise coronal, sagittal, and axial views.
14. An image harmonization method for a brain, the method comprising:
providing an original PET image of the brain using an original PET tracer;
providing a target PET tracer; and
converting, using a deep learning neural network, the original PET image into a target PET image simulating an image that would be obtained had the target PET tracer been used,
wherein the deep learning neural network comprises a U-Net-like architecture,
wherein the deep learning neural network comprises at least one residual block,
wherein the at least one residual block comprises one or more encoding blocks and one or more decoding blocks, and
wherein each decoding block receives a second input matrix from a corresponding encoding block and a third input matrix from another decoding block, concatenates the second input matrix with the third input matrix to form a concatenated input matrix, each decoding block comprising:
a third convolutional path comprising:
a third convolutional layer comprising a fourth kernel and a fourth filter bank, wherein the third convolutional layer receives the concatenated input matrix and generates a fifth output matrix, and
a fourth convolutional layer comprising a fifth kernel and a fifth filter bank, wherein the fourth convolutional layer receives the fifth output matrix and generates a sixth output matrix;
a fourth convolutional path comprising a sixth filter bank, wherein the fourth convolutional path receives the concatenated input matrix and generates a seventh output matrix; and
a sixth kernel,
wherein each decoding block (i) sums together the sixth output matrix and the seventh output matrix resulting in a second summed matrix, (ii) processes the second summed matrix using the sixth kernel, and (iii) generates an eighth output matrix.
15. The method of claim 14 , wherein:
the fourth kernel comprises a 3 by 3 kernel;
the fifth kernel comprises a 3 by 3 kernel;
the sixth kernel comprises a 1 by 1 kernel;
the fourth, fifth, and sixth filter banks each comprise a 32-channel filter bank;
the second input matrix comprises a 256 by 256 by 32 matrix;
the third input matrix comprises a 256 by 256 by 32 matrix;
the concatenated input matrix comprises a 256 by 256 by 64 matrix;
the fifth output matrix comprises a 256 by 256 by 32 matrix;
the sixth output matrix comprises a 256 by 256 by 32 matrix;
the seventh output matrix comprises a 256 by 256 by 32 matrix;
the second summed matrix comprises 256 by 256 by 32 matrix; and
the eighth output matrix comprises a 256 by 256 matrix and comprises a 2D slice of the target PET image.
16. The method of claim 14 , wherein the at least one residual block comprises five encoding blocks and four decoding blocks.
17. The method of claim 14 , wherein the original PET tracer comprises one of the [C11]-Pittsburgh compound (PIB), florbetapir (FBP), florbetaben (FBB), flutemetamol (FTE), or NAV4694 (NAV).
18. The method of claim 14 , wherein the target PET tracer comprises one of the [C11]-Pittsburgh compound (PIB), florbetapir (FBP), florbetaben (FBB), flutemetamol (FTE), or NAV4694 (NAV).