Scatter estimation for PET from image-based convolutional neural network
A method, system, and computer readable medium to perform nuclear medicine scatter correction estimation, sinogram estimation and image reconstruction from emission and attenuation correction data using deep convolutional neural networks. In one embodiment, a Deep Convolutional Neural network (DCNN) is used, although multiple neural networks can be used (e.g., for angle-specific processing). In one embodiment, a scatter sinogram is directly estimated using a DCNN from emission and attenuation correction data. In another embodiment a DCNN is used to estimate a scatter-corrected image and then the scatter sinogram is computed by a forward projection.
1 . A nuclear medicine diagnosis apparatus, comprising:
processing circuitry configured to:
receive first attenuation correction data of a subject and a first nuclear medicine image of the subject, wherein the first nuclear medicine image includes a scatter effect, and
output a scatter sinogram indicating the scatter effect in the first nuclear medicine image by inputting both the received first attenuation correction data and the received first nuclear medicine image to a trained neural network, which outputs the scatter sinogram,
wherein the trained neural network was trained by (1) inputting, as input data sets, (1a) attenuation correction training data and (1b) training nuclear medicine images that include the scatter effect, and (2) inputting a corresponding set of scatter sinogram training outputs with reduced scatter effects as compared with the training nuclear medicine images, the scatter sinogram training outputs being generated using a scatter estimation method different from the neural network.
2 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the trained neural network comprises at least one deep convolutional neural network (DCNN).
3 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the trained neural network comprises at least one deep convolutional neural network (DCNN) trained using supervised learning.
4 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the trained neural network comprises at least one deep convolutional neural network (DCNN) trained using supervised learning using a similarity metric.
5 . The nuclear medicine diagnosis apparatus according to claim 4 , wherein the similarity metric comprises at least one of a root-mean-square error, weighted sums of intensity differences, a cross correlation, an adversarial loss, and mutual information between image histograms.
6 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the processing circuitry is further configured to reconstruct a second nuclear medicine image by processing the scatter sinogram indicating the estimated scatter in the first nuclear medicine image.
7 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the set of scatter sinogram training outputs with reduced scatter effects as compared with the training nuclear medicine images are based on corresponding Monte Carlo simulations.
8 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the set of scatter sinogram training outputs with reduced scatter effects as compared with the training nuclear medicine images are based on model-based scatter corrections.
9 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the attenuation correction training data is obtained from helical computed tomography (CT) scans.
10 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the first nuclear medicine image of the subject is converted to a lower resolution prior to input into the trained neural network.
11 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the nuclear medicine diagnosis apparatus is a PET scanner.
12 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the processing circuitry is further configured to:
produce a difference image from (a) a scatter-corrected image with reduced scatter as compared with the first nuclear medicine image, and (b) the first nuclear medicine image; and
produce a sinogram image by forward projecting the produced difference image.
13 . A method of producing a scatter-based output, the method comprising:
receiving, by a nuclear medicine diagnosis apparatus, a first attenuation correction data of a subject and a first nuclear medicine image of the subject, wherein the first nuclear medicine image includes a scatter effect, and
outputting, by the nuclear medicine diagnosis apparatus a scatter sinogram indicating the scatter effect in the first nuclear medicine image by inputting both the received first attenuation correction data and the received first nuclear medicine image to a trained neural network, wherein the trained neural network was trained by (1) inputting, as input data sets, (1a) attenuation correction training data and (1b) training nuclear medicine images that include the scatter effect, and (2) inputting a corresponding set of scatter sinogram training outputs with reduced scatter effects as compared with the training nuclear medicine images, the scatter sinogram training outputs being generated using a scatter estimation method different form the neural network.
14 . The method of claim 13 wherein the trained neural network comprises at least one deep convolutional neural network (DCNN).
15 . The method of claim 13 wherein the trained neural network comprises at least one deep convolutional neural network (DCNN) trained using supervised learning using a similarity metric.
16 . A non-transitory computer-readable medium having instructions stored therein that, when executed by one or more processors, cause the one or more processors to:
receive a first attenuation correction data of a subject and a first nuclear medicine image of the subject, wherein the first nuclear medicine image includes a scatter effect, and
output a scatter sinogram indicating the scatter effect in the first nuclear medicine image by inputting both the received first attenuation correction data and the received first nuclear medicine image to a trained neural network, wherein the trained neural network was trained by (1) inputting, as input data sets, (1a) attenuation correction training data and (1b) training nuclear medicine images that include the scatter effect, and (2) inputting a corresponding set of scatter sinogram training outputs with reduced scatter effects as compared with the training nuclear medicine images, the scatter sinogram training outputs being generated using a scatter estimation method different form the neural network.
17 . The non-transitory computer-readable medium of claim 16 having further instructions stored therein to train at least one deep convolutional neural network (DCNN) using supervised learning.
18 . The nuclear medicine diagnosis apparatus according to claim 1 , wherein the processing circuitry is further configured to determine a network that produces a scatter distribution that minimizes a cost function to train the trained neural network.