Hybrid image reconstruction system
Generally, there is provided a hybrid image reconstruction system. The hybrid image reconstruction system includes a deep learning stage and a compressed sensing stage. The deep learning stage is configured to receive an input data set that includes measured tomographic data and to produce a deep learning stage output. The deep learning stage includes a mapping circuitry, and at least one artificial neural network. The mapping circuitry is configured to map image domain data to a tomographic data domain. The compressed sensing stage is configured to receive the deep learning stage output and to provide refined image data as output.
1. A hybrid image reconstruction system, the system comprising:
a deep learning stage configured to receive an input data set comprising measured tomographic data and to produce a deep learning stage output, the deep learning stage comprising a mapping circuitry, and at least one artificial neural network, the mapping circuitry configured to map image domain data to a tomographic data domain; and
a compressed sensing stage configured to receive the deep learning stage output and to provide refined image data as output.
2. The system of claim 1 , wherein the deep learning stage comprises an initial reconstruction network circuitry, and a deep learning stage refinement circuitry comprising at least one mapping circuitry, and at least one residual reconstruction network circuitry, and
the compressed sensing stage comprises an initial compressed sensing circuitry, and a compressed sensing stage refinement circuitry comprising at least one refinement compressed sensing circuitry,
at least a portion of the deep learning stage refinement circuitry and at least a portion of the compressed sensing stage refinement circuitry corresponding to a refinement stage,
the initial reconstruction network circuitry configured to receive the input data set and to reconstruct a corresponding initial image data, and the initial compressed sensing circuitry configured to regularize the initial image data to yield an estimated image data,
each mapping circuitry configured to receive a prior compressed sensing circuitry image data output and to produce a respective corresponding projection data set, and
each residual reconstruction network circuitry configured to receive a respective residual projection data set corresponding to a difference between the input data set and the respective prior corresponding projection data set and to determine a respective corresponding residual image data, and
each refinement compressed sensing circuitry configured to receive a sum of a prior compressed sensing circuitry image data output and the respective corresponding residual image data and to produce a respective refined image data output.
3. The system of claim 2 , wherein a system architecture corresponds to an unrolled network architecture that comprises a plurality of refinement stages.
4. The system of claim 3 , wherein a respective refinement stage comprises a respective mapping circuitry, a respective residual reconstruction circuitry and a respective refinement compressed sensing circuitry.
5. The system of claim 2 , wherein each residual projection data set is normalized, and each residual image data is denormalized.
6. The system of claim 1 , wherein the deep learning stage comprises an initial reconstruction circuitry, and a refinement circuitry,
the initial reconstruction circuitry configured to receive the input data set, to determine an estimated projection data set based, at least in part, on the input data set, and to determine an refined image data set based, at least in part, on the estimated projection data set, and
the refinement circuitry configured to receive the estimated projection data set and the refined image domain data set, and to determine an updated data-image pair, the updated data-image pair corresponding to the deep learning stage output.
7. The system of claim 6 , wherein the initial reconstruction circuitry comprises a projection network circuitry, and an image domain network circuitry, and
the refinement circuitry comprises a residual data network circuitry, and an image residual network circuitry, each network circuitry corresponding to an artificial neural network configured to operate in a projection data domain or an image data domain.
8. The system of claim 6 , wherein the input data set is sparse and the estimated measurements are relatively highly dimensional.
9. The system of claim 7 , wherein the projection network circuitry and the image domain network circuitry each corresponds to a respective encode-decode network and the image domain network circuitry corresponds to a generative adversarial network (GAN).
10. The system of claim 1 , wherein the input data is selected from the group comprising computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and single photon emission computed tomography (SPECT) tomographic data.
11. A method for hybrid image reconstruction, the method comprising:
receiving, by a deep learning stage, an input data set comprising measured tomographic data;
producing, by the deep learning stage, a deep learning stage output, the deep learning stage comprising a mapping circuitry, and at least one artificial neural network, the mapping circuitry configured to map image domain data to a tomographic data domain;
receiving, by a compressed sensing stage, the deep learning stage output; and
providing, by the compressed sensing stage, refined image data as output.
12. The method of claim 11 , wherein the deep learning stage comprises an initial reconstruction network circuitry, and a deep learning stage refinement circuitry comprising at least one mapping circuitry, and at least one residual reconstruction network circuitry, and the compressed sensing stage comprises an initial compressed sensing circuitry, and a compressed sensing stage refinement circuitry comprising at least one refinement compressed sensing circuitry, at least a portion of the deep learning stage refinement circuitry and at least a portion of the compressed sensing stage refinement circuitry corresponding to a refinement stage, and further comprising:
reconstructing, by the initial reconstruction network circuitry, a corresponding initial image data based, at least in part, on the input data set; and
regularizing, by the initial compressed sensing circuitry, the initial image data to yield an estimated image data;
producing, by each mapping circuitry, a respective corresponding projection data set based, at least in part on, a prior compressed sensing circuitry image data output;
producing, by each residual reconstruction network circuitry, a respective corresponding residual image data based, at least in part, on a respective residual projection data set corresponding to a difference between the input data set and the respective prior corresponding projection data set;
receiving, by each refinement compressed sensing circuitry, a sum of a prior compressed sensing circuitry image data output and the respective corresponding residual image data; and
producing, by each refinement compressed sensing circuitry, a respective refined image data output.
13. The method of claim 12 , wherein a system architecture corresponds to an unrolled network architecture that comprises a plurality of refinement stages.
14. The method of claim 13 , wherein a respective refinement stage comprises a respective mapping circuitry, a respective residual reconstruction circuitry and a respective refinement compressed sensing circuitry.
15. The method of claim 12 , wherein each residual projection data set is normalized and each residual image data is denormalized.
16. The method of claim 11 , wherein the deep learning stage comprises an initial reconstruction circuitry, and a refinement circuitry, and further comprising:
receiving, by the initial reconstruction circuitry, the input data set;
determining, by the initial reconstruction circuitry, an estimated projection data set based,
at least in part, on the input data set;
determining, by the initial reconstruction circuitry, an refined image data set based, at least in part, on the estimated projection data set;
receiving, by the refinement circuitry, the estimated projection data set and the refined image domain data set; and
determining, by the refinement circuitry, an updated data-image pair, the updated data image pair corresponding to the deep learning stage output.
17. The method of claim 16 , wherein the initial reconstruction circuitry comprises a projection network circuitry, and an image domain network circuitry, and
the refinement circuitry comprises a residual data network circuitry, and an image residual network circuitry, each network circuitry corresponding to an artificial neural network configured to operate in a projection data domain or an image data domain.
18. The method of claim 16 , wherein the input data set is sparse and the estimated measurements are relatively highly dimensional.
19. The method of claim 17 , wherein the projection network circuitry and the image domain network circuitry each corresponds to a respective encode-decode network and the image domain network circuitry corresponds to a generative adversarial network (GAN).
20. The method of claim 11 , wherein the input data is selected from the group comprising computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and single photon emission computed tomography (SPECT) input data.
21. The system of claim 1 , wherein the compressed sensing stage implements a total variation method.
22. The system of claim 1 , wherein the compressed sensing stage implements a low-rank method.
23. The system of claim 1 , wherein the compressed sensing stage implements a dictionary learning method.