Systems and methods for utilizing synthetic medical images generated using a neural network
A system for completing a medical image having at least one obscured region includes an input for receiving a first classification map generated using an acquired optical coherence tomography (OCT) image having at least one obscured region, the acquired OCT image acquired using an imaging system and a pre-processing module coupled to the input and configured to create an obscured region mask. The pre-processing module also generates a second classification map that has the at least one obscured region filled in. The system also includes a generative network coupled to the pre-processing module and configured to generate a synthetic OCT image based on the second classification map and a post-processing module coupled to the generative network. The post-processing module is configured to receive the synthetic OCT image and the acquired OCT image and to generate a completed image based on the synthetic OCT image and the acquired OCT image.
1. A system for completing a medical image having at least one obscured region, the system comprising:
an input for receiving a first classification map generated using an acquired optical coherence tomography (OCT) image having at least one obscured region, the acquired OCT image acquired using an imaging system;
a pre-processing module coupled to the input and configured to create an obscured region mask and to generate a second classification map that has the at least one obscured region filled in;
a generative network coupled to the pre-processing module and configured to generate a synthetic OCT image based on the second classification map; and
a post-processing module coupled to the generative network and configured to receive the synthetic OCT image and the acquired OCT image and to generate a completed image based on the synthetic OCT image and the acquired OCT image.
2. The system according to claim 1 , further comprising a memory coupled to the post-processing module for storing the completed image.
3. The system according to claim 1 , further comprising a display coupled to the post-processing module and configured to display the completed image.
4. The system according to claim 1 , wherein the obscured region mask is created based on the classification map or the acquired OCT image.
5. The system according to claim 1 , wherein the completed image is generated by replacing obscured pixels in the acquired OCT image with corresponding pixels in the synthetic OCT image.
6. The system according to claim 1 , wherein the generative network is trained using a conditional generative adversarial network.
7. The system according to claim 1 , wherein the classification map is generated by identifying a wall area of a vessel in the acquired OCT image and classifying at least one type of tissue in the wall area using a convolution neural network.
8. The system according to claim 7 , wherein the at least one tissue type is one of calcium, lipid tissue, fibrous tissue, mixed tissue, non-pathological tissue or media, and no visible tissue.
9. The system according to claim 1 , wherein the at least one obscured region is filled by determining the expected, likely or nominal classifications of a plurality of pixels in the obscured region.
10. A method for completing a medical image having at least one obscured region, the method comprising:
receiving a first classification map generated using an acquired optical coherence tomography (OCT) image having at least one obscured region, the acquired OCT image acquired using an imaging system;
creating an obscured region mask;
generating a second classification map that has the at least one obscured region filled in;
generating a synthetic OCT image based on the second classification map using a generative network;
generating a completed image based on the synthetic OCT image and the acquired OCT image; and
displaying the completed image on a display or storing the completed image in a memory.
11. The method according to claim 10 , wherein the obscured region mask is created based on the classification map or the acquired OCT image.
12. The method according to claim 10 , wherein generating the completed image includes replacing obscured pixels in the acquired OCT image with corresponding pixels in the synthetic OCT image.
13. The method according to claim 10 , wherein the generative network is trained using a conditional generative adversarial network.
14. The method according to claim 10 , wherein the first classification map is generated by identifying a wall area of a vessel in the acquired OCT image and classifying at least one type of tissue in the wall area using a convolution neural network.
15. The method according to claim 14 , wherein the at least one tissue type is one of calcium, lipid tissue, fibrous tissue, mixed tissue, non-pathological tissue or media, and no visible tissue.
16. The method according to claim 10 , wherein filling the obscured region mask of the classification map includes determining the expected, likely or nominal classifications of a plurality of pixels in the obscured region.