Systems and methods for training generative adversarial networks and use of trained generative adversarial networks
The present disclosure relates to computer-implemented systems and methods for training and using generative adversarial networks. In one implementation, a system for training a generative adversarial network may include at least one processor that may provide a first plurality of images including representations of a feature-of-interest and indicators of locations of the feature-of-interest and use the first plurality and indicators to train an object detection network. Further, the processor(s) may provide a second plurality of images including representations of the feature-of-interest, and apply the trained object detection network to the second plurality to produce a plurality of detections of the feature-of-interest. Additionally, the processor(s) may provide manually set verifications of true positives and false positives with respect to the plurality of detections, use the verifications to train a generative adversarial network, and retrain the generative adversarial network using at least one further set of images, further detections, and further manually set verifications.
1. A method for training a generative adversarial network using image frames including representations of a feature-of-interest, comprising:
a first training phase including:
extracting from a database a first plurality of image frames;
displaying to an operator, via a display device, the first plurality of image frames;
receiving from the operator, via an input device, a first plurality of feature indicators for at least one region in the first plurality of image frames, the at least one region including one or more representations of the feature-of-interest; and
training a discriminator network using a first training set including the first plurality of image frames and the first plurality of feature indicators;
a second training phase including:
further extracting from the database a second plurality of image frames;
applying the trained discriminator network to the second plurality of image frames to produce a second plurality of feature indicators for at least one region in the second plurality of image frames, the at least one region including one or more representations of the feature-of-interest;
receiving verifications of true positives and false positives with respect to the second plurality of feature indicators; and
training a generative adversarial network using a second training set including the second plurality of image frames and the verifications, wherein the generative adversarial network includes a generative network and an adversarial network, and wherein training the generative adversarial network includes training the generative network to produce artificial representations of the feature-of-interest and training the adversarial network to distinguish the artificial representations of the feature-of-interest from real representations of the feature-of-interest.
2. The method of claim 1 , wherein the first plurality of image frames and the second plurality of image frames comprise medical images captured by an imaging device used during at least one of an endoscopy, a gastroscopy, a colonoscopy, and an enteroscopy.
3. The method of claim 1 , wherein the feature-of-interest includes an abnormality comprising at least one of a formation on or of human tissue, a change in human tissue from one type of cell to another type of cell, an absence of human tissue from a location where the human tissue is expected, and a lesion.
4. The method of claim 1 , wherein at least one of the first plurality of image frames and the second plurality of image frames are extracted from the database randomly or using one or more patterns.
5. The method of claim 1 , wherein the discriminator network comprises an object detection network.
6. The method of claim 1 , wherein the input device includes at least one of a knob, a keyboard, a mouse, and a touch screen.
7. The method of claim 1 , wherein training the discriminator network includes adjusting one or more weights associated with one or more nodes of the discriminator network or adjusting a function associated with one or more nodes of the discriminator network.
8. The method of claim 1 , wherein training the generative adversarial network further comprises training the generative network to produce artificial representations of a false feature-of-interest that looks similar to a true feature-of-interest.
9. The method of claim 1 , wherein the verifications are received from the operator via the input device.
10. The method of claim 1 , further comprising retraining the trained discriminator network using the verifications of the true positives and false positives.
11. A non-transitory computer-readable medium comprising instructions configured to cause at least one processor to perform a method, the method comprising:
a first training phase including:
extracting from a database a first plurality of image frames;
displaying to an operator, via a display device, the first plurality of image frames;
receiving from the operator, via an input device, a first plurality of feature indicators for at least one region in the first plurality of image frames, the at least one region including one or more representations of the feature-of-interest; and
training a discriminator network using a first training set including the first plurality of image frames and the first plurality of feature indicators;
a second training phase including:
further extracting from the database a second plurality of image frames;
applying the trained discriminator network to the second plurality of image frames to produce a second plurality of feature indicators for at least one region in the second plurality of image frames, the at least one region including one or more representations of the feature-of-interest;
receiving verifications of true positives and false positives with respect to the second plurality of feature indicators; and
training a generative adversarial network using a second training set including the second plurality of image frames and the verifications, wherein the generative adversarial network includes a generative network and an adversarial network, and wherein training the generative adversarial network includes training the generative network to produce artificial representations of the feature-of-interest and training the adversarial network to distinguish the artificial representations of the feature-of-interest from real representations of the feature-of-interest.
12. The non-transitory computer-readable medium of claim 11 , wherein the first plurality of image frames and the second plurality of image frames comprise medical images captured by an imaging device used during at least one of an endoscopy, a gastroscopy, a colonoscopy, and an enteroscopy.
13. The non-transitory computer-readable medium of claim 11 , wherein the feature-of-interest includes an abnormality comprising at least one of a formation on or of human tissue, a change in human tissue from one type of cell to another type of cell, an absence of human tissue from a location where the human tissue is expected, and a lesion.
14. The non-transitory computer-readable medium of claim 11 , wherein at least one of the first plurality of image frames and the second plurality of image frames are extracted from the database randomly or using one or more patterns.
15. The non-transitory computer-readable medium of claim 11 , wherein the discriminator network comprises an object detection network.
16. The non-transitory computer-readable medium of claim 11 , wherein the input device includes at least one of a knob, a keyboard, a mouse, and a touch screen.
17. The non-transitory computer-readable medium of claim 11 , wherein training the discriminator network includes adjusting one or more weights associated with one or more nodes of the discriminator network or adjusting a function associated with one or more nodes of the discriminator network.
18. The non-transitory computer-readable medium of claim 11 , wherein training the generative adversarial network further comprises training the generative network to produce artificial representations of a false feature-of-interest that looks similar to a true feature-of-interest.
19. The non-transitory computer-readable medium of claim 11 , wherein the verifications are received from the operator via the input device.
20. The non-transitory computer-readable medium of claim 11 , further comprising retraining the trained discriminator network using the verifications of the true positives and false positives.