Systems and methods for training generative adversarial networks and use of trained generative adversarial networks
The present disclosure relates to computer-implemented systems and methods for training and using generative adversarial networks. In one implementation, a system for training a generative adversarial network may include at least one processor that may provide a first plurality of images including representations of a feature-of-interest and indicators of locations of the feature-of-interest and use the first plurality and indicators to train an object detection network. Further, the processor(s) may provide a second plurality of images including representations of the feature-of-interest, and apply the trained object detection network to the second plurality to produce a plurality of detections of the feature-of-interest. Additionally, the processor(s) may provide manually set verifications of true positives and false positives with respect to the plurality of detections, use the verifications to train a generative adversarial network, and retrain the generative adversarial network using at least one further set of images, further detections, and further manually set verifications.
1. A system for training a generative adversarial network using images including representations of a feature-of-interest, comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
provide a first plurality of images that include representations of the feature-of-interest and indicators of the locations of the feature-of-interest in images of the first plurality of images;
using the first plurality of images and indicators of the locations of the feature-of-interest, train an object detection network to detect the feature-of-interest;
provide a second plurality of images that include representations of the feature-of-interest, the second plurality of images comprising a larger number of images than that included in the first plurality of images;
apply the trained object detection network to the second plurality of images to produce a first plurality of detections of the feature-of-interest;
provide manually set verifications of true positives and false positives with respect to the first plurality of detections;
using the verifications of the true positives and false positives with respect to the first plurality of detections, train a generative adversarial network; and
retrain the generative adversarial network using at least one further set of images and detections of the feature-of-interest, together with further manually set verifications of true positives and false positives with respect to the further detections of the feature-of-interest.
2. The system of claim 1 , wherein the at least one processor is further configured to retrain the generative adversarial network by providing verifications of false negatives for missed detections of the feature-of-interest in two or more images.
3. The system of claim 1 , wherein the object detection network is a convolutional neural network.
4. The system of claim 1 , wherein the number of images in the second plurality of images is at least 100 times larger than that included in the first plurality of images.
5. The system of claim 1 , wherein the first plurality of images and the second plurality of images comprise medical images.
6. The system of claim 5 , wherein the medical images comprise images of a gastro-intestinal organ.
7. The system of claim 1 , wherein at least one the first plurality of images and the second plurality of images comprise images from an endoscopy device.
8. The system of claim 1 , wherein at least one the first plurality of images and the second plurality of images comprise images from imaging device used during at least one of a gastroscopy, a colonoscopy, or an enteroscopy.
9. The system of claim 1 , wherein the feature-of-interest is an abnormality.
10. The system of claim 9 , wherein the abnormality comprises a change in human tissue from one type of cell to another type of cell.
11. The system of claim 9 , wherein the abnormality comprises an absence of human tissue from a location where the human tissue is expected.
12. The system of claim 9 , wherein the abnormality comprises a formation on or of human tissue.
13. The system of claim 12 , wherein the abnormality comprises a lesion.
14. The system of claim 13 , wherein the lesion comprises a polypoid lesion.
15. The system of claim 13 , wherein the lesion comprises a non-polypoid lesion.
16. The system of claim 1 , wherein the first plurality of detections comprises a location indicator when a feature-of-interest is present or a null indicator when no feature-of-interest is present.
17. The system of claim 1 , wherein the at least one processor is further configured to provide verifications of a false negative for images that had no feature-of-interest detected but include the feature-of-interest.
18. The system of claim 1 , wherein the indicators of the locations of the feature-of-interest comprise at least one of points, regions, or manual annotations.
19. The system of claim 1 , wherein the verifications of true positives and false positives are received from at least one of one or more databases or one or more inputs.
20. A method for training a generative adversarial network using images including representations of a feature-of-interest, the method comprising the following steps performed by at least one processor:
providing a first plurality of images that include representations of the feature-of-interest and indicators of the locations of the feature-of-interest in images of the first plurality of images;
using the first plurality of images and indicators of the locations of the feature-of-interest to train an object detection network to detect the feature-of-interest;
providing a second plurality of images that include representations of the feature-of-interest, the second plurality of images comprising a larger number of images than that included in the first plurality of images;
applying the trained object detection network to the second plurality of images to produce a first plurality of detections of the feature-of-interest;
providing manually set verifications of true positives and false positives with respect to the first plurality of detections;
using the verifications of the true positives and false positives with respect to the first plurality of detections to train a generative adversarial network; and
retraining the generative adversarial network using at least one further set of images and detections of the feature-of-interest, together with further manually set verifications of true positives and false positives with respect to the further detections of the feature-of-interest.
21. The method of claim 20 , wherein the method further comprises retraining the generative adversarial network by providing verifications of false negatives for missed detections of the feature-of-interest in two or more images.
22. The method of claim 20 , wherein the object detection network is a convolutional neural network.
23. The method of claim 20 , wherein the number of images in the second plurality of images is at least 100 times larger than that included in the first plurality of images.
24. The method of claim 20 , wherein the first plurality of images and the second plurality of images comprise medical images.
25. The method of claim 20 , wherein at least one the first plurality of images and the second plurality of images comprise images from imaging device used during at least one of a gastroscopy, a colonoscopy, or an enteroscopy.
26. The method of claim 20 , wherein the first plurality of detections comprises a location indicator when a feature-of-interest is present or a null indicator when no feature-of-interest is present.