Systems and media for automatically diagnosing thyroid nodules

1. A system for automatically diagnosing thyroid nodules, the system comprising:

at least one hardware processor that is programmed to:

receive a B-mode ultrasound image of a subject's thyroid;

provide the B-mode ultrasound image to a first trained classification model, wherein the first trained classification model was trained to automatically segment B-mode ultrasound images input to the first trained classification model based on training data comprising manually segmented B-mode ultrasound images;

receive, from the first trained classification model, an output indicating which portions of the B-mode ultrasound image correspond to a nodule;

provide at least a portion of the B-mode ultrasound image corresponding to the nodule to a second trained classification model, wherein the second trained classification model was trained to automatically classify thyroid nodules based on manually labeled portions of B-mode ultrasound image data, color Doppler ultrasound image data, and shear wave elastography ultrasound image data corresponding to benign and malignant nodules; and

receive, from the second trained classification model, an output indicative of a likelihood that the nodule is malignant.

2. The system of claim 1 , further comprising:

an ultrasound machine configured to output B-mode image data; and

the at least one hardware processor is further programmed to receive the B-mode ultrasound image from the ultrasound machine.

3. The system of claim 1 , wherein the at least one hardware processor is further programmed to:

receive a color Doppler ultrasound image of the subject's thyroid;

provide at least a portion of the color Doppler ultrasound image corresponding to the nodule to the second trained classification model;

receive a shear wave elastography ultrasound image of the subject's thyroid;

provide at least a portion of the shear wave elastography ultrasound image corresponding to the nodule to the second trained classification model;

receive, from the second trained classification model, the output indicative of the likelihood that the nodule is malignant based on the information included in the B-mode ultrasound image, the color Doppler ultrasound image, and the shear wave elastography ultrasound image; and

concurrently provide at least the portion of the B-mode ultrasound image on a first input channel of the second trained classification model, at least the portion of the color Doppler ultrasound image on a second input channel of the second trained classification model, and at least the portion of the shear wave elastography ultrasound image on a third input channel of the second trained classification model.

4. The system of claim 3 , wherein the at least one hardware processor is further programmed to:

automatically segment the shear wave elastography ultrasound image based on the segmentation of the B-mode ultrasound image.

5. The system of claim 1 , wherein the at least one hardware processor is further programmed to:

receive, from the first trained classification model, a mask indicating which pixels of the B-mode ultrasound image correspond to the nodule.

6. The system of claim 1 , wherein the second trained classification model is a convolutional neural network with an architecture based on Resnet 50.

7. The system of claim 1 ,

wherein the second trained classification model is configured to classify nodules into a first class and a second class, the first class corresponding to benign nodules and the second class corresponding to malignant nodules, and

wherein the at least one hardware processor is further programmed to:

indicate that the nodule is benign in response to a confidence value corresponding to a likelihood that the nodule is a member of the first class is indicative of a 100% confidence that the nodule is an example of the first class.

8. A method for automatically diagnosing thyroid nodules, the method comprising:

receiving a B-mode ultrasound image of a subject's thyroid;

providing the B-mode ultrasound image to a first trained classification model, wherein the first trained classification model was trained to automatically segment B-mode ultrasound images input to the first trained classification model based on training data comprising manually segmented B-mode ultrasound images;

receiving, from the first trained classification model, an output indicating which portions of the B-mode ultrasound image correspond to a nodule;

providing at least a portion of the B-mode ultrasound image corresponding to the nodule to a second trained classification model, wherein the second trained classification model was trained to automatically classify thyroid nodules based on manually labeled portions of B-mode ultrasound image data, color Doppler ultrasound image data, and shear wave elastography ultrasound image data corresponding to benign and malignant nodules; and

receiving, from the second trained classification model, an output indicative of a likelihood that the nodule is malignant.

9. A system for automatically diagnosing thyroid nodules, the system comprising:

at least one hardware processor that is programmed to:

receive a B-mode ultrasound image of a subject's thyroid;

receive information indicating which portions of the B-mode ultrasound correspond to a nodule;

provide at least a portion of the B-mode ultrasound image corresponding to the nodule to a trained classification model, wherein the trained classification model was trained to automatically classify thyroid nodules based on training data comprising a plurality of B-mode ultrasound images and at least one of a plurality of Doppler ultrasound images or a plurality of shear wave elastography ultrasound images each labeled as including a benign nodule or a malignant nodule; and

receive, from the trained classification model, an output indicative of a likelihood that the nodule is malignant.

10. The system of claim 9 , wherein the at least one hardware processor is further programmed to:

provide at least the portion of the B-mode ultrasound image on a first input channel of the trained classification model, and the information indicating which portions of the B-mode ultrasound correspond to the nodule on a second input channel of the trained classification model.

11. The system of claim 9 , wherein the information indicating which portions of the B-mode ultrasound correspond to the nodule comprises a mask indicating which pixels of the B-mode ultrasound image correspond to the nodule.

12. The system of claim 11 , wherein the at least one hardware processor is further programmed to:

provide the mask as an input to the trained classification model.

13. The system of claim 9 , wherein the at least one hardware processor is further programmed to:

provide the B-mode ultrasound image to a first trained classification model, wherein the first trained classification model was trained to automatically segment B-mode ultrasound images based on manually segmented B-mode ultrasound images; and

receive, from the first trained classification model, the mask.

14. The system of claim 9 , wherein the trained classification model is a convolutional neural network with an architecture based on Resnet 50.

15. The system of claim 9 , wherein the trained classification model is a convolutional neural network with an architecture based on InceptionV3.

16. The system of claim 9 , wherein the trained classification model is configured to classify nodules into a first class and a second class, the first class corresponding to benign nodules and the second class corresponding to malignant nodules.

17. The system of claim 16 , wherein the output of the trained classification model comprises a first confidence value corresponding to a likelihood that the nodule is a member of the first class, and a second confidence value corresponding to a likelihood that the nodule is a member of the second class.

18. The system of claim 17 , wherein the at least one hardware processor is further programmed to:

indicate that the nodule is benign in response to the first confidence value corresponding to a 100% confidence that the nodule is a member of the first class.

19. The system of claim 9 , wherein the at least one hardware processor is further programmed to:

receive a longitudinal B-mode ultrasound image of the subject's thyroid;

receive information indicating which portions of the longitudinal B-mode ultrasound correspond to the nodule;

provide at least a portion of the longitudinal B-mode ultrasound image to the trained classification model; and

receive, from the trained classification model, a second output indicative of a likelihood that the nodule is malignant.

20. The system of claim 19 , wherein the at least one hardware processor is further programmed to:

indicate that the nodule is benign in response to the output and the second output both indicating with 100% confidence that the nodule is benign.