Calibrating output from an image classifier
An apparatus for calibrating output from an image classifier. The apparatus has an image classifier trained to compute, from an image, confidence values for each of a plurality of skin conditions potentially depicted in the image. The apparatus has a processor to compute a probability score for at least one skin condition by adjusting an associated confidence value using information from an incidence corrected data set which is a plurality of images resampled according to data about an incidence of the skin condition in a population from which the image was captured.
1 . A computer-implemented method of calibrating output from an image classifier, the method comprising the steps of:
using the image classifier to compute, from an image, confidence values for each of a plurality of skin conditions potentially depicted in the image;
computing a probability score for each skin condition by adjusting the confidence values using information from an incidence corrected data set which is a plurality of images resampled according to data about an incidence of the skin conditions in a population from which the image was captured so that the incidence corrected data set contains a number of images of each skin condition which reflects incidence of each skin condition in the population,
wherein the probability score for each skin condition is computed using a neural network to perform regression to predict the probability score for each skin condition from the confidence value for each skin condition, or using a Support Vector Regression (SVR) algorithm trained to predict the probability score for each skin condition from the confidence value for each skin condition; and
sending a selected skin condition of the plurality of skin conditions, the probability score for the selected skin condition, or both, to a display.
2 . The computer-implemented method of claim 1 , wherein the incidence corrected data set comprises images resampled from a validation data set of the image classifier.
3 . The computer-implemented method of claim 1 , wherein the data about the incidence of the skin conditions is obtained from an incidence data store using information about a geographical location where the image was captured, or information about a type of health care facility where the image was captured.
4 . The computer-implemented method of claim 3 , wherein the information about geographical location or type of health care facility is received as a result of user input at a user interface.
5 . The computer-implemented method of claim 1 , wherein the regression performed by the neural network is a first order logistic regression.
6 . The computer-implemented method of claim 1 , further comprising selecting one of the skin conditions to be ruled out for a patient depicted in the image, by comparing each confidence value to a corresponding decision threshold having been computed from a plurality of validation images of the image classifier, and sending a probability score for the skin condition to be ruled out to the display.
7 . The computer-implemented method of claim 1 , further comprising selecting one of the skin conditions as a highest likelihood skin condition for a patient depicted in the image, by comparing each confidence value to a corresponding decision threshold having been computed from a plurality of validation images of the image classifier, in a specified order and sending a probability score for the highest likelihood skin condition to the display.
8 . The computer-implemented method of claim 7 comprising sending the image to the display for display adjacent to the probability score for the highest likelihood skin condition and a graphical representation of the probability score.
9 . The computer-implemented method of claim 7 comprising displaying the probability score for the highest likelihood skin condition as a percentile of training data used to train the image classifier.
10 . The computer-implemented method of claim 1 , further comprising displaying a table with a column for skin condition and a column for probability score.
11 . The computer-implemented method of claim 1 , further comprising displaying a list of skin conditions which exceed an associated decision threshold, in order of probability score.
12 . An apparatus for calibrating output from an image classifier, the apparatus comprising:
an image classifier trained to compute, from an image, confidence values for each of a plurality of skin conditions potentially depicted in the image;
a processor to compute a probability score for each skin condition by adjusting the confidence values using information from an incidence corrected data set which is a plurality of images resampled according to data about an incidence of the skin conditions in a population from which the image was captured so that the incidence corrected data set contains a number of images of each skin condition which reflects incidence of each skin condition in the population, wherein the probability score for each skin condition is computed using a neural network to perform regression to predict the probability score for each skin condition from the confidence value for each skin condition, or using a Support Vector Regression (SVR) algorithm trained to predict the probability score for each skin condition from the confidence value for each skin condition; and
a display operable to display a selected skin condition of the plurality of skin conditions, the probability score for the selected skin condition, or both.
13 . A dermascope imaging device comprising a processor and storage having instructions which when executed by the processor:
compute, from an image, confidence values for each of a plurality of skin conditions potentially depicted in the image;
compute a probability score for at least one of the skin conditions by adjusting the confidence value for the skin condition using information from an incidence corrected data set which is a plurality of images resampled according to data about an incidence of the skin condition in a population from which the image was captured so that the incidence corrected data set contains a number of images of each skin condition which reflects incidence of each skin condition in the population,
wherein the probability score for each skin condition is computed using a neural network to perform regression to predict the probability score for each skin condition from the confidence value for each skin condition, or using a Support Vector Regression (SVR) algorithm trained to predict the probability score for each skin condition from the confidence value for each skin condition; and
send a selected skin condition of the plurality of skin conditions, the probability score for the selected skin condition, or both, to a display.