Confidence measure for a deployed machine learning model

1. A method for obtaining a confidence measure for a machine learning model, the method comprising:

processing input data with the machine learning model to generate a primary result;

generating a plurality of modified instances of the input data;

processing the plurality of modified instances of the input data with the machine learning model to generate a respective plurality of secondary results; and

determining a confidence measure relating to the primary result based on the secondary results,

wherein determining a confidence measure comprises:

determining a measure of distribution or variance of the secondary results; and

determining a confidence measure based on the determined measure of distribution or variance,

wherein determining the measure of distribution or variance of the secondary results comprises determining at least one of:

the inverse variance of the secondary results;

the Shannon entropy of the secondary results;

the gini coefficient of the secondary results;

the Kullback-Liebler divergence of the secondary results; and

a concentration measure of the secondary results,

wherein generating a plurality of modified instances of the input data comprises:

applying a first spatial warping transformation to the input data to generate a first modified instance of the input data,

applying a first inverse spatial warping transformation to the secondary result generated for the first modified instance of the input data.

2. The method of claim 1 , wherein generating a plurality of modified instances of the input data comprises:

adding noise to the input data to generate a second modified instance of the input data.

3. The method of claim 1 , wherein generating a plurality of modified instances of the input data comprises:

applying a local deformation transformation to the input data to generate a third modified instance of the input data.

4. The method of claim 3 , further comprising:

applying a first inverse local deformation transformation to the secondary result generated for the third modified instance of the input data.

5. The method of claim 1 , wherein the machine learning model comprises at least one of:

an artificial neural network;

a Generative Adversarial Network, GAN; and

a Bayesian network.

6. The method of claim 1 , further comprising:

associating the determined confidence measure with the primary result.

7. A non-transitory computer readable medium storing instructions for obtaining a confidence measure for a machine learning model that, when executed by one or more processors, causes the one or more processors to:

process input data with the machine learning model to generate a primary result;

generate a plurality of modified instances of the input data;

process the plurality of modified instances of the input data with the machine learning model to generate a respective plurality of secondary results; and

determine a confidence measure relating to the primary result based on the secondary results, wherein determining a confidence measure comprises:

determining a measure of distribution or variance of the secondary results; and

determining a confidence measure based on the determined measure of distribution or variance,

wherein determining the measure of distribution or variance of the secondary results comprises determining at least one of:

the inverse variance of the secondary results;

the Shannon entropy of the secondary results;

the gini coefficient of the secondary results;

the Kullback-Liebler divergence of the secondary results; and

a concentration measure of the secondary results,

wherein generating a plurality of modified instances of the input data comprises:

applying a first spatial warping transformation to the input data to generate a first modified instance of the input data,

applying a first inverse spatial warping transformation to the secondary result generated for the first modified instance of the input data.

8. A system for obtaining a confidence measure for a machine learning model, the system comprising:

an input interface configured to obtain input data;

a data modification component configured to generate a plurality of modified instances of the input data;

a machine learning model interface configured to communicate the input data and the plurality of modified instances of the input data to a machine learning model and further configured to receive a primary result generated by the machine learning model processing the input data and to receive a plurality of secondary results generated by the machine learning model processing the respective plurality of modified instances of the input data; and

an analysis component configured to determine a confidence measure relating to the primary result based on the secondary results, wherein the data modification component is configured to apply a first spatial warping transformation to the input data to generate a first modified instance of the input data, wherein the analysis component is configured to determine a measure of distribution or variance of the secondary results and to determine a confidence measure based on the determined measure of distribution or variance, wherein determining the measure of distribution or variance of the secondary results comprises determining at least one of:

the inverse variance of the secondary results;

the Shannon entropy of the secondary results;

the gini coefficient of the secondary results;

the Kullback-Liebler divergence of the secondary results; and

a concentration measure of the secondary results, wherein the data modification component is further configured to apply a first inverse spatial warping transformation to the secondary result generated for the first modified instance of the input data.