Method, device, and computer program for an uncertainty assessment of an image classification
A method for ascertaining an uncertainty of a prediction of a first machine learning system. The method includes: processing a detected input variable by way of the first machine learning system, intermediate results, which are ascertained during the processing of the input variable by way of the machine learning system, being stored. Processing at least one of the stored intermediate results by way of a second machine learning system, the second machine learning system outputting an output variable, which characterizes an uncertainty of the output classification. A method for training the second machine learning system and a computer system, computer program, and a machine-readable memory element, on which the computer program is stored, are also described.
1. A method for ascertaining an uncertainty of a classification, which is output by a first machine learning system, the method comprising the following steps:
processing a detected input variable using the first machine learning system and outputting the classification;
processing, using a second machine learning system, at least one intermediate result of a plurality of intermediate results, which are ascertained during the processing of the input variable by the first machine learning system; and
outputting, by the second machine learning system, an output variable, which characterizes an uncertainty of the classification of the first machine learning system, as a function of the at least one intermediate result;
controlling, based on the output classification, an actuator of a technical system, wherein the controlling of the actuator takes into consideration the output variable from the second machine learning system;
wherein the second machine learning system has been trained in such a way that it outputs an uncertainty of the classification of the first machine learning system as a function of the at least one intermediate result.
2. The method as recited in claim 1 , wherein each of the first and second machine learning systems is a neural network, an architecture of the neural network of the second machine learning system being smaller than an architecture of the neural network of the first machine learning system.
3. The method as recited in claim 1 , wherein when the second machine learning system outputs an uncertainty greater than a predefined threshold value, a warning is output.
4. The method as recited in claim 1 , wherein the detected wherein the detected input variable is detected by a sensor of the technical system, the sensor including an imaging sensor.
5. The method as recited in claim 4 , wherein the actuator is a motor.
6. The method as recited in claim 5 , wherein the technical system is an at least semi-autonomous vehicle.
7. The method as recited in claim 5 , wherein the technical system is a robot or a tool.
8. The method as recited in 1 , further comprising:
ascertaining a control variable as a function of the output classification;
controlling the actuator of the technical system based on the ascertained control variable, and taking into consideration the output variable from the second machine learning system.
9. A method for training a second machine learning system, comprising the following steps:
providing a set of training data, the set of training data containing a plurality of training input data, which are ascertained intermediate results of a first machine learning system and labels associated with each of the training input data, which characterize an uncertainty, training the second machine learning system in such a way that it, as a function of the intermediate results, ascertains their associated labels;
after the training, providing the trained second machine learning system to a vehicle;
processing a detected input variable using the first machine learning system and outputting by the first machine learning system a classification;
processing, using the trained second machine learning system, at least one intermediate result of a plurality of intermediate results, which are ascertained during the processing of the input variable by the first machine learning system;
outputting, by the second machine learning system, an output variable, which characterizes an uncertainty of the classification of the first machine learning system, as a function of the at least one intermediate result; and
controlling, based on the output classification, an actuator of the vehicle, wherein the controlling of the actuator takes into consideration the output variable from the second machine learning system.
10. The method as recited in claim 9 , wherein the labels correspond to an expected accuracy of a classification of the first machine learning system.
11. The method as recited in claim 10 , wherein the expected accuracy is ascertained using a reference uncertainty estimation method.
12. The method as recited in claim 10 , wherein the expected accuracy is ascertained as a function of a plurality of augmented input variables of the first machine learning system and a calculation of a portion of incorrect classifications of the first machine learning system as a function of the augmented input variables.
13. The method as recited in claim 10 , wherein the classification of the first machine learning system is a semantic segmentation, the expected accuracy being a function of a mean classification error rate in various regions of the semantic segmentation.
14. A non-transitory machine-readable memory element on which is stored a computer program for ascertaining an uncertainty of a classification, which is output by a first machine learning system, the computer program, when executed by a computer, causing the computer to perform the following steps:
processing a detected input variable using the first machine learning system and outputting the classification;
processing, using a second machine learning system, at least one intermediate result of a plurality of intermediate results, which are ascertained during the processing of the input variable by the first machine learning system; and
outputting, by the second machine learning system, an output variable, which characterizes an uncertainty of the classification of the first machine learning system, as a function of the at least one intermediate result;
controlling, based on the output classification, an actuator of a technical system, wherein the controlling of the actuator takes into consideration the output variable from the second machine learning system;
wherein the second machine learning system has been trained in such a way that it outputs an uncertainty of the classification of the first machine learning system as a function of the at least one intermediate result.
15. A device configured to ascertain an uncertainty of a classification, which is output by a first machine learning system, the device configured to:
process a detected input variable using the first machine learning system and outputting the classification;
process, using a second machine learning system, at least one intermediate result of a plurality of intermediate results, which are ascertained during the processing of the input variable by the first machine learning system; and
output, by the second machine learning system, an output variable, which characterizes an uncertainty of the classification of the first machine learning system, as a function of the at least one intermediate result;
control, based on the output classification, an actuator of a technical system, wherein the controlling of the actuator takes into consideration the output variable from the second machine learning system;
wherein the second machine learning system has been trained in such a way that it outputs an uncertainty of the classification of the first machine learning system as a function of the at least one intermediate result.