Computer-implemented method and device for evaluating a dataset based on a range of uncertainty
A computer-implemented method for evaluating a dataset. The dataset is generated by a physical process. According to the method, the dataset has a) a plurality of first data points, which can be assigned to a first label, wherein the first data points at least partially form at least one first local cluster area, and b) a plurality of second data points, which can be assigned to a second label and the second data points at least partially form at least one second local cluster area. The first data points and the second data points are separable. Creating a range of uncertainty at least between the first cluster area of the first data points and the second cluster area of the second data points; and evaluating the dataset based on the range of uncertainty so created.
1 . A computer-implemented method comprising:
performing a physical process and acquiring data from the physical process;
creating a dataset based on the physical process;
evaluating a probability of a misclassification of the dataset by the following steps:
providing the dataset, which includes:
a plurality of first data points configured to be assigned to a first label, wherein the first data points at least partially form at least one first local cluster area; and
a plurality of second data points configured to be assigned to a second label, wherein the second data points at least partially form at least one second local cluster area; and
wherein the first data points and the second data points are separable;
establishing a range of uncertainty between the first local cluster area of the first data points and the second local cluster area of the second data points;
evaluating the dataset based on the range of uncertainty by determining a limit range of the range of uncertainty based on a stochastic distribution; and
executing the step of evaluating a probability of a misclassification of the dataset when the following assumptions hold true for the dataset or a transformation of the dataset was successful such that the following assumptions hold true:
the data points are separated in the m-dimensional space by (m−1) dimensional sets of hyperareas or hyperplanes,
the hyperarea and/or the limit area of the hyperarea is definable or measurable and is finite,
the data points that are configured to be assigned to a label cluster or form a local cluster area, and
in the vicinity of the limit range of the hyperarea, the distribution of the data points is locally homogeneous.
2 . The method according to claim 1 , wherein the evaluating step comprises:
estimating a probability that a data point is assigned to an incorrect label.
3 . The method according to claim 2 , which comprises estimating the probability that one of the first data points is assigned to the second label and/or that one of the second data points is assigned to the first label.
4 . The method according to claim 1 , wherein the step of creating the range of uncertainty furthermore comprises determining a finite hyperarea of the range of uncertainty.
5 . The method according to claim 4 , wherein the step of determining the hyperarea comprises at least one of the following: Delaunay triangulation, limit search, maximum margin hyperplane.
6 . The method according to claim 1 which comprises determining the limit range of the range of uncertainty based on the stochastic distribution by estimating a quantile of a minimum distance distribution.
7 . The method according to claim 1 , wherein the dataset originates from a rail transportation technology environment and related to at least one of a signal, an axle counter, a point mechanism.
8 . The method according to claim 7 , wherein the dataset originates from a light signal.
9 . The method according to claim 1 , wherein the dataset is selected from the group consisting of a data set with at least 100,000 data points, a data set with at least 500,000 data points, and a data set with at least 1,000,000 data points.
10 . The method according to claim 1 , wherein the problem space of the dataset is defined in a plurality of dimensions.
11 . The method according to claim 10 , wherein the problem space has three or more dimensions.
12 . The method according to claim 1 , which further comprises reducing the applied dimensions by way of a principal component analysis.
13 . The method according to claim 1 , which further comprises increasing the applied dimensions by transforming data.
14 . The method according to claim 1 , which further comprises:
separating at least one sub-dataset from the dataset, so that a distribution of the data points in a vicinity of the range of uncertainty is substantially locally homogeneous for the at least one sub-dataset.
15 . The method according to claim 10 , which further comprises merging an evaluation of at least two sub-datasets.
16 . The method according to claim 1 , wherein the evaluating step comprises determining whether the dataset fulfils a predetermined safety criterion.
17 . The method according to claim 16 , wherein the predetermined safety criterion is a safety criterion defined in the field of rail transportation technology.
18 . A method, comprising:
performing a physical process;
creating a dataset, by way of an artificial intelligence (AI) algorithm, based on the physical process; and
evaluating the dataset according to claim 1 .
19 . A data processing apparatus, comprising at least one processor and being configured to execute the method according to claim 1 .
20 . A non-transitory computer program product comprising computer-executable instructions which, when the program is executed by a computer, cause the computer to execute the method according to claim 1 .