Systems and methods for out-of-distribution detection
Some embodiments of the current disclosure disclose methods and systems for detecting out-of-distribution (ODD) data. For example, a method for detecting ODD data includes obtaining, at a neural network composed of a plurality of layers, a set of training data generated according to a distribution. Further, the method comprises generating, via a processor, a feature map by combining mapping functions corresponding to the plurality of layers into a vector of mapping function elements and mapping, by the feature map, the set of training data to a set of feature space training data in a feature space. Further, the method comprises identifying, via the processor, a hyper-ellipsoid in the feature space enclosing the feature space training data based on the generated feature map. In addition, the method comprises determining, via the processor, the first test data sample is OOD data when a mapped first test data sample in the feature space is outside the hyper-ellipsoid.
1 . A method for out-of-distribution (OOD) anomaly detection of images for an autonomous driving system, the method comprising:
implementing a neural network composed of a plurality of layers on a processor as a traffic sign classification model for the autonomous driving system;
receiving, for the neural network, a set of training images containing traffic signs and other items generated according to a preset distribution of the set of training images, wherein each layer of the plurality of layers of the neural network transforms a respective input to a respective output via a respective mapping function of a plurality of mapping functions;
training the neural network using the set of training images, to classify an incoming image depicting a traffic scene as an anomaly by determining whether the incoming image is OOD, wherein the training includes:
applying one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to each training image of the set of the training images to create a set of transformed training images;
generating, by the plurality of layers of the neural network implemented on the processor, a feature map by combining mapping functions of the plurality of layers into a vector of mapping function elements that comprises: a first mapping function element of a first mapping function from the plurality of mapping functions, and a second mapping function element of a function composition of the first mapping function and a second mapping function from the plurality of mapping functions;
mapping, by the feature map generated by the plurality of layers of the neural network, the set of transformed training images to a set of feature space training data in a feature space; and
identifying, via the processor, a hyper-ellipsoid in the feature space enclosing the set of feature space training data based on the generated feature map as a mechanism for determining whether the incoming image is OOD;
receiving, at inference time of the neural network associated with the identified hyper-ellipsoid used as the traffic sign classification model in the autonomous driving system, a first test image depicting a real world scene outside the set of training images;
applying, via the processor, one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to the first test image to create one or more transformed test images;
using the trained neural network associated with the identified hyper-ellipsoid to determine whether the first test image is OOD, including:
mapping the one or more transformed test images into the feature space by the generated feature map;
generating, via the processor, a prediction indicating that the first test image is OOD based on whether the mapped one or more transformed test images in the feature space are outside the hyper-ellipsoid identified from the training of the neural network; and
responsive to determining that the first test image is OOD, generating, by the trained neural network associated with the identified hyper-ellipsoid implemented as the traffic sign classification model in the autonomous driving system, a traffic sign classification output indicating that the first test image depicting the real world scene is an anomaly.
2 . The method of claim 1 , wherein the vector of mapping function elements is generated at least in part by performing a function composition of two or more mapping functions of the plurality of mapping functions to form a mapping function element in the vector.
3 . The method of claim 1 , wherein:
the set of training images includes a single class training dataset; and
the neural network is a classifier neural model pre-trained with the single class training dataset.
4 . The method of claim 1 , wherein:
the set of training images includes a multi-class training dataset; and
the neural network is a classifier neural model pre-trained with the multi-class training dataset.
5 . The method of claim 1 , wherein:
the set of training images includes an intent classification training dataset; and
the neural network is a classifier neural model pre-trained with the intent classification training dataset.
6 . The method of claim 1 , wherein:
the set of training images includes a question answering training dataset; and
the neural network is a non-classifier neural model pre-trained with the question answering training dataset.
7 . The method of claim 1 , wherein the neural network is a non-classifier neural model pre-trained with data lacking an OOD sample.
8 . The method of claim 1 , further comprising:
determining one hyper-ellipsoid that has a smallest volume among a set of candidate hyper-ellipsoids that enclose the set of feature space training data in the feature space as the hyper-ellipsoid.
9 . A system for out-of-distribution (OOD) anomaly detection of images for an autonomous driving system, the system comprising:
a memory storing instructions for OOD anomaly detection of test data; and
a processor configured to read and execute the instructions from the memory to:
implement a neural network composed of a plurality of layers on the processor as a traffic sign classification model for the autonomous driving system;
receive, for the neural network, a set of training images containing traffic signs and other items generated according to a preset distribution of the set of training images, wherein each layer of the plurality of layers of the neural network transforms a respective input to a respective output via a respective mapping function of a plurality of mapping functions;
train the neural network using the set of training images, to classify an incoming image depicting a traffic scene as an anomaly by determining whether the incoming image is OOD, wherein the training includes:
applying one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to each training image of the set of the training images to create a set of transformed training images;
generating, by the plurality of layers of the neural network implemented on the processor, a feature map by combining mapping functions of the plurality of layers into a vector of mapping function elements that comprises: a first mapping function element of a first mapping function from the plurality of mapping functions, and a second mapping function element of a function composition of the first mapping function and a second mapping function from the plurality of mapping functions;
mapping, by the feature map generated by the plurality of layers of the neural network, the set of transformed training images to a set of feature space training data in a feature space; and
identifying, via the processor, a hyper-ellipsoid in the feature space enclosing the set of feature space training data based on the generated feature map as a mechanism for determining whether the incoming image is OOD;
receive, at inference time of the neural network associated with the identified hyper-ellipsoid used as the traffic sign classification model in the autonomous driving system, a first test image depicting a real world scene outside the set of training images;
apply, via the processor, one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to the first test image to create one or more transformed test images;
use the trained neural network associated with the identified hyper-ellipsoid to determine whether the first test image is OOD, including:
mapping the one or more transformed test images into the feature space by the generated feature map;
generating, via the processor, a prediction indicating that the first test image is OOD based on whether the mapped one or more transformed test images in the feature space are outside the hyper-ellipsoid identified from the training of the neural network; and
responsive to determining that the first test image is OOD, generate, by the trained neural network associated with the identified hyper-ellipsoid implemented as the traffic sign classification model in the autonomous driving system, a traffic sign classification output indicating that the first test image depicting the real world scene is an anomaly.
10 . The system of claim 9 , wherein:
the vector of mapping function elements is generated at least in part by performing a function composition of two or more mapping functions of the plurality of mapping functions to form a mapping function element in the vector.
11 . The system of claim 9 , wherein:
the set of training images includes a single class training dataset, a multi-class training dataset or an intent classification training dataset; and
the neural network is a classifier neural model pre-trained with the single class training dataset, the multi-class training dataset or the intent classification training dataset.
12 . The system of claim 9 , wherein:
the set of training images includes a question answering training dataset; and
the neural network is a non-classifier neural model pre-trained with the question answering training dataset.
13 . The system of claim 9 , wherein the neural network is a non-classifier neural model pre-trained with data lacking an OOD sample.
14 . The system of claim 9 , further the processor is further configured to:
determine one hyper-ellipsoid that has a smallest volume among a set of candidate hyper-ellipsoids that enclose the set of feature space training data in the feature space as the hyper-ellipsoid.
15 . A non-transitory computer-readable medium (CRM) having stored thereon machine-readable instructions for out-of-distribution (OOD) anomaly detection of images for an autonomous driving system, the machine-readable instructions executable to cause a processor to perform operations comprising:
implementing a neural network composed of a plurality of layers on the processor as a traffic sign classification model for the autonomous driving system;
receiving, for the neural network, a set of training images containing traffic signs and other items generated according to a preset distribution of the set of training images, wherein each layer of the plurality of layers of the neural network transforms a respective input to a respective output via a respective mapping function of a plurality of mapping functions;
training the neural network using the set of training images, to classify an incoming image depicting a traffic scene as an anomaly by determining whether the incoming image is OOD, wherein the training includes:
applying one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to each training image of the set of the training images to create a set of transformed training images;
generating, by the plurality of layers of the neural network implemented on the processor, a feature map by combining mapping functions of the plurality of layers into a vector of mapping function elements that comprises: a first mapping function element of a first mapping function from the plurality of mapping functions, and a second mapping function element of a function composition of the first mapping function and a second mapping function from the plurality of mapping functions;
mapping, by the feature map generated by the plurality of layers of the neural network, the set of transformed training images to a set of feature space training data in a feature space; and
identifying, via the processor, a hyper-ellipsoid in the feature space enclosing the set of feature space training data based on the generated feature map as a mechanism for determining whether the incoming image is OOD;
receiving, at inference time of the neural network associated with the identified hyper-ellipsoid used as the traffic sign classification model in the autonomous driving system, a first test image depicting a real world scene outside the set of training images;
applying, via the processor, one or more geometric transformations including vertical and/or horizontal translation of a specific number of pixels to the first test image to create one or more transformed test images;
using the trained neural network associated with the identified hyper-ellipsoid to determine whether the first test image is OOD, including:
mapping the one or more transformed test images into the feature space by the generated feature map;
generating, via the processor, a prediction indicating that the first test image is OOD based on whether the mapped one or more transformed test images in the feature space are outside the hyper-ellipsoid identified from the training of the neural network; and
responsive to determining that the first test image is OOD, generating, by the trained neural network associated with the identified hyper-ellipsoid implemented as the traffic sign classification model in the autonomous driving system, a traffic sign classification output indicating that the first test image depicting the real world scene is an anomaly.
16 . The non-transitory CRM of claim 15 , wherein:
the vector of mapping function elements is generated at least in part by performing a function composition of two or more mapping functions of the plurality of mapping functions to form a mapping function element in the vector.
17 . The non-transitory CRM of claim 15 , wherein:
the set of training images includes a single class training dataset, a multi-class training dataset or an intent classification training dataset; and
the neural network is a classifier neural model pre-trained with the single class training dataset, the multi-class training dataset or the intent classification training dataset.
18 . The non-transitory CRM of claim 15 , wherein:
the set of training images includes a question answering training dataset; and
the neural network is a non-classifier neural model pre-trained with the question answering training dataset.
19 . The non-transitory CRM of claim 15 , wherein the neural network is a non-classifier neural model pre-trained with data lacking an OOD sample.
20 . The non-transitory CRM of claim 15 , wherein the operations further comprise determining one hyper-ellipsoid that has a smallest volume among a set of candidate hyper-ellipsoids that enclose the set of feature space training data in the feature space as the hyper-ellipsoid.