Method and system for creating an ensemble of neural network-based classifiers that optimizes a diversity metric
One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric. The system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.
1 . A computer-executable method, comprising:
defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers;
including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;
replacing a final classification layer in one or more neural networks with the three or more affine classifiers;
training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;
predicting an output for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric;
displaying, on a device associated with a user, information related to the predicted output for the respective data object, the information comprising one or more of:
a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;
a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;
a size of the ensemble of neural network-based classifiers;
an approach associated with the respective data object, including whether the respective data object is associated with diversity;
a type of classifier used in a final classifier layer;
the predicted outcome based on a respective classifier; and
an overall ensemble result;
receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classifier layer, a selected classifier, and the regularization term;
predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and
increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
2 . The method of claim 1 ,
wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.
3 . The method of claim 1 ,
wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.
4 . The method of claim 1 ,
wherein the three or more affine classifiers comprise a multi-class classification layer.
5 . The method of claim 1 ,
wherein the three or more affine classifiers comprise a one-versus-all classification layer.
6 . The method of claim 1 ,
wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.
7 . The method of claim 1 ,
wherein the three or more affine classifiers comprise an odd number of affine classifiers, and
wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.
8 . The method of claim 1 , further comprising:
predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.
9 . The method of claim 8 , further comprising:
wherein predicting the outcome for the testing data object is further based an ensemble decision rule.
10 . The method of claim 1 ,
wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and
wherein training the one or more neural networks is further based on the training data.
11 . A computer system for facilitating data classification, the computer system comprising:
a processor; and
a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising:
defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers;
including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;
replacing a final classification layer in one or more neural networks with the three or more affine classifiers;
training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;
predicting an outcome for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric;
displaying, on a device associated with a user, information related to the predicted outcome for the respective data object, the information comprising one or more of:
a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;
a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;
a size of the ensemble of neural network-based classifiers;
an approach associated with the respective data object, including whether the respective data object is associated with diversity;
a type of classifier used in a final classifier layer;
the predicted outcome based on a respective classifier; and
an overall ensemble result;
receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term;
predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and
increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
12 . The computer system of claim 11 ,
wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.
13 . The computer system of claim 11 ,
wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.
14 . The computer system of claim 11 ,
wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.
15 . The method of claim 11 ,
wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.
16 . The computer system of claim 11 ,
wherein the three or more affine classifiers comprise an odd number of affine classifiers, and
wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.
17 . The computer system of claim 11 , wherein the method further comprises:
predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.
18 . The computer system of claim 11 ,
wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and
wherein training the one or more neural networks is further based on the training data.
19 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers for training data;
including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;
replacing a final classification layer in one or more neural networks with the three or more affine classifiers;
training, based on the training data, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;
predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric;
displaying, on a device associated with a user, information related to the predicted outcome for a respective data object, the information comprising one or more of:
a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;
a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;
a size of the ensemble of neural network-based classifiers;
an approach associated with the respective data object, including whether the respective data object is associated with diversity;
a type of classifier used in a final classifier layer;
the predicted outcome based on a respective classifier; and
an overall ensemble result;
receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term;
predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and
increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.
20 . The non-transitory computer-readable storage medium of claim 19 ,
wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal,
wherein at least one of the following:
one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers; and
a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers, and
wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.