Diversity-aware weighted majority vote classifier for imbalanced datasets supporting decision making
An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.
1 . An ensemble learning based method executed by one or more computer processors for a binary classification task on an imbalanced dataset, wherein the imbalanced dataset, comprises a minority class comprising positive samples and a majority class comprising negative samples, the method comprising:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples to generate a dataset of the minority class, using the dataset of the minority class to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers,
learning a weighted majority vote classifier by combining outputs of the base classifiers, and
executing the binary classification task by applying the learned weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as positive samples or negative samples, and
wherein each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.
2 . The method according to claim 1 , wherein a minimization of the diversity between the base classifiers on the positive samples is determined by an optimization problem.
3 . The method according to claim 2 , wherein the optimization problem is based on a pairwise comparison of the base classifiers and on a classification error of the base classifiers.
4 . The method according to claim 1 , wherein the synthetically generating minority class examples comprises:
selecting, from the positive samples of the imbalanced dataset, safe examples using k-nearest neighbors,
wherein the positive sample is considered to be a safe example if based upon determining the majority vote of k-nearest neighbors for that example is positive.
5 . The method according to claim 4 , wherein the synthetically generating minority class examples further comprises:
learning, for the selected safe examples, parameters for a multivariate probability distribution, and
generating the minority class examples based on a multivariate probability distribution using the parameters learned from the safe examples.
6 . The method according to claim 1 ,
wherein the subsamples are generated in such a way that each of the subsamples contains random examples from the generated dataset.
7 . The method according to claim 1 ,
wherein the subsamples are generated in such a way that the examples contained in each of the subsamples have the same ratio of class distribution as the examples contained in the generated dataset.
8 . The method according to claim 1 ,
wherein the base classifiers learned on the subsamples are based Decision Trees, Logistic Regression, or Support Vector Machines.
9 . The method according to claim 1 ,
wherein the same base classifier is learned on each of the subsamples.
10 . The method according to claim 1 ,
wherein heterogeneous base classifiers are learned on different subsamples.
11 . The method according to claim 1 ,
wherein the imbalanced dataset comprises medical diagnosis data, comprising samples of photographic, computer-graphic, or medical-tissue images, and wherein the binary classification task is a differentiation between positive samples diagnosed with a condition in question and negative samples not diagnosed with the condition in question.
12 . The method according to claim 1 ,
wherein the imbalanced dataset comprises predictive maintenance data, comprising equipment status samples of a set of assets in operation, and wherein the binary classification task is a differentiation between positive samples related to assets that are qualified to be defective or to become defective in the near future and negative samples related to assets that are qualified to function properly without becoming defective in the near future.
13 . The method according to claim 1 ,
wherein the imbalanced dataset comprises credit cards transactions data, and
wherein the binary classification task is a differentiation between positive samples related to transactions that are qualified to be fraudulent and negative samples related to transactions that are qualified to be legitimate.
14 . A system for executing an ensemble learning based method for a binary classification task on an imbalanced dataset, the imbalanced dataset comprising a minority class comprising positive samples and a majority class comprising negative samples, the system comprising of one or more computer processors which, alone or in combination, provide for the execution of the following:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples to generate a dataset of the minority class, using the dataset of the minority class to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers,
learning a weighted majority vote classifier by combining outputs of the base classifiers, and
executing the binary classification task by applying the learned weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as positive samples or negative samples, and
wherein each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples only is minimized.
15 . A non-transitory computer readable medium having, stored thereon, instructions for performing an ensemble learning based method to be executed by one or more computer processors for a binary classification task on an imbalanced dataset, the imbalanced dataset comprising a minority class comprising positive samples and a majority class comprising negative samples, the method comprising:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples to generate a dataset of the minority class, using the dataset of the minority class to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers,
learning a weighted majority vote classifier by combining the outputs of the base classifiers, and
executing the binary classification task by applying the learned weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as positive samples or negative samples, and
wherein each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples only is minimized.