Diversity-aware weighted majority vote classifier for decision making on imbalanced datasets
An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.
1 . An ensemble learning based method, executed by one or more computer processors, for a binary classification task on an imbalanced dataset, wherein the imbalanced dataset comprises a minority class comprising positive samples and a majority class comprising negative samples, the ensemble learning based method comprising:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples using computer-implemented algorithms, thereby generating a modified dataset D′ of the minority class;
using the modified dataset D′ of the minority class to generate a plurality of subsamples in a random and stratified way such that each subsample, of the subsamples, contains random examples while preserving a class-distribution ratio of the modified dataset D′, and learning a base classifier on each of the subsamples using machine learning algorithms executed on the one or more computer processors to determine a plurality of base classifiers;
learning a diversity-aware weighted majority vote classifier by using PAC-Bayesian theory to control operation of the one or more computer processors, the learning comprising minimizing an objective function including:
a classification-error term of the diversity-aware weighted majority vote classifier, and
a pairwise diversity term computed over positive examples only; and
executing the binary classification task by applying the diversity-aware weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as ones of the positive samples and the negative samples, and
wherein the diversity-aware weighted majority vote classifier is configured to process input data and generate classification outputs with low false positive rates.
2 . The ensemble learning based method according to claim 1 , wherein a minimization of a diversity between the base classifiers on the positive samples is determined by an optimization problem.
3 . The ensemble learning based method according to claim 2 , wherein the optimization problem is based on a pairwise comparison of the base classifiers and on a classification error of the base classifiers.
4 . The ensemble learning based method according to claim 1 , wherein synthetically generating the minority class examples comprises:
selecting, from the positive samples of the imbalanced dataset, safe examples using k-nearest neighbors, wherein a positive sample, of the positive samples, is considered to be a safe example, of the safe examples, based upon determining that a majority vote of k-nearest neighbors for that example is positive.
5 . The ensemble learning based method according to claim 4 , wherein synthetically generating the minority class examples further comprises:
learning, for the selected safe examples, parameters for a multivariate probability distribution, and
generating the minority class examples based on the multivariate probability distribution using the parameters learned from the safe examples.
6 . The ensemble learning based method according to claim 1 , wherein the random examples are of the modified dataset D′.
7 . The ensemble learning based method according to claim 1 , wherein the subsamples are generated in such a way that the examples contained in each of the subsamples have the same ratio of class distribution as the examples contained in the modified dataset D′.
8 . The ensemble learning based method according to claim 1 , wherein the base classifiers learned on the subsamples are based Decision Trees, Logistic Regression, or Support Vector Machines.
9 . The ensemble learning based method according to claim 1 , wherein the base classifier is learned on each of the subsamples.
10 . The ensemble learning based method according to claim 1 , wherein heterogeneous base classifiers are learned on different subsamples.
11 . The ensemble learning based method according to claim 1 , wherein the imbalanced dataset comprises medical diagnosis data, comprising samples of photographic, computer-graphic, or medical-tissue images, and wherein the binary classification task is a differentiation between the positive samples, as diagnosed with a condition in question, and the negative samples as not diagnosed with the condition in question.
12 . The ensemble learning based method according to claim 1 , wherein the imbalanced dataset comprises predictive maintenance data, comprising equipment status samples of a set of assets in operation, and wherein the binary classification task is a differentiation between the positive samples, as related to assets that are qualified to be defective or to become defective in the near future, and the negative samples as related to assets that are qualified to function properly without becoming defective in the near future.
13 . The ensemble learning based method according to claim 1 , wherein the imbalanced dataset comprises credit cards transactions data, and wherein the binary classification task is a differentiation between the positive samples, as related to transactions that are qualified to be fraudulent, and the negative samples as related to transactions that are qualified to be legitimate.
14 . A system for configured to execute an ensemble learning based method, by one or more computer processors of the system, for a binary classification task on an imbalanced dataset, the imbalanced dataset comprising a minority class comprising positive samples and a majority class comprising negative samples, the ensemble learning based method comprising:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples using computer-implemented algorithms, thereby generating a modified dataset D′ of the minority class;
using the modified dataset D′ of the minority class to generate a plurality of subsamples in a random and stratified way such that each subsample, of the subsamples, contains random examples while preserving a class-distribution ratio of the modified dataset D′, and learning a base classifier on each of the subsamples using machine learning algorithms executed on the one or more computer processors to determine a plurality of base classifiers;
learning a diversity-aware weighted majority vote classifier by using PAC-Bayesian theory to control operation of the one or more computer processors, the learning comprising minimizing an objective function including:
a classification-error term of the diversity-aware weighted majority vote classifier, and
a pairwise diversity term computed over positive examples only; and
executing the binary classification task by applying the diversity-aware weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as ones of the positive samples and the negative samples, and
wherein the diversity-aware weighted majority vote classifier is configured to process input data and generate classification outputs with low false positive rates.
15 . A non-transitory computer readable medium having, stored thereon, instructions for performing an ensemble learning based method, executed by one or more computer processors, for a binary classification task on an imbalanced dataset, the imbalanced dataset comprising a minority class comprising positive samples and a majority class comprising negative samples, the ensemble learning based method comprising:
generatively oversampling minority class examples included in the imbalanced dataset by synthetically generating the minority class examples using computer-implemented algorithms, thereby generating a modified dataset D′ of the minority class;
using the modified dataset D′ of the minority class to generate a plurality of subsamples in a random and stratified way such that each subsample, of the subsamples, contains random examples while preserving a class-distribution ratio of the modified dataset D′, and learning a base classifier on each of the subsamples using machine learning algorithms executed on the one or more computer processors to determine a plurality of base classifiers;
learning a diversity-aware weighted majority vote classifier by using PAC-Bayesian theory to control operation of the one or more computer processors, the learning comprising minimizing an objective function including:
a classification-error term of the diversity-aware weighted majority vote classifier, and
a pairwise diversity term computed over positive examples only; and
executing the binary classification task by applying the diversity-aware weighted majority vote classifier to the imbalanced dataset,
wherein the binary classification task outputs a final prediction indicating samples as ones of the positive samples and the negative samples, and
wherein the diversity-aware weighted majority vote classifier is configured to process input data and generate classification outputs with low false positive rates.