System and method for differentially private pool-based active learning
The system determines a version space associated with a set of data comprising a pool of unlabeled samples and a first plurality of labeled samples, wherein the version space includes a first set of classifiers corresponding to the first plurality of labeled samples. The system selects, from the pool of unlabeled samples, a second plurality of unlabeled samples comprising informative samples and non-informative samples. A respective informative sample corresponds to a first hyperplane which intersects the version space, and a respective non-informative sample corresponds to a second hyperplane which does not intersect the version space. The system acquires labels corresponding to the second plurality of unlabeled samples to obtain a third plurality of labeled samples. The system updates the first set of classifiers based on the third plurality of labeled samples, thereby improving accuracy of the first set of classifiers.
1. A computer-executable method for facilitating data classification, the method comprising:
determining a version space associated with a set of data comprising a pool of unlabeled samples and a first plurality of labeled samples,
wherein the version space includes a first set of classifiers corresponding to the first plurality of labeled samples;
selecting, from the pool of unlabeled samples, a second plurality of unlabeled samples comprising informative samples and non-informative samples,
wherein a respective informative sample corresponds to a first hyperplane which intersects the version space, and wherein a respective non-informative sample corresponds to a second hyperplane which does not intersect the version space;
acquiring labels corresponding to the second plurality of unlabeled samples to obtain a third plurality of labeled samples; and
updating the first set of classifiers based on the third plurality of labeled samples to obtain a second set of classifiers in the version space, thereby improving accuracy of the first set of classifiers.
2. The method of claim 1 , wherein selecting the second plurality of unlabeled samples is determined using randomized trials with respect to a Bernoulli distribution.
3. The method of claim 1 , wherein selecting the second plurality of unlabeled samples comprises:
selecting the informative samples of the second plurality of unlabeled samples by:
in response to determining that a first informative sample should be selected in a randomized trial with respect to a first random probability distribution, acquiring a label corresponding to the first informative sample; and
in response to determining that a second informative sample should not be selected in a randomized trial with respect to the first random probability distribution, returning the second informative sample to the pool of unlabeled samples.
4. The method of claim 1 , wherein selecting the second plurality of unlabeled samples comprises:
selecting the non-informative samples of the second plurality of unlabeled samples by:
in response to determining that a first non-informative sample should be selected in a randomized trial with respect to a second random probability distribution, acquiring a label corresponding to the first non-informative sample; and
in response to determining that a second non-informative sample should not be selected in a randomized trial with respect to the second random probability distribution, removing the second non-informative sample from the pool of unlabeled samples.
5. The method of claim 1 , wherein the version space represents a volume comprising:
the first set of classifiers indicated as points in an input space associated with the set of data;
the pool of unlabeled samples indicated as a first set of hyperplanes in the input space; and
labeled samples, including one or more of the first and the third plurality of labeled samples, indicated as a second set of hyperplanes in the input space.
6. The method of claim 1 , further comprising:
updating the first set of classifiers based on the third plurality of labeled samples and further based on the first plurality of labeled samples.
7. The method of claim 1 , wherein the first plurality of labeled samples and the third plurality of labeled samples comprise currently labeled samples, and wherein the method further comprises:
training a classifier for the set of training data based on all the currently labeled samples.
8. The method of claim 1 , wherein the first plurality of labeled samples and the third plurality of labeled samples comprise currently labeled samples, and wherein the method further comprises:
training a classifier for the set of training data based on a subset of the currently labeled samples,
wherein the subset contains a plurality of recently labeled samples and excludes a plurality of older labeled samples.
9. The method of claim 1 , wherein updating the first set of classifiers is based on one or more of:
an output perturbation;
an objective perturbation; and
an exponential mechanism.
10. The method of claim 1 , wherein a respective classifier is a Support Vector Machine (SVM) classifier, and wherein the method further comprises:
ordering the unlabeled samples based on a closeness to an optimal classifier for the labeled samples to obtain an ordered list of unlabeled samples; and
for each unlabeled sample in a first portion of the ordered list, in descending order:
in response to determining that a first unlabeled sample should be selected in a randomized trial with respect to a first random probability distribution, acquiring a label corresponding to the first unlabeled sample; and
in response to determining that a second unlabeled sample should not be selected in a randomized trial with respect to the first random probability distribution, returning the second unlabeled sample to the pool of unlabeled samples.
11. The method of claim 10 ,
wherein determining the first portion of the ordered list is based on determining whether a respective sample falls in an informative band associated with the optimal classifier.
12. A computer system for facilitating data classification, the computer system comprising:
a processor; and
a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising
determining a version space associated with a set of data comprising a pool of unlabeled samples and a first plurality of labeled samples,
wherein the version space includes a first set of classifiers corresponding to the first plurality of labeled samples;
selecting, from the pool of unlabeled samples, a second plurality of unlabeled samples comprising informative samples and non-informative samples,
wherein a respective informative sample corresponds to a first hyperplane which intersects the version space, and wherein a respective non-informative sample corresponds to a second hyperplane which does not intersect the version space;
acquiring labels corresponding to the second plurality of unlabeled samples to obtain a third plurality of labeled samples; and
updating the first set of classifiers based on the third plurality of labeled samples to obtain a second set of classifiers in the version space, thereby improving accuracy of the first set of classifiers.
13. The computer system of claim 12 , wherein selecting the second plurality of unlabeled samples is based on a Bernoulli distribution.
14. The computer system of claim 12 , wherein selecting the second plurality of unlabeled samples comprises:
selecting the informative samples of the second plurality of unlabeled samples by:
in response to determining that a first informative sample should be selected in a randomized trial with respect to a first random probability distribution, acquiring a label corresponding to the first informative sample; and
in response to determining that a second informative sample should not be selected in a randomized trial with respect to the first random probability distribution, returning the second informative sample to the pool of unlabeled samples.
15. The computer system of claim 12 , wherein selecting the second plurality of unlabeled samples comprises:
selecting the non-informative samples of the second plurality of unlabeled samples by:
in response to determining that a first non-informative sample should be selected in a randomized trial with respect to a second random probability distribution, acquiring a label corresponding to the first non-informative sample; and
in response to determining that a second non-informative sample should not be selected in a randomized trial with respect to the second random probability distribution, removing the second non-informative sample from the pool of unlabeled samples.
16. The computer system of claim 12 , wherein the version space represents a volume comprising:
the first set of classifiers indicated as points in an input space associated with the set of data;
the pool of unlabeled samples indicated as a first set of hyperplanes in the input space; and
labeled samples, including one or more of the first and the third plurality of labeled samples, indicated as a second set of hyperplanes in the input space.
17. The computer system of claim 12 , wherein the first plurality of labeled samples and the third plurality of labeled samples comprise currently labeled samples, and wherein the method further comprises:
training a classifier for the set of training data based on all the currently labeled samples.
18. The computer system of claim 12 , wherein the first plurality of labeled samples and the third plurality of labeled samples comprise currently labeled samples, and wherein the method further comprises:
training a classifier for the set of training data based on a subset of the currently labeled samples,
wherein the subset contains a plurality of recently labeled samples and excludes a plurality of older labeled samples.
19. The computer system of claim 12 , wherein updating the first set of classifiers is based on one or more of:
an output perturbation;
an objective perturbation; and
an exponential mechanism.
20. The computer system of claim 12 , wherein a respective classifier is a Support Vector Machine (SVM) classifier, and wherein the method further comprises:
ordering the unlabeled samples based on a closeness to an optimal classifier for the labeled samples to obtain an ordered list of unlabeled samples; and
for each unlabeled sample in a first portion of the ordered list, in descending order:
in response to determining that a first unlabeled sample should be selected in a randomized trial with respect to a first random probability distribution, acquiring a label corresponding to the first unlabeled sample; and
in response to determining that a second unlabeled sample should not be selected in a randomized trial with respect to the first random probability distribution, returning the second unlabeled sample to the pool of unlabeled samples,
wherein determining the first portion of the ordered list is based on determining whether a respective sample falls in an informative band associated with the optimal classifier.