METHOD AND SYSTEM FOR FACILITATING BATCH MODE ACTIVE LEARNING
A method and system for performing batch mode active learning to train a classifier. According to embodiments of the present invention, unlabeled documents are selected from a corpus based on rewards associated with each unlabeled document. The reward is an indication of the increase to the accuracy of a classifier which may result if the document is used to train the classifier. When calculating a given reward, embodiments of the present invention address the uncertainty and diversity of a given document. Embodiments of the present invention reduce the resources utilized to perform classifier training.
1 . A computer-implemented method for selecting a batch of unlabeled documents from a plurality of unlabeled documents, comprising:
(a) calculating, by the computer, a reward associated with each unlabeled document within the plurality of unlabeled documents;
(b) receiving, by the computer, a desired batch size for the batch of unlabeled documents; and
(c) iteratively including, by the computer, an unlabeled document in the batch of unlabeled documents based on the reward associated with the unlabeled document, until the desired batch size is achieved.
2 . The computer-implemented method of claim 1 , further comprising:
(d) receiving, by the computer, a batch of labeled documents based on the batch of unlabeled documents; and
(e) training, by the computer, a classifier based on the batch of labeled documents.
3 . The computer-implemented method of claim 2 , further comprising:
(f) receiving, by the computer, a desired accuracy for the classifier;
(g) calculating, by the computer, an accuracy of the classifier; and
(h) determining, by the computer, that the accuracy of the classifier meets the desired accuracy.
4 . The computer-implemented method of claim 2 , further comprising:
(i) receiving, by the computer, a desired accuracy for the classifier;
(j) calculating, by the computer, an accuracy of the classifier;
(k) determining, by the computer, that the accuracy of the classifier does not meet the desired accuracy; and
repeating steps (a) through (e) until the desired accuracy of the classifier is met.
5 . The computer-implemented method of claim 1 , wherein step (c), comprises:
selecting, by the computer, a first unlabeled document, wherein the reward associated with the first unlabeled document is the highest reward from among the plurality of unlabeled documents,
updating, by the computer, the reward associated with the first unlabeled document based on a diversity between the first unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining, by the computer, that the updated reward associated with the first document remains the highest from among the plurality of unlabeled documents, and
including, by the computer, the first unlabeled document in the batch of unlabeled documents.
6 . The computer-implemented method of claim 1 , wherein step (c), comprises:
selecting, by the computer, a first unlabeled document, wherein the reward associated with the first unlabeled document is the highest reward from among the plurality of unlabeled documents,
updating, by the computer, the reward associated with the first unlabeled document based on a diversity between the first unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining, by the computer, that the updated reward associated with the first document is not the highest from among the plurality of unlabeled documents,
selecting, by the computer, a second unlabeled document having the highest reward from among the plurality of unlabeled documents,
updating, by the computer, the reward associated with the second unlabeled document based on the diversity between the second unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining, by the computer, that the updated reward associated with the second document remains the highest from among the plurality of unlabeled documents, and
including, by the computer, the second unlabeled document in the batch of unlabeled documents.
7 . The computer-implemented method of claim 1 , wherein the reward is based on an uncertainty associated with an unlabeled document.
8 . The computer-implemented method of claim 1 , wherein the reward is based on a length of an unlabeled document.
9 . A system for selecting a batch of unlabeled documents from a plurality of unlabeled documents, comprising:
a batch creation module configured to:
(a) calculate a reward associated with each unlabeled document within the plurality of unlabeled documents,
(b) receive a desired batch size for the batch of unlabeled documents; and
(c) iteratively include an unlabeled document in the batch of unlabeled documents based on the reward associated with the unlabeled document, until the desired batch size is achieved.
10 . The system of claim 9 , further comprising:
a labeling module configured to:
(d) receive a batch of labeled documents based on the batch of unlabeled documents; and
a classifier training module configure to:
(e) train a classifier based on the batch of labeled documents.
11 . The systems of claim 9 , wherein the classifier training module is further configured to:
(f) receive a desired accuracy for the classifier;
(g) calculate an accuracy of the classifier; and
(h) determine that the accuracy of the classifier meets the desired accuracy.
12 . The systems of claim 9 , wherein the classifier training module is further configured to:
(i) receive a desired accuracy for the classifier;
(j) calculate an accuracy of the classifier; and
(k) determine that the accuracy of the classifier does not meet the desired accuracy.
13 . The system of claim 12 , wherein the batch creation module to repeats functions (a) through (c), the labeling module to repeat function (d), and the classifier training module repeats function (e) until the desired accuracy of the classifier is met.
14 . The system of claim 9 , wherein function (c), comprises:
selecting a first unlabeled document, wherein the reward associated with the first unlabeled document is the highest reward from among the plurality of unlabeled documents,
updating the reward associated with the first unlabeled document based on a diversity between the first unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining that the updated reward associated with the first document remains the highest from among the plurality of unlabeled documents, and
including the first unlabeled document in the batch of unlabeled documents.
15 . The system of claim 9 , wherein function (c), comprises:
selecting a first unlabeled document, wherein the reward associated with the first unlabeled document is the highest reward from among the plurality of unlabeled documents,
updating the reward associated with the first unlabeled document based on a diversity between the first unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining that the updated reward associated with the first document is not the highest from among the plurality of unlabeled documents, selecting a second unlabeled document having the highest reward from among the plurality of unlabeled documents,
updating the reward associated with the second unlabeled document based on the diversity between the second unlabeled document and each unlabeled document within the batch of unlabeled documents,
determining that the updated reward associated with the second document remains the highest from among the plurality of unlabeled documents, and
including the second unlabeled document in the batch of unlabeled documents.
16 . The system of claim 9 , wherein the reward is based on an uncertainty associated with an unlabeled document.
17 . The system claim 9 , wherein the reward is based on a length of an unlabeled document.