Clustering historical images using a convolutional neural net and labeled data bootstrapping
Systems and methods for classifying historical images. A feature extractor may create feature vectors corresponding to a plurality of images. A first classification of the plurality of images may be performed based on the plurality of feature vectors, which may include assigning a label to each of the plurality of images and assigning a probability for each of the assigned labels. The assigned probability for each of the assigned labels may be related to a statistical confidence that a particular assigned label is correctly assigned to a particular image. A subset of the plurality of images may be displayed to a display device. An input corresponding to replacement of an incorrect label with a corrected label for a certain image may be received from a user. A second classification of the plurality of images based on the input from the user may be performed.
1. A method of using pre-trained neural networks to train classifiers to classify images, the method comprising:
providing a plurality of genealogical images to an image classifier; and
for a genealogical image of the plurality of genealogical images:
creating a feature vector for the genealogical image using a feature extractor of the image classifier, wherein the feature extractor includes a pre-trained neural network previously trained to extract feature vectors from images within a non-genealogical image database;
based on the feature vector created using the feature extractor trained on the non-genealogical image database, assigning a label to the genealogical image using a feature classifier of the image classifier, wherein the feature classifier is separate from the feature extractor and the pre-trained neural network;
receiving a corrected label for the genealogical image;
determining an error between the label and the corrected label; and
adjusting the feature classifier to improve classification of genealogical images based on the error without adjustment to the feature extractor and the pre-trained neural network.
2. The method of claim 1 , further comprising:
utilizing a subset selector to select a subset of genealogical images corresponding to assigned labels from the feature classifier that fail to satisfy a probability threshold; and
wherein receiving the corrected label for the genealogical image comprises receiving a new label for one of the subset of genealogical images corresponding to assigned labels that fail to satisfy the probability threshold.
3. The method of claim 1 , wherein the pre-trained neural network was previously trained using photographs.
4. The method of claim 1 , wherein labels for the plurality of genealogical images include a text a label, a typed label, a form label, and a census record label.
5. The method of claim 1 , further comprising:
for the genealogical image:
assigning a probability for the label, wherein the probability indicates a statistical confidence that the label is correctly assigned to the genealogical image.
6. The method of claim 5 , further comprising:
for the genealogical image:
determining that the probability is less than a threshold; and
in response to determining that the probability is less than the threshold, providing the genealogical image for display at a user interface.
7. The method of claim 1 , wherein the corrected label is received via a user interface.
8. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
providing a plurality of genealogical images to an image classifier; and
for a genealogical image of the plurality of genealogical images:
creating a feature vector for the genealogical image using a feature extractor of the image classifier, wherein the feature extractor includes a pre-trained neural network trained to extract feature vectors from images within a non-genealogical image database;
based on the feature vector created using the feature extractor trained on the non-genealogical image database, assigning a label to the input genealogical image using a feature classifier of the image classifier, wherein the feature classifier is separate from the feature extractor and the pre-trained neural network;
receiving a corrected label for the genealogical image;
determining an error between the label and the corrected label; and
adjusting the feature classifier to improve classification of genealogical images based on the error without adjustment to the feature extractor and the pre-trained neural network.
9. The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
utilizing a subset selector to select a subset of genealogical images corresponding to assigned labels from the feature classifier that fail to satisfy a probability threshold; and
wherein receiving the corrected label for the genealogical image comprises receiving a new label for one of the subset of genealogical images corresponding to assigned labels that fail to satisfy the probability threshold.
10. The non-transitory computer-readable medium of claim 8 , wherein the pre-trained neural network was previously trained using photographs.
11. The non-transitory computer-readable medium of claim 8 , wherein labels for the plurality of genealogical images include a text a label, a typed label, a form label, and a census record label.
12. The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
for the genealogical image:
assigning a probability for the label, wherein the probability indicates a statistical confidence that the label is correctly assigned to the genealogical image.
13. The non-transitory computer-readable medium of claim 12 , wherein the operations further comprise:
for the genealogical image:
determining that the probability is less than a threshold; and
in response to determining that the probability is less than the threshold, providing the genealogical image for display at a user interface.
14. The non-transitory computer-readable medium of claim 8 , wherein the corrected label is received via a user interface.
15. A system comprising:
one or more processors; and
a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
providing a plurality of genealogical images to an image classifier; and
for a genealogical image of the plurality of genealogical images:
creating a feature vector for the genealogical image using a feature extractor of the image classifier, wherein the feature extractor includes a pre-trained neural network trained to extract feature vectors from images within a non-genealogical image database;
based on the feature vector created using the feature extractor trained on the non-genealogical image database, assigning a label to the genealogical image using a feature classifier of the image classifier, wherein the feature classifier is separate from the feature extractor and the pre-trained neural network;
receiving a corrected label for the genealogical image;
determining an error between the label and the corrected label; and
adjusting the feature classifier to improve classification of genealogical images based on the error without adjustment to the feature extractor and the pre-trained neural network.
16. The system of claim 15 , wherein the operations further comprise:
utilizing a subset selector to select a subset of genealogical images corresponding to assigned labels from the feature classifier that fail to satisfy a probability threshold; and
wherein receiving the corrected label for the genealogical image comprises receiving a new label for one of the subset of genealogical images corresponding to assigned labels that fail to satisfy the probability threshold.
17. The system of claim 16 , wherein labels for the plurality of genealogical images include a text a label, a typed label, a form label, and a census record label.
18. The system of claim 15 , wherein the pre-trained neural network was previously trained using photographs.
19. The system of claim 15 , wherein the operations further comprise:
for the genealogical image:
assigning a probability for the label, wherein the probability indicates a statistical confidence that the label is correctly assigned to the genealogical image.
20. The system of claim 19 , wherein the operations further comprise:
for the genealogical image:
determining that the probability is less than a threshold; and
in response to determining that the probability is less than the threshold, providing the genealogical image for display at a user interface.