Systems and methods for image recognition normalization and calibration
Systems, methods, and non-transitory computer-readable media can calculate raw scores for a plurality of media items based on a classifier model and a target concept. The plurality of media items are ranked based on the raw scores. A review set of the plurality of media items is determined, the review set comprising a subset of the plurality of media items. Each of the media items of the review set is associated with a content depiction determination. A normalized score formula is calculated based on the raw scores and the content depiction determinations for the media items of the review set.
1. A computer-implemented method comprising:
calculating, by a computing system, raw scores for a plurality of media items based on a classifier model and a target concept;
ranking, by the computing system, the plurality of media items based on the raw scores;
determining, by the computing system, a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating, by the computing system, each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating, by the computing system, a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.
2. The computer-implemented method of claim 1 , wherein the review set is determined based on a sampling rate.
3. The computer-implemented method of claim 2 , further comprising receiving a sampling rate selection from a user.
4. The computer-implemented method of claim 1 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.
5. The computer-implemented method of claim 1 , further comprising presenting a user interface configured to receive content depiction determinations for the media items of the review set.
6. The computer-implemented method of claim 1 , further comprising re-training the classifier model based on the normalized score formula.
7. The computer-implemented method of claim 6 , further comprising repeating the computer-implemented method with the re-trained classifier model.
8. The computer-implemented method of claim 1 , wherein the normalized score formula is configured to convert a raw score calculated by the classifier model into a normalized score.
9. The computer-implemented method of claim 8 , wherein the normalized score is a probability value.
10. The computer-implemented method of claim 1 , wherein the review set comprises a fixed number of media items.
11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
ranking the plurality of media items based on the raw scores;
determining a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.
12. The system of claim 11 , wherein the review set is determined based on a sampling rate.
13. The system of claim 12 , wherein the method further comprises receiving a sampling rate selection from a user.
14. The system of claim 11 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.
15. The system of claim 11 , wherein the method further comprises presenting a user interface configured to receive content depiction determinations for the media items of the review set.
16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
calculating raw scores for a plurality of media items based on a classifier model and a target concept;
ranking the plurality of media items based on the raw scores;
determining a review set of the plurality of media items, the review set comprising a subset of the plurality of media items;
associating each of the media items of the review set with a content depiction determination, wherein the content depiction determination is indicative of whether a media item depicts the target concept; and
calculating a normalized score formula based on the raw scores and the content depiction determinations for the media items of the review set.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the review set is determined based on a sampling rate.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises receiving a sampling rate selection from a user.
19. The non-transitory computer-readable storage medium of claim 16 , wherein calculating the normalized score formula comprises calculating a logistic regression formula based on the raw scores and the content depiction determinations for the media items of the review set.
20. The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises presenting a user interface configured to receive content depiction determinations for the media items of the review set.