Multiclass classification with diversified precision and recall weightings
Described herein are systems, methods, and other techniques for evaluating a classifier model. The classifier model may be provided with a set of elements to be classified into N classes. Classification results may be obtained from the classifier model. N class-specific precisions and N class-specific recalls for the N classes may be computed based on the classification results. N class-specific precision weights and N class-specific recall weights corresponding to the N classes may be obtained. A weighted f-measure may be computed by weighting the N class-specific precisions with the N class-specific precision weights and weighting the N class-specific recalls with the N class-specific recall weights.
1. A computer-implemented method of evaluating a classifier model, the computer-implemented method comprising:
providing the classifier model with a set of elements to be classified into N classes, wherein Nis a number greater than or equal to two;
obtaining results from the classifier model based on the classifier model classifying the set of elements into the N classes;
computing N class-specific precisions and N class-specific recalls for the N classes based on the results;
obtaining, for each of the N classes, a class-specific precision weight and a class-specific recall weight, wherein one or both of N class-specific precision weights or N class-specific recall weights obtained for the N classes are nonuniform; and
computing a weighted f-measure value by weighting, as part of the weighted f-measure value, the N class-specific precisions with the N class-specific precision weights and weighting, as part of the weighted f-measure value, the N class-specific recalls with the N class-specific recall weights.
2. The computer-implemented method of claim 1 , wherein N is greater than or equal to three.
3. The computer-implemented method of claim 1 , wherein computing the N class-specific precisions and the N class-specific recalls for the N classes based on the results includes:
counting a number of true positives in the results;
counting a number of false positives in the results; and
counting a number of false negatives in the results.
4. The computer-implemented method of claim 1 , wherein each of the N class-specific precision weights and each of the N class-specific recall weights is greater than or equal to one.
5. The computer-implemented method of claim 1 , wherein the set of elements include text, image, video, or audio data elements.
6. The computer-implemented method of claim 1 , wherein the classifier model is a machine learning (ML) model.
7. The computer-implemented method of claim 6 , wherein the classifier model is a long short-term memory (LSTM) network.
8. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
providing a classifier model with a set of elements to be classified into N classes, wherein Nis a number greater than or equal to two;
obtaining results from the classifier model based on the classifier model classifying the set of elements into the N classes;
computing N class-specific precisions and N class-specific recalls for the N classes based on the results;
obtaining, for each of the N classes, a class-specific precision weight and a class-specific recall weight, wherein one or both of N class-specific precision weights or N class-specific recall weights obtained for the N classes are nonuniform; and
computing a weighted f-measure value by weighting, as part of the weighted f-measure value, the N class-specific precisions with the N class-specific precision weights and weighting, as part of the weighted f-measure value, the N class-specific recalls with the N class-specific recall weights.
9. The non-transitory computer-readable medium of claim 8 , wherein Nis greater than or equal to three.
10. The non-transitory computer-readable medium of claim 8 , wherein computing the N class-specific precisions and the N class-specific recalls for the N classes based on the results includes:
counting a number of true positives in the results;
counting a number of false positives in the results; and
counting a number of false negatives in the results.
11. The non-transitory computer-readable medium of claim 8 , wherein each of the N class-specific precision weights and each of the N class-specific recall weights is greater than or equal to one.
12. The non-transitory computer-readable medium of claim 8 , wherein the set of elements include text, image, video, or audio data elements.
13. The non-transitory computer-readable medium of claim 8 , wherein the classifier model is a machine learning (ML) model.
14. The non-transitory computer-readable medium of claim 13 , wherein the classifier model is a long short-term memory (LSTM) network.
15. A system comprising:
one or more processors; and
a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
providing a classifier model with a set of elements to be classified into N classes, wherein Nis a number greater than or equal to two;
obtaining results from the classifier model based on the classifier model classifying the set of elements into the N classes;
computing N class-specific precisions and N class-specific recalls for the N classes based on the results;
obtaining, for each of the N classes, a class-specific precision weight and a class-specific recall weight, wherein one or both of N class-specific precision weights or N class-specific recall weights obtained for the N classes are nonuniform; and
computing a weighted f-measure value by weighting, as part of the weighted f-measure value, the N class-specific precisions with the N class-specific precision weights and weighting, as part of the weighted f-measure value, the N class-specific recalls with the N class-specific recall weights.
16. The system of claim 15 , wherein Nis greater than or equal to three.
17. The system of claim 15 , wherein computing the N class-specific precisions and the N class-specific recalls for the N classes based on the results includes:
counting a number of true positives in the results;
counting a number of false positives in the results; and
counting a number of false negatives in the results.
18. The system of claim 15 , wherein each of the N class-specific precision weights and each of the N class-specific recall weights is greater than or equal to one.
19. The system of claim 15 , wherein the set of elements include text, image, video, or audio data elements.
20. The system of claim 15 , wherein the classifier model is a machine learning (ML) model.