Optimized bias self-detection based on performance and importance
Embodiments of the present disclosure provide systems and methods for implementing self-bias detection based on performance and importance. A disclosed computer implemented method aggregates continuous input data through a K-means clustering algorithm to reduce the number of aggregated sub-group data pairs, enabling a reduced calculation time for computing bias and enhanced performance. The self-bias detection identifies a scale factor and a balance factor of aggregated sub-group data pairs, which indicate the importance of the detected bias.
1. A method comprising:
obtaining input data with continuous feature from a data set;
applying a K-means clustering algorithm to the input data to provide K sub-groups of aggregated sub-group data;
computing a base bias based on two or more aggregated sub-group data pairs of the K sub-groups of the aggregated sub-group data;
collecting a scale factor and a balance factor for the two or more aggregated sub-group data pairs;
computing a new bias based on the computed base bias with a weight based on the scale factor and the balance factor of the two or more aggregated sub-group data pairs;
identifying a final bias based the computed new bias of the two or more aggregated sub-group data pairs; and
providing an output of the final bias to a user.
2. The method of claim 1 , further comprising:
sequentially merging adjacent sub-groups of the aggregated sub-group data, applying an updated K value to the K-means clustering algorithm to provide merged aggregated sub-group data; and for two or more merged aggregated sub-group data pairs, sequentially (i) computing the base bias; (ii) collecting the scale factor and collecting the balance factor; and (iii) computing the new bias.
3. The method of claim 2 , further comprises identifying the final bias based on a largest value of the computed new bias for the two or more merged aggregated sub-group data pairs.
4. The method of claim 3 , wherein each updated K value includes a K′-value with a feature name, and wherein identifying a final bias further comprises identifying the K′-value for the final bias; and storing the K′-value with the feature name in a K-value center data store.
5. The method of claim 1 , wherein applying the K-means clustering algorithm further comprises providing a specified K-value with a feature name to the K-means clustering algorithm and wherein the specified K-value with the feature name is stored in a K-value center data store.
6. The method of claim 1 , wherein a value of the scale factor for the two or more aggregated sub-group data pairs indicates an importance of the final bias, where a higher value indicates more importance.
7. The method of claim 1 , wherein a value of the balance factor for the two or more aggregated sub-group data pairs indicates an importance of the final bias, where a higher value indicates more importance.
8. The method of claim 1 , wherein collecting the scale factor further comprises identifying a mean value of a number of sub-group data elements of the two or more aggregated sub-group data pairs.
9. The method of claim 1 , wherein collecting the balance factor further comprises identifying a ratio value of a number of sub-group data elements of the two or more aggregated sub-group data pairs.
10. The method of claim 1 , wherein identifying the final bias further comprises sequentially computing the new bias based on the computed base bias for two or more merged aggregated sub-group data pairs to identify a largest computed new bias.
11. A system, comprising:
a processor; and
a memory, wherein the memory includes a computer program product configured to perform operations for implementing self-bias detection, the operations comprising:
obtaining input data with continuous feature from a data set;
applying a K-means clustering algorithm to the input data to provide K sub-groups of aggregated sub-group data;
computing a base bias based on two or more aggregated sub-group data pairs of the K sub-groups of the aggregated sub-group data;
collecting a scale factor and a balance factor for the two or more aggregated sub-group data pairs;
computing a new bias based on the computed base bias with a weight based on the scale factor and the balance factor of the two or more aggregated sub-group data pairs;
identifying a final bias based the computed new bias of the two or more aggregated sub-group data pairs; and
providing an output of the final bias to a user.
12. The system of claim 11 , further comprising:
sequentially merging adjacent sub-groups of the aggregated sub-group data, applying an updated K value to the K-means clustering algorithm to provide merged aggregated sub-group data; and for two or more merged aggregated sub-group data pairs, sequentially (i) computing the base bias; (ii) collecting the scale factor and collecting the balance factor; and (iii) computing the new bias.
13. The system of claim 12 , further comprises identifying the final bias based on a largest value of the computed new bias for the two or more merged aggregated sub-group data pairs.
14. The system of claim 12 , wherein each updated K value includes a K′-value with a feature name, and further comprises identifying the K′-value for the final bias; and storing the K′-value with the feature name in a K-value center data store.
15. The system of claim 11 , wherein applying the K-means clustering algorithm further comprises providing a specified K-value with a feature name to the K-means clustering algorithm and wherein the specified K-value with the feature name is stored in a K-value center data store.
16. A computer program product for implementing self-bias detection, the computer program product comprising:
a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising:
obtaining input data with continuous feature from a data set;
applying a K-means clustering algorithm to the input data to provide K sub-groups of aggregated sub-group data;
computing a base bias based on two or more aggregated sub-group data pairs of the K sub-groups of the aggregated sub-group data;
collecting a scale factor and a balance factor for the two or more aggregated sub-group data pairs;
computing a new bias based on the computed base bias with a weight based on the scale factor and the balance factor of the two or more aggregated sub-group data pairs;
identifying a final bias based the computed new bias of the two or more aggregated sub-group data pairs; and
providing an output of the final bias to a user.
17. The computer program product of claim 16 , further comprising:
sequentially merging adjacent sub-groups of the aggregated sub-group data, applying an updated K value to the K-means clustering algorithm to provide merged aggregated sub-group data; and for two or more merged aggregated sub-group data pairs, sequentially (i) computing the base bias; (ii) collecting the scale factor and collecting the balance factor; and (iii) computing the new bias.
18. The computer program product of claim 17 , further comprises identifying the final bias based on a largest value of the computed new bias for the two or more merged aggregated sub-group data pairs.
19. The computer program product of claim 18 , wherein identifying the final bias further comprises identifying a K-value for the final bias; and storing the identified K-value in a K-value center data store.
20. The computer program product of claim 16 , wherein applying the K-means clustering algorithm further comprises providing a specified K-value with a feature name to the K-means clustering algorithm and wherein the specified K-value with the feature name is stored in a K-value center data store.