Burden Score for an Opaque Model
A method, system and computer-readable storage medium for performing a cognitive information processing operation. The cognitive information processing operation includes: receiving data from a plurality of data sources; processing the data from the plurality of data sources to provide cognitively processed insights via an augmented intelligence system, the augmented intelligence system executing on a hardware processor of an information processing system, the augmented intelligence system and the information processing system providing a cognitive computing function; performing an impartiality assessment operation via an impartiality assessment engine, the impartiality assessment operation detecting a presence of bias in an outcome of the cognitive computing function, the impartiality assessment operation generating a burden score representing the presence of bias in the outcome; and, providing the cognitively processed insights to a destination, the destination comprising a cognitive application, the cognitive application enabling a user to interact with the cognitive insights.
1 . A computer-implementable method for cognitive information processing, comprising:
receiving data from a data source;
processing the data from the data source to provide cognitively processed insights via an augmented intelligence system, the augmented intelligence system executing on a hardware processor of an information processing system, the augmented intelligence system providing a cognitive computing function;
performing an impartiality assessment operation via an impartiality assessment engine, the impartiality assessment operation detecting a presence of bias in an outcome of the cognitive computing function, the impartiality assessment operation generating a burden score representing the presence of bias in the outcome, the burden score representing the presence of bias in the outcome representing an impartiality assessment of the cognitive computing function; and,
providing the burden score representing the presence of bias in the outcome to a destination, the destination comprising a cognitive application, the cognitive application providing a user with the impartiality assessment of the cognitive computing function based upon the burden score representing the presence of bias in the outcome.
2 . The method of claim 1 , wherein:
the impartiality assessment engine is implemented within the augmented intelligence system, the augmented intelligence system comprising a cognitive process foundation.
3 . The method of claim 2 , wherein:
the cognitive process foundation performs an augmented intelligence assurance operation using the burden score.
4 . The method of claim 1 , wherein:
the impartiality assessment operation includes assessing an effect of the presence of bias on the outcome of the cognitive computing function.
5 . The method of claim 4 , wherein:
the impartiality assessment engine receives a counterfactual; and,
the impartiality assessment operation contrasts features between an input subject data point and the counterfactual and identifies contrasts between the input data point and the counterfactual.
6 . The method of claim 1 , wherein:
the cognitive computing function is performed using an opaque model; and,
the burden score represents a bias of the opaque model for a particular group of individuals.
7 . A system comprising:
a hardware processor;
a data bus coupled to the hardware processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the hardware processor and configured for:
receiving data from a data source;
processing the data from the data source to provide cognitively processed insights via an augmented intelligence system, the augmented intelligence system executing on a hardware processor of an information processing system, the augmented intelligence system providing a cognitive computing function;
performing an impartiality assessment operation via an impartiality assessment engine, the impartiality assessment operation detecting a presence of bias in an outcome of the cognitive computing function, the impartiality assessment operation generating a burden score representing the presence of bias in the outcome, the burden score representing the presence of bias in the outcome representing an impartiality assessment of the cognitive computing function; and,
providing the burden score representing the presence of bias in the outcome to a destination, the destination comprising a cognitive application, the cognitive application providing a user with the impartiality assessment of the cognitive computing function based upon the burden score representing the presence of bias in the outcome.
8 . The system of claim 7 , wherein:
the impartiality assessment engine is implemented within the augmented intelligence system, the augmented intelligence system comprising a cognitive process foundation.
9 . The system of claim 8 , wherein:
the impartiality assessment engine is implemented within the augmented intelligence system, the augmented intelligence system comprising a cognitive process foundation.
10 . The system of claim 7 , wherein:
the impartiality assessment operation includes assessing an effect of the presence of bias on the outcome of the cognitive computing function.
11 . The system of claim 10 , wherein:
the impartiality assessment engine receives a counterfactual; and,
the impartiality assessment operation contrasts features between an input subject data point and the counterfactual and identifies contrasts between the input data point and the counterfactual.
12 . The system of claim 7 , wherein:
the cognitive computing function is performed using an opaque model; and,
the burden score represents a bias of the opaque model for a particular group of individuals.
13 . A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving data from a data source;
processing the data from the data source to provide cognitively processed insights via an augmented intelligence system, the augmented intelligence system executing on a hardware processor of an information processing system, the augmented intelligence system providing a cognitive computing function;
performing an impartiality assessment operation via an impartiality assessment engine, the impartiality assessment operation detecting a presence of bias in an outcome of the cognitive computing function, the impartiality assessment operation generating a burden score representing the presence of bias in the outcome, the burden score representing the presence of bias in the outcome representing an impartiality assessment of the cognitive computing function; and,
providing the burden score representing the presence of bias in the outcome to a destination, the destination comprising a cognitive application, the cognitive application providing a user with the impartiality assessment of the cognitive computing function based upon the burden score representing the presence of bias in the outcome.
14 . The non-transitory, computer-readable storage medium of claim 13 , wherein:
the impartiality assessment engine is implemented within the augmented intelligence system, the augmented intelligence system comprising a cognitive process foundation.
15 . The non-transitory, computer-readable storage medium of claim 14 , wherein:
the cognitive process foundation performs an augmented intelligence assurance operation using the burden score.
16 . The non-transitory, computer-readable storage medium of claim 13 , wherein:
the impartiality assessment operation includes assessing an effect of the presence of bias on the outcome of the cognitive computing function.
17 . The non-transitory, computer-readable storage medium of claim 16 , wherein:
the impartiality assessment engine receives a counterfactual; and,
the impartiality assessment operation contrasts features between an input subject data point and the counterfactual and identifies contrasts between the input data point and the counterfactual.
18 . The non-transitory, computer-readable storage medium of claim 13 , wherein:
the cognitive computing function is performed using an opaque model; and,
the burden score represents a bias of the opaque model for a particular group of individuals.