Game-theoretic invariant rationalization of machine-learning results
To improve actual labels that are produced by a black box computer classifier system from inputs, identify, using an environment-aware predictor and an environment-agnostic predictor, a subset of the inputs. The subset of the inputs has a stable correlation with the actual labels across a plurality of environments. Identify the subset of the inputs as an explanatory rationale for the actual labels. Display the explanatory rationale with the actual labels to a consumer of the actual labels. Optionally, in response to the explanatory rationale failing a rubric established by the consumer, generate revised inputs by removing the explanatory rationale from the inputs; and produce revised labels by processing the revised inputs with the environment-agnostic predictor.
1. A method for improving actual labels that are produced by a black box computer classifier system from inputs, the method comprising:
identifying, using an environment-aware predictor and an environment-agnostic predictor, a subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system, wherein the subset of the inputs has a stable correlation with the actual labels across a plurality of environments;
identifying the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system as an explanatory rationale for the actual labels;
displaying the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system with the actual labels to a consumer of the actual labels; and
in response to the explanatory rationale failing a rubric established by the consumer:
generating revised inputs by removing the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system from the inputs; and
producing revised labels by processing the revised inputs with the environment- agnostic predictor.
2. A method for improving actual labels that are produced by a black box computer classifier system from inputs, the method comprising:
identifying, using an environment-aware predictor and an environment-agnostic predictor, a subset of the inputs, wherein the subset of the inputs has a stable correlation with the actual labels across a plurality of environments;
identifying the subset of the inputs as an explanatory rationale for the actual labels;
displaying the explanatory rationale with the actual labels to a consumer of the actual labels; and
in response to the explanatory rationale failing a rubric established by the consumer:
generating revised inputs by removing the explanatory rationale from the inputs; and
producing revised labels by processing the revised inputs with the environment-agnostic predictor, wherein identifying the subset of the inputs comprises applying game-theoretic invariant rationalization to maximize a performance of the environment-agnostic predictor while minimizing a difference in performance between the environment-agnostic predictor and the environment-aware predictor.
3. The method of claim 1 , further comprising:
dividing the inputs into first and second groups of data; and
training the environment-aware predictor and the environment-agnostic predictor by repeatedly:
generating environment-aware labels by the environment-aware predictor processing the first and second groups of data and identifiers of the first and second groups of data;
generating environment-agnostic labels by the environment-agnostic predictor processing the first and second groups of data without the identifiers of the first and second groups of data;
calculating information losses for the environment-aware predictor and for the environment-agnostic predictor by comparing the environment-aware labels and the environment-agnostic labels to the actual labels; and
adjusting weights of the environment-aware predictor and of the environment-agnostic predictor to achieve local minima of information losses.
4. The method of claim 1 , further comprising:
generating from the inputs a first putative rationale and a second putative rationale, wherein at least one putative rationale includes a portion of the inputs different from any portion included by the other putative rationale;
generating first environment-aware labels and first environment-agnostic labels by the environment-aware predictor and the environment-agnostic predictor respectively processing the first putative rationale;
generating second environment-aware labels and second environment-agnostic labels by the environment-aware predictor and the environment-agnostic predictor respectively processing the second putative rationale;
calculating a difference in first information losses between the environment-aware predictor and the environment-agnostic predictor for the first putative rationale, wherein the first information losses are obtained by comparison of the first environment-aware labels and environment-agnostic labels to the actual labels;
calculating a difference in second information losses between the environment-aware predictor and the environment-agnostic predictor for the second putative rationale, wherein the second information losses are obtained by comparison of the second environment-aware labels and environment-agnostic labels to the actual labels; and
selecting, as the explanatory rationale, the one of the first and second putative rationales that generates the smaller difference in information losses between the environment-aware predictor and the environment-agnostic predictor.
5. The method of claim 1 wherein the environment-agnostic predictor performs natural language processing (NLP) on a corpus of text that constitutes the inputs.
6. The method of claim 1 wherein the rubric includes a list of verbs.
7. The method of claim 1 wherein the rubric includes a list of nouns.
8. The method of claim 1 wherein the rubric includes a list of adjectives.
9. The method of claim 1 wherein the rubric includes a list of geographic coordinates.
10. A method for correcting implicit bias in human decision making, the method comprising:
obtaining actual labels that were applied by a human decisionmaker to inputs that include confounding data and an explanatory rationale, wherein the confounding data and the explanatory rationale are not initially distinguished from each other;
distinguishing the explanatory rationale from the confounding data, by identifying a stable correlation of the explanatory rationale with the actual labels across a plurality of environments using an environment-aware predictor and an environment-agnostic predictor;
informing a responsible party that the explanatory rationale matches a pertinent rubric; and
generating revised inputs by removing the explanatory rationale.
11. The method of claim 10 , further comprising:
training the environment-aware predictor and the environment-agnostic predictor by repeatedly:
generating environment-aware labels by the environment-aware predictor processing the first and second groups of data and identifiers of the first and second groups of data;
generating environment-agnostic labels by the environment-agnostic predictor processing the first and second groups of data without the identifiers of the first and second groups of data;
calculating information losses for the environment-aware predictor and for the environment-agnostic predictor by comparing the environment-aware labels and the environment-agnostic labels to the actual labels; and
adjusting weights of the environment-aware predictor and of the environment-agnostic predictor to achieve local minima of information losses.
12. The method of claim 11 , wherein identifying the stable correlation of the explanatory rationale with the actual labels comprises:
dividing the inputs into first and second groups of data;
producing a first putative rationale by selecting first matching sections of the first and second groups of data;
producing a second putative rationale by selecting second matching sections of the first and second groups of data, wherein the second matching sections differ from the first matching sections;
calculating a difference in information loss between the environment-aware predictor and the environment-agnostic predictor for the first putative rationale and for the second putative rationale; and
selecting as the explanatory rationale the one of the first and second putative rationales that generates the smaller difference in information loss between the environment-aware predictor and the environment-agnostic predictor.
13. The method of claim 10 wherein the machine learning algorithm performs natural language processing (NLP) on a corpus of text that constitutes the inputs.
14. The method of claim 10 , further comprising:
generating revised inputs by removing the explanatory rationale from the inputs; and
producing revised labels by processing the revised inputs with the environment-agnostic predictor.
15. A non-transitory computer readable medium embodying computer executable instructions which when executed by a computer cause the computer to facilitate a method for improving actual labels produced by a black box computer classifier system from inputs, the method comprising:
identifying, using an environment-aware predictor and an environment-agnostic predictor, a subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system, wherein the subset of the inputs has a stable correlation with the actual labels across a plurality of environments;
identifying the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system as an explanatory rationale for the actual labels;
displaying the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system with the actual labels to a consumer of the actual labels; and
in response to the explanatory rationale failing a rubric established by the consumer:
generating revised inputs by removing the subset of the inputs that suffice to yield the actual labels produced by the black box computer classifier system from the inputs; and
producing revised labels by processing the revised inputs with the environment-agnostic predictor.
16. A non-transitory computer readable medium embodying computer executable instructions which when executed by a computer cause the computer to facilitate a method for improving actual labels produced by a black box computer classifier system from inputs, the method comprising:
identifying, using an environment-aware predictor and an environment-agnostic predictor, a subset of the inputs, wherein the subset of the inputs has a stable correlation with the actual labels across a plurality of environments;
identifying the subset of the inputs as an explanatory rationale for the actual labels;
displaying the explanatory rationale with the actual labels to a consumer of the actual labels; and
in response to the explanatory rationale failing a rubric established by the consumer:
generating revised inputs by removing the explanatory rationale from the inputs; and
producing revised labels by processing the revised inputs with the environment-agnostic predictor, wherein identifying the subset of the inputs comprises applying game-theoretic invariant rationalization to maximize a performance of the environment-agnostic predictor while minimizing a difference in performance between the environment-agnostic predictor and the environment-aware predictor.
17. The computer readable medium of claim 15 , the method further comprising:
dividing the inputs into first and second groups of data; and
training the environment-aware predictor and the environment-agnostic predictor by repeatedly:
generating environment-aware labels by the environment-aware predictor processing the first and second groups of data and identifiers of the first and second groups of data;
generating environment-agnostic labels by the environment-agnostic predictor processing the first and second groups of data without the identifiers of the first and second groups of data;
calculating information losses for the environment-aware predictor and for the environment-agnostic predictor by comparing the environment-aware labels and the environment-agnostic labels to the actual labels; and
adjusting weights of the environment-aware predictor and of the environment-agnostic predictor to achieve local minima of information losses.
18. The computer readable medium of claim 15 , the method further comprising:
generating from the inputs a first putative rationale and a second putative rationale, wherein at least one putative rationale includes a portion of the inputs different from any portion included by the other putative rationale;
generating first environment-aware labels and first environment-agnostic labels by the environment-aware predictor and the environment-agnostic predictor respectively processing the first putative rationale;
generating second environment-aware labels and second environment-agnostic labels by the environment-aware predictor and the environment-agnostic predictor respectively processing the second putative rationale;
calculating a difference in first information losses between the environment-aware predictor and the environment-agnostic predictor for the first putative rationale, wherein the first information losses are obtained by comparison of the first environment-aware labels and environment-agnostic labels to the actual labels;
calculating a difference in second information losses between the environment-aware predictor and the environment-agnostic predictor for the second putative rationale, wherein the second information losses are obtained by comparison of the second environment-aware labels and environment-agnostic labels to the actual labels; and
selecting, as the explanatory rationale, the one of the first and second putative rationales that generates the smaller difference in information losses between the environment-aware predictor and the environment-agnostic predictor.
19. The computer readable medium of claim 18 , wherein the environment-agnostic predictor performs natural language processing (NLP) on a corpus of text that constitutes the inputs.
20. The computer readable medium of claim 18 , wherein the rubric includes at least one of a list of verbs, a list of nouns, a list of adjectives, and a list of geographic coordinates.