IP Library › Granted Patent US 12,469,075
Granted Patent B2
US 12,469,075 · App. 17/900,753 · Granted Nov 11, 2025

Computing system and method for creating a data science model having reduced bias

Inventors: Alexey Miroshnikov (Evanston, IL); Konstandinos Kotsiopoulos (Easthampton, MA); Arjun Ravi Kannan (Buffalo Grove, IL); Raghu Kulkarni (Buffalo Grove, IL); Steven Dickerson (Deerfield, IL); Ryan Franks (Deerfield, IL)
Assignee: Capital One Financial Corporation
G06Q40/03
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,469,075
App. No.
17/900,753
Granted
Nov 11, 2025
Kind
B2
Abstract

A computing platform may be configured to (i) train an initial model object for a data science model using a machine learning process, (ii) determine that the initial model object exhibits a threshold level of bias, and (iii) thereafter produce an updated version of the initial model object having mitigated bias by (a) identifying a subset of the initial model object's set of input variables that are to be replaced by transformations, (b) producing a post-processed model object by replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that has one or more unknown parameters, (c) producing a parameterized family of the post-processed model object, and (d) selecting, from the parameterized family of the post-processed model object, one given version of the post-processed model object to use as the updated version of the initial model object for the data science model.

Claims (83)

1 . A computing platform comprising:

at least one network interface for communicating over at least one data network;

at least one processor;

at least one non-transitory computer-readable medium; and

program instructions stored on the at least one non-transitory computer-readable medium that, when executed by the at least one processor, cause the computing platform to:

train an initial model object for a data science model using a machine learning process, wherein the initial model object functions to receive values for a set of input variables and generate an output value;

based on an evaluation of the initial model object's bias, determine that the initial model object exhibits a threshold level of bias with respect to at least one given attribute; and

after determining that the initial model object exhibits the threshold level of bias, produce an updated version of the initial model object having mitigated bias by:

based on an evaluation of the initial model object's set of input variables, identifying a subset of the initial model object's set of input variables that are to be replaced by transformations;

producing a post-processed model object by replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that has one or more unknown parameters, wherein the respective transformation of each respective input variable in the identified subset comprises one of (i) a first type of transformation that compresses or expands the respective input variable in a linear and symmetric manner, (ii) a second type of transformation that compresses or expands the respective input variable in a linear and asymmetric manner, (iii) a third type of transformation that compresses or expands the respective input variable in a non-linear and symmetric manner, or (iv) a fourth type of transformation that compresses or expands the respective input variable in a non-linear and asymmetric manner;

producing a parameterized family of the post-processed model object; and

selecting, from the parameterized family of the post-processed model object, one given version of the post-processed model object to use as the updated version of the initial model object for the data science model.

2 . The computing platform of claim 1 , wherein the threshold level of bias with respect to the at least one given attribute comprises a threshold level of bias with respect to a pair of subpopulations defined based on the given attribute that comprises a protected subpopulation and a non-protected subpopulation.

3 . The computing platform of claim 2 , wherein the evaluation of the initial model object's bias involves:

accessing a historical dataset comprising a first set of historical data records for individuals belonging the protected subpopulation and a second set of historical data records for individuals belonging the non-protected subpopulation;

inputting the first set of historical data records into the initial model object and thereby generating a first set of model scores for the protected subpopulation;

inputting the second set of historical data records into the initial model object and thereby generating a second set of model scores for the non-protected subpopulation; and

based on the first and second sets of model scores, quantifying the bias exhibited by the initial model object for the protected and non-protected subpopulations.

4 . The computing platform of claim 3 , wherein quantifying the bias exhibited by the initial model object for the protected and non-protected subpopulations comprises:

determining at least one of (i) a positive bias metric that quantifies a portion of the initial model object's bias that favors the non-protected subpopulation or (ii) a negative bias metric that quantifies a portion of the initial model object's bias that favors the protected subpopulation.

5 . The computing platform of claim 2 , wherein the evaluation of the initial model object's set of input variables involves:

based on an evaluation of dependencies between the initial model object's set of input variables, dividing the initial model object's set of input variables into a set of variable groups that each comprises one or more input variables; and

quantifying a respective bias contribution of each respective variable group in defined set of variable groups using an explanability technique and a historical dataset comprising a first set of historical data records for individuals belonging the protected subpopulation and a second set of historical data records for individuals belonging the non-protected subpopulation.

6 . The computing platform of claim 5 , wherein quantifying the respective bias contribution of each respective variable group comprises:

for each respective variable group, determining at least one of (i) a respective positive bias contribution metric that quantifies the respective variable group's contribution to either increasing a bias favoring the non-protected subpopulation or decreasing a bias favoring the protected subpopulation or (ii) a respective negative bias contribution metric that quantifies the respective variable group's contribution to either increasing a bias favoring the protected subpopulation or decreasing a bias favoring the non-protected subpopulation.

7 . The computing platform of claim 1 , wherein producing the post-processed model object by replacing each respective input variable in the identified subset with the respective transformation of the respective input variable comprises:

replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that is selected based on a determination of the respective input variable's contribution to the initial model object's bias.

8 . The computing platform of claim 1 , wherein producing the post-processed model object further comprises calibrating the post-processed model object to align a scale of post-processed model object's output with a scale of the initial model object's output.

9 . The computing platform of claim 1 , wherein producing the parameterized family of the post-processed model object comprises:

using a Bayesian optimization technique that functions to evaluate a bias and a performance of different versions of the post-processed model object that are produced by using different combinations of values for the unknown parameters included within the post-processed model object and thereby producing a parameterized family of the post-processed model object based on versions of the post-processed model object that form an efficient frontier for a tradeoff between the post-processed model object's bias and the post-processed model object's performance.

10 . The computing platform of claim 9 , wherein producing the parameterized family of the post-processed model object further comprises:

after producing the parameterized family of the post-processed model object using the Bayesian optimization technique, expanding the parameterized family of the post-processed model object to include additional versions of the post-processed model object.

11 . The computing platform of claim 10 , wherein expanding the parameterized family of the post-processed model object to include additional versions of the post-processed model object comprises:

constructing combined versions of the post-processed model object from respective pairs of versions of the post-processed model object that are in the parameterized family of the post-processed model object produced using the Bayesian optimization technique.

12 . The computing platform of claim 1 , wherein the data science model comprises a classification model, wherein the initial model object's output value comprises a score for use in making a classification decision, and wherein the computing platform further comprises program instructions stored on the at least one non-transitory computer-readable medium that, when executed by the at least one processor, cause the computing platform to:

define at least one score threshold that is to be utilized by the data science model for use in making classification decisions based on the score output by the updated version of the initial model object.

13 . The computing platform of claim 1 , wherein the identified subset of the initial model object's set of input variables that are to be replaced by transformations consists of input variables that behave in a numerical manner.

14 . The computing platform of claim 13 , wherein the identified subset of the initial model object's set of input variables comprises a first subset of the initial model object's set of input variables, and wherein producing the post-processed model object further comprises:

identifying a second subset of the initial model object's set of input variables for which to incorporate additional compensation into the post-processed model object, wherein the second subset of the initial model object's set of input variables consists of input variables that behave in a categorical manner; and

incorporating, into the post-processed model object, a respective compensation function for each respective input variable in the second subset of the initial model object's set of input variables that serves to compensate for the respective input variable's bias contribution.

15 . A non-transitory computer-readable medium comprising program instructions that, when executed by at least one processor, cause a computing platform to:

train an initial model object for a data science model using a machine learning process, wherein the initial model object functions to receive values for a set of input variables and generate an output value;

based on an evaluation of the initial model object's bias, determine that the initial model object exhibits a threshold level of bias with respect to at least one given attribute; and

after determining that the initial model object exhibits the threshold level of bias, produce an updated version of the initial model object having mitigated bias by:

based on an evaluation of the initial model object's set of input variables, identifying a subset of the initial model object's set of input variables that are to be replaced by transformations;

producing a post-processed model object by replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that has one or more unknown parameters, wherein the respective transformation of each respective input variable in the identified subset comprises one of (i) a first type of transformation that compresses or expands the respective input variable in a linear and symmetric manner, (ii) a second type of transformation that compresses or expands the respective input variable in a linear and asymmetric manner, (iii) a third type of transformation that compresses or expands the respective input variable in a non-linear and symmetric manner, or (iv) a fourth type of transformation that compresses or expands the respective input variable in a non-linear and asymmetric manner;

producing a parameterized family of the post-processed model object; and

selecting, from the parameterized family of the post-processed model object, one given version of the post-processed model object to use as the updated version of the initial model object for the data science model.

16 . The non-transitory computer-readable medium of claim 15 , wherein the threshold level of bias with respect to the at least one given attribute comprises a threshold level of bias with respect to a pair of subpopulations defined based on the given attribute that comprises a protected subpopulation and a non-protected subpopulation.

17 . The non-transitory computer-readable medium of claim 16 , wherein the evaluation of the initial model object's bias involves:

accessing a historical dataset comprising a first set of historical data records for individuals belonging the protected subpopulation and a second set of historical data records for individuals belonging the non-protected subpopulation;

inputting the first set of historical data records into the initial model object and thereby generating a first set of model scores for the protected subpopulation;

inputting the second set of historical data records into the initial model object and thereby generating a second set of model scores for the non-protected subpopulation; and

based on the first and second sets of model scores, quantifying the bias exhibited by the initial model object for the protected and non-protected subpopulations.

18 . The non-transitory computer-readable medium of claim 17 , wherein quantifying the bias exhibited by the initial model object for the protected and non-protected subpopulations comprises:

determining at least one of (i) a positive bias metric that quantifies a portion of the initial model object's bias that favors the non-protected subpopulation or (ii) a negative bias metric that quantifies a portion of the initial model object's bias that favors the protected subpopulation.

19 . The non-transitory computer-readable medium of claim 16 , wherein the evaluation of the initial model object's set of input variables involves:

based on an evaluation of dependencies between the initial model object's set of input variables, dividing the initial model object's set of input variables into a set of variable groups that each comprises one or more input variables; and

quantifying a respective bias contribution of each respective variable group in defined set of variable groups using an explanability technique and a historical dataset comprising a first set of historical data records for individuals belonging the protected subpopulation and a second set of historical data records for individuals belonging the non-protected subpopulation.

20 . The non-transitory computer-readable medium of claim 19 , wherein quantifying the respective bias contribution of each respective variable group comprises:

for each respective variable group, determining at least one of (i) a respective positive bias contribution metric that quantifies the respective variable group's contribution to either increasing a bias favoring the non-protected subpopulation or decreasing a bias favoring the protected subpopulation or (ii) a respective negative bias contribution metric that quantifies the respective variable group's contribution to either increasing a bias favoring the protected subpopulation or decreasing a bias favoring the non-protected subpopulation.

21 . The non-transitory computer-readable medium of claim 15 , wherein producing the post-processed model object by replacing each respective input variable in the identified subset with the respective transformation of the respective input variable comprises:

replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that is selected based on a determination of the respective input variable's contribution to the initial model object's bias.

22 . The non-transitory computer-readable medium of claim 15 , wherein producing the post-processed model object further comprises calibrating the post-processed model object to align a scale of post-processed model object's output with a scale of the initial model object's output.

23 . The non-transitory computer-readable medium of claim 15 , wherein producing the parameterized family of the post-processed model object comprises:

using a Bayesian optimization technique that functions to evaluate a bias and a performance of different versions of the post-processed model object that are produced by using different combinations of values for the unknown parameters included within the post-processed model object and thereby producing a parameterized family of the post-processed model object based on versions of the post-processed model object that form an efficient frontier for a tradeoff between the post-processed model object's bias and the post-processed model object's performance.

24 . The non-transitory computer-readable medium of claim 23 , wherein producing the parameterized family of the post-processed model object further comprises:

after producing the parameterized family of the post-processed model object using the Bayesian optimization technique, expanding the parameterized family of the post-processed model object to include additional versions of the post-processed model object.

25 . The non-transitory computer-readable medium of claim 24 , wherein expanding the parameterized family of the post-processed model object to include additional versions of the post-processed model object comprises:

constructing combined versions of the post-processed model object from respective pairs of versions of the post-processed model object that are in the parameterized family of the post-processed model object produced using the Bayesian optimization technique.

26 . The non-transitory computer-readable medium of claim 15 , wherein the data science model comprises a classification model, wherein the initial model object's output value comprises a score for use in making a classification decision, and wherein the non-transitory computer-readable medium further comprises program instructions that, when executed by at least one processor, cause the computing platform to:

define at least one score threshold that is to be utilized by the data science model for use in making classification decisions based on the score output by the updated version of the initial model object.

27 . The non-transitory computer-readable medium of claim 15 , wherein the identified subset of the initial model object's set of input variables that are to be replaced by transformations comprises a first subset of the initial model object's set of input variables consisting of input variables that behave in a numerical manner, and wherein producing the post-processed model object further comprises:

identifying a second subset of the initial model object's set of input variables for which to incorporate additional compensation into the post-processed model object, wherein the second subset of the initial model object's set of input variables consists of input variables that behave in a categorical manner; and

incorporating, into the post-processed model object, a respective compensation function for each respective input variable in the second subset of the initial model object's set of input variables that serves to compensate for the respective input variable's bias contribution.

28 . A computer-implemented method comprising:

training an initial model object for a data science model using a machine learning process, wherein the initial model object functions to receive values for a set of input variables and generate an output value;

based on an evaluation of the initial model object's bias, determining that the initial model object exhibits a threshold level of bias with respect to at least one given attribute; and

after determining that the initial model object exhibits the threshold level of bias, producing an updated version of the initial model object having mitigated bias by:

based on an evaluation of the initial model object's set of input variables, identifying a subset of the initial model object's set of input variables that are to be replaced by transformations;

producing a post-processed model object by replacing each respective input variable in the identified subset with a respective transformation of the respective input variable that has one or more unknown parameters, wherein the respective transformation of each respective input variable in the identified subset comprises one of (i) a first type of transformation that compresses or expands the respective input variable in a linear and symmetric manner, (ii) a second type of transformation that compresses or expands the respective input variable in a linear and asymmetric manner, (iii) a third type of transformation that compresses or expands the respective input variable in a non-linear and symmetric manner, or (iv) a fourth type of transformation that compresses or expands the respective input variable in a non-linear and asymmetric manner;

producing a parameterized family of the post-processed model object; and

selecting, from the parameterized family of the post-processed model object, one given version of the post-processed model object to use as the updated version of the initial model object for the data science model.

Assignments (2)
MERGER Recorded Jul 2, 2025
From: DISCOVER FINANCIAL SERVICES
To: CAPITAL ONE FINANCIAL CORPORATION
Reel/Frame 071784/0903 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2022
From: MIROSHNIKOV, ALEXEY; KOTSIOPOULOS, KONSTANDINOS; KANNAN, ARJUN RAVI; KULKARNI, RAGHU; DICKERSON, STEVEN; FRANKS, RYAN
To: DISCOVER FINANCIAL SERVICES
Reel/Frame 061274/0319 →
Continuity (2)
Continuation In Part 16891989 · Jun 3, 2020
Related Publication 20220414766A1 · Dec 29, 2022
References Cited (180)
US 8370253B1 · Grossman · 2013 [cited by applicant]
US 11030583B1 · Garg et al. · 2021 [cited by applicant]
US 11636386B2 · Lohia · 2023 [cited by examiner]
US 12020133B2 · Goldszmidt · 2024 [cited by examiner]
US 12277455B2 · Walters · 2025 [cited by examiner]
US 20110270782A1 · Trenner et al. · 2011 [cited by applicant]
US 20140229415A1 · Martineau et al. · 2014 [cited by applicant]
US 20180285685A1 · Singh et al. · 2018 [cited by applicant]
US 20190087744A1 · Schiemenz · 2019 [cited by applicant]
US 20190188605A1 · Zavesky et al. · 2019 [cited by applicant]
US 20190279111A1 · Merrill et al. · 2019 [cited by applicant]
US 20190378210A1 · Merrill et al. · 2019 [cited by applicant]
US 20200081865A1 · Farrar et al. · 2020 [cited by applicant]
US 20200082299A1 · Affonso et al. · 2020 [cited by applicant]
US 20200184350A1 · Bhide et al. · 2020 [cited by applicant]
US 20200218987A1 · Jiang et al. · 2020 [cited by applicant]
US 20200302309A1 · Golding · 2020 [cited by applicant]
US 20200302524A1 · Kamkar et al. · 2020 [cited by applicant]
US 20200311486A1 · Dey et al. · 2020 [cited by applicant]
US 20200320428A1 · Chaloulos et al. · 2020 [cited by applicant]
US 20200372035A1 · Tristan et al. · 2020 [cited by applicant]
US 20200372304A1 · Kenthapadi et al. · 2020 [cited by applicant]
US 20200372406A1 · Wick · 2020 [cited by applicant]
US 20200380398A1 · Weirder et al. · 2020 [cited by applicant]
US 20200410129A1 · Nadler et al. · 2020 [cited by applicant]
US 20210081828A1 · Lopez De Prado · 2021 [cited by applicant]
US 20210133870A1 · Kamkar et al. · 2021 [cited by applicant]
US 20210158102A1 · Lohia · 2021 [cited by examiner]
US 20210173402A1 · Chang · 2021 [cited by examiner]
US 20210174222A1 · Dodwell et al. · 2021 [cited by applicant]
US 20210224605A1 · Zhang et al. · 2021 [cited by applicant]
US 20210224687A1 · Goldszmidt et al. · 2021 [cited by applicant]
US 20210241033A1 · Yang · 2021 [cited by applicant]
US 20210248503A1 · Hickety et al. · 2021 [cited by applicant]
US 20210256832A1 · Weisz et al. · 2021 [cited by applicant]
US 20210304039A1 · Tang et al. · 2021 [cited by applicant]
US 20210334654A1 · Himanshi et al. · 2021 [cited by applicant]
US 20210350272A1 · Miroshnikov et al. · 2021 [cited by applicant]
US 20210383268A1 · Miroshnikov et al. · 2021 [cited by applicant]
US 20210383275A1 · Miroshnikov et al. · 2021 [cited by applicant]
US 20220036203A1 · Nachum et al. · 2022 [cited by applicant]
US 20240095605A1 · Fylstra et al. · 2024 [cited by applicant]
WO 2021074491A1 · 2021 [cited by applicant]
Blum et al. Recovering from Biased Data: Can Fairness Constraints Improve Accuracy? Toyota Technological Institute at Chicago arXiv:1912.01094v1 [cs.LG] Dec. 4, 2019, 20 pages. [cited by applicant]
Varley et al. Fairness in Machine Learning with Tractable Models. School of Informatics, University of Edinburgh, UK arXiv:1905.07026v2 [cs.LG] Jan. 13, 2020, 26 pages. [cited by applicant]
Hickey et al. Fairness by Explicability and Adversarial SHAP Learning. Experian UK&I and EMEA DataLabs, London, UK arXiv:2003.05330v3 [cs.LG] Jun. 26, 2020, 17 pages. [cited by applicant]
Manu Joseph. Interpretability Cracking open the black box—Part 11. Machine Learning. Deep & Shallow, Nov. 16, 2019-Nov. 28, 2019, 23 pages. [cited by applicant]
Bellamy et al. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias, IBM J. Res. & Dev. vol. 63, No. 4/5, Paper 4. Jul./Sep. 2019. 15 pages. [cited by applicant]
Lohia et al. Bias Mitigation Post-Processing for Individual and Group Fairness. IBM Research and IBM Watson AI Platform. 978-1-5386-4658-8/18/$31.00. 2019. IEEE. 5 pages. [cited by applicant]
International Search Report and Written Opinion issued in International Application No. PCT/US2023/031454, mailed on Dec. 12, 2023, 7 pages. [cited by applicant]
Casalicchio, Giuseppe. Visualizing the Feature Importance for Black Box Models. Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer Science. vol. 11051. Springer. pp. 655-670… [cited by applicant]
Aas et al. Explaining Individual Predictions When Features are Dependent: More Accurate Approximations to Shapley Values. arXiv preprint arXiv:1903.10464v3, Feb. 6, 2020, 28 pages. [cited by applicant]
Balashankar et al. Pareto-Efficient Fairness for Skewed Subgroup Data. In the International Conference on Machine learning AI for Social Good Workshop. vol. 8. Long Beach, United States, 2019, 8 pages. [cited by applicant]
Banzhaf III. Weighted Voting Doesn't Work: A Mathematical Analysis. Rutgers Law Review, vol. 19, No. 2, 1965, 28 pages. [cited by applicant]
Barocas et al. Fairness and Machine Learning: Limitations and Opportunities. [online], 253 pages, 1999 [retrieved online on May 26, 2022]. Retrieved from the Internet:<URL:https://fairmlbook.org/>. [cited by applicant]
Bergstra et al. Algorithms for Hyper-Parameter Optimization. NIPS 11: Proceedings of the 24th International conference on Neural Information Processing Systems. Dec. 2011, 9 pages. [cited by applicant]
Bousquet et al. Stability and Generalization. Journal of Machine Learning Research 2. Mar. 2002, pp. 499-526. [cited by applicant]
Chen et al. True to Model or True to the Data? arXiv preprint arXiv:2006.1623v1, Jun. 29, 2020, 7 pages. [cited by applicant]
Cover et al. Elements of Information Theory. A John Wiley & Sons, Inc., Publication. Second Edition, 2006, 774 pages. [cited by applicant]
Del Barrio et al. Obtaining Fairness Using Optimal Transport Theory. arXiv preprint arXiv:1806.03195v2. Jul. 19, 2018, 25 pages. [cited by applicant]
Del Barrio et al. On Approximate Validation of Models: A Kolmogorov-Smirnov Based Approach. arXiv:1903.08687v1, Mar. 20, 2019, 32 pages. [cited by applicant]
Dickerson et al. Machine Learning. Considerations for fairly and transparently expanding access to credit. 2020, 29 pages [online]. Retrieved online: URL<https://info.h2o.ai/rs/644-PKX-778/images/Machine%20Learning%20-%… [cited by applicant]
Dwork et al. Generalization in Adaptive Data Analysis and Holdout Reuse. NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, vol. 2, Dec. 2015, 9 pages. [cited by applicant]
Dwork et al. Fairness Through Awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 2012, 214-226. [cited by applicant]
Equal Credit Opportunity Act (ECOA). FDIC Law, Regulations, Related Actions, 9 pages [online]. Retrieved online: URL< https://www.fdic.gov/regulations/laws/rules/6000-1200.html>. [cited by applicant]
Equal Employment Opportunity Act. 2 pages [online]. Retrieved online: URL<https://www.dol.gov/sites/dolgov/files/ofccp/regs/compliance/posters/pdf/eeopost.pdf>. [cited by applicant]
Fair Housing Act (FHA). FDIC Law, Regulations, Related Actions, 2022, 3 pages [online]. Retrieved online: URL<https://www.ecfr.gov/current/title-12/chapter-III/subchapter-B/part-338. [cited by applicant]
Feldman et al. Certifying and Removing Disparate Impact. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, 259-268. [cited by applicant]
Friedman, Jerome. 1999 Reitz Lecture. Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, vol. 29, No. 6. 2001, pp. 1189-1232. [cited by applicant]
G. Owen. Modification of the Banzhaf-Coleman index for Games with a Priori Unions. Power, Voting and Voting Power. 1981, pp. 232-238. [cited by applicant]
Goldstein et al. Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation. arXiv:1309.6392v2J. Mar. 20, 2014, 22 pages. [cited by applicant]
Hall et al. A United States Fair Lending Perspective on Machine Learning. Frontiers in Artificial Intelligence. doi: 10.3389/frai.2021.695301, Jun. 7, 2021, 9 pages. [cited by applicant]
Hall, Patrick. On the Art and Science of Explainable Machine Learning: Techniques, Recommendations, and Responsibilities. KDD'19 XAI Workshop. arXiv preprint arxiv:1810.02909, Aug. 2, 2019, 10 pages. [cited by applicant]
Hardt et al. Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems, 2015, 3315-3323. [cited by applicant]
Hashimoto et al. Fairness Without Demographics in Repeated Loss Minimization. In ICML, 2018, 10 pages. [cited by applicant]
Hastie et al. The Elements of Statistical Learning: Data Mining, Interference, and Prediction. Springer. Second edition, Jan. 13, 2017, 764 pages. [cited by applicant]
Heller et al. A consistent multivariate test of association based on ranks of distances. Biometrika. arXiv:1201.3522v1. Jan. 17, 2012, 14 pages. [cited by applicant]
Heller et al. Consistent Distribution-Free K-Sample and Independence Tests for Univariate Random Variables. Journal of Machine Learning Research. vol. 17, No. 29. Feb. 2016, 54 pages. [cited by applicant]
Janzing et al. Feature relevance quantification in explainable AI:A causal problem. arXiv preprint arXiv:1910.13413v2. Nov. 25, 2019, 11 pages. [cited by applicant]
Jiang et al. Identifying and Correcting Label Bias in Machine Learning. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS). 2020, 10 pages. [cited by applicant]
Jiang et al. Smooth Isotonic Regression: A New Method to Calibrate Predictive Models. AMIA Joint Summits on Translational Science proceedings. 2011, pp. 16-20. [cited by applicant]
Kamiran et al. Classifying without Discriminating. 2009 2nd International Conference on Computer, Control and Communication. doi: 10.1109/IC4.2009.4909197. 2009, pp. 1-6. [cited by applicant]
Kamiran et al. Data Preprocessing Techniques for Classification Without Discrimination. Knowl. Inf. Syst. DOI 10.1007/s10115-011-0463-8. Dec. 3, 2011, 33 pages. [cited by applicant]
Kamiran et al. Discrimination Aware Decision Tree Learning. 2010 IEEE International Conference on Data Mining. doi: 10.1109/ICDM.2010.50. 2010, pp. 869-874. [cited by applicant]
Kamishima et al. Fairness-Aware Classifier with Prejudice Remover Regularizer. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), Part … [cited by applicant]
Karush, William. Minima of Functions of Several Variables with Inequalities as Side Constraints. A Dissertation Submitted to the Faculty of the Division of the Physical Sciences in Candidacy for the Degree of Master in … [cited by applicant]
Kearns et al. Algorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation. Neural Computation, vol. 11, No. 6. 1999, pp. 1427-1453. [cited by applicant]
Koralov et al. Theory of Probability and Random Processes. Second Edition. Springer. 1998, 349 pages. [cited by applicant]
Kovalev et al. A Robust Algorithm for Explaining Unreliable Machine Learning Survival Models Using the Kolmogorov-Smirnov Bounds. Neural Networks. arXiv:2005.02249v1, May 5, 2020. pp. 1-39. [cited by applicant]
Kuhn et al. Nonlinear programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, Calif. University of California Press. 90. 1951, pp. 481-492. [cited by applicant]
Lahoti et al. Fairness without Demographics through Adversarially Reweighted Learning. arXiv preprint arXiv:2006.13114. 2020, 13 pages. [cited by applicant]
Lipovetsky et al. Analysis of Regression in Game Theory Approach. Applied Stochastic Models Business and Industry. vol. 17. Apr. 26, 2001, pp. 319-330. [cited by applicant]
Lorenzo-Freire, Silvia. New characterizations of the Owen and Banzhaf-Owen values using the intracoalitional balanced contributions property. Department of Mathematics. 2017, 23 pages. [cited by applicant]
Lundberg et al. Consistent Individualized Feature Attribution for Tree Ensembles. arXiv preprint arxiv:1802.03888v3. Mar. 7, 2019, 9 pages. [cited by applicant]
Lundberg et al. A Unified Approach to Interpreting Model Predictions. 31st Conference on Neural Information Processing Systems, 2017, 10 pages. [cited by applicant]
Markowitz, Harry. Portfolio Selection. The Journal of Finance. vol. 7, No. 1. 1952, pp. 77-91. [cited by applicant]
Miroshinikov et al. Model-Agnostic Bias Mitigation Methods with Regressor Distribution Control for Wasserstein-Based Fairness Metrics. arXiv:2111.11259v1. Nov. 19, 2021, 29 pages. [cited by applicant]
Miroshinikov et al. Mutual Information-Based Group Explainers with Coalition Structure for Machine Learning Model Explanations. Computer Science and Game Theory. arXiv:2102.10878v1. Feb. 22, 2021, 46 pages. [cited by applicant]
Miroshinikov et al. Wasserstein-Based Fairness Interpretability Framework for Machine Learning Models. arXiv:2011.03156v1. Nov. 6, 2020, 34 pages. [cited by applicant]
Owen, Guillermo. Values of Games with a Priori Unions. In: Essays in Mathematical Economics and Game Theory. Springer, 1977, pp. 76-88. [cited by applicant]
Pareto. Optimality. In: Multiobjective Linear Programming. Springer International Publishing. DOI 10.1007/978-3-319-21091-9_4. 2016, pp. 85-118. [cited by applicant]
Wehle et al., “Machine Learning, Deep Learning, and AI: What's the Difference?” Data Scientist Innovation Day (2017). [cited by applicant]
Wuest et al., “Machine Learning in Manufacturing: Advantages, Challenges and Applications,” Production and Manufacturing Research, vol. 4, 23-45 (2016). [cited by applicant]
Zeng et al. Jackknife approach to the estimation of mutual information. PNAS, vol. 115, No. 40. Oct. 18, 2018. 6 pages. [cited by applicant]
Zhao et al. Causal Interpretations of Black-Box Models. J Bus., Econ. Stat., DOI:10.1080/07350015.2019.1624293, 2019, 16 pages. [cited by applicant]
International Search Report and Written Opinion issued in International Application No. PCT/US2022/029670, mailed on Sep. 7, 2022, 10 pages. [cited by applicant]
Kwak, Nojun et al. Input Feature Selection for Classification Problems. IEEE Transactions on Neural Networks. vol. 13, No. 1. pp. 143-159. 2002. 17 pages. [cited by applicant]
Lundberg, Scott M., et al. Consistent Individualized Feature Attribution for Tree Ensembles. arXiv preprint arxiv:1802.03888v2. Jun. 18, 2018. 9 pages. [cited by applicant]
Perrone et al. Fair Bayesian Optimization. AIES '21, May 19-21, 2021, Virtual Event, USA. pp. 854-863. [cited by applicant]
Rawls, John. Justice as Fairness: A Restatement. Harvard University Press. 2001, 32 pages. [cited by applicant]
Reshef et al. An Empirical Study of Leading Measures of Dependence. arXiv preprint arXiv:1505.02214. May 12, 2015, 42 pages. [cited by applicant]
Reshef et al. Detecting Novel Associations in Large Data Sets. Science, vol. 334. Dec. 2011, pp. 1518-1524. [cited by applicant]
Reshef et al. Measuring Dependence Powerfully and Equitably. Journal of Machine Learning Research, vol. 17. 2016, 63 pages. [cited by applicant]
Ribeiro et al. “Why Should I Trust You?” Explaining the Predictions of Any Classifier. 22nd Conference on Knowledge Discovery and Data Mining, San Francisco, CA. arXiv:1602.04938v3. Aug. 9, 2016, 10 pages. [cited by applicant]
Royden et al. Real Analysis. Boston: Prentice Hall, 4th Edition. 2010, 516 pages. [cited by applicant]
Santambrogio, Filippo. Optimal Transport for Applied Mathematicians. Calculus of Variations, PDEs and Modeling. Springer. May 2015, 356 pages. [cited by applicant]
Schmidt et al. An Introduction to Artificial Intelligence and Solutions to the Problems of Algorithmic Discrimination. arXiv preprint arXiv:1911.05755. vol. 73, No. 2. 2019, pp. 130-144. [cited by applicant]
Shapley, L.S. A Value for n-Person Games. Annals of Mathematics Studies. 1953, 13 pages. [cited by applicant]
Shiryaev, A.N. Probability. Springer, Second Edition. 1980, 22 pages. [cited by applicant]
Shorack et al. Empirical Processes with Applications to Statistics. Wiley, New York. 1986, 36 pages. [cited by applicant]
Strumbelji et al. Explaining Prediction Models and Individual Predictions with Feature Contributions. Springer. Knowledge Information System, vol. 41, No. 3. 2014, pp. 647-665. [cited by applicant]
Sundararajan et al. The Many Shapley Values for Model Explanation. arXiv preprint arXiv:1908.08474v1, Aug. 22, 2019, 9 pages. [cited by applicant]
Szekely et al. Measuring and Testing Dependence by Correlation of Distances. The Annals of Statistics, vol. 35, No. 6. ArXiv:0803.4101v1, Mar. 28, 2008, 26 pages. [cited by applicant]
Thorpe, Matthew, Introduction to Optimal Transport, University of Cambridge, 2018, 56 pages. [cited by applicant]
Villani, Cedric. Optimal Transport, Old and New. Springer. Jun. 13, 2008, 998 pages. [cited by applicant]
Wasserman et al. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. 2004, 14 pages. [cited by applicant]
Woodworth et al. Learning Non-Discriminatory Predictors. Proceedings of Machine Learning Research, vol. 65, No. 1-34. 2017, 34 pages. [cited by applicant]
Young. Monotonic Solutions of Cooperative Games International Journal of Game Theory, vol. 14, Issue 2. 1985. 8 pages. [cited by applicant]
Zemel et al. Learning Fair Representations. Proceedings of the 30th International Conference on Machine Learning, PMLR vol. 28, No. 3. 2013, pp. 325-333. [cited by applicant]
Zhang et al. Mitigating Unwanted Biases with Adversarial Learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Feb. 2-3, 2018, pp. 335-340. [cited by applicant]
Manu Joseph. Interpretability Cracking open the black box—Part 111. Machine Learning. Deep & Shallow, Nov. 24, 2019-Dec. 30, 2020, 29 pages. [cited by applicant]
Krivoruchko et al. Using Multivariate Interpolation for Estimating Well Performance, Esri.com, Arcuser. Esri Technology Summer 2014, 13 pages. URL:<URL: https://www.esri.com/abouUnewsroom/arcuser/using-multivariate-inte… [cited by applicant]
Chen et al. Comparison and Improvement of the predictability and interpretability with ensemble learning models in QSPR applications. Journal of Cheminformatics, Mar. 30, 2020, 50 pages. <URL:https://jcheminf.biomedcent… [cited by applicant]
Ding et al. Comparisons of Two Ensemble Mean Methods in Measuring the Average Error Growth and the Predictability. The Chinese Meteorological Society and Springer-Verlag Berlin Heidelberg, No. 4, vol. 25, Mar. 21, 2011,… [cited by applicant]
Hu et al. Locally Interpretable Models and Effects based on Supervised Partitioning, Wells Fargo, Jun. 1, 2018, 15 pages <URL: https://arxiv.org/pdf/1806.00663>. [cited by applicant]
Jiang et al. Wasserstein Fair Classification. Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, arXiv:1907.12059v1 [stat.ML] Jul. 28, 2019, 15 pages. <URL: chrome-extension://efaidnbm… [cited by applicant]
Wei et al. Optimized Score Transformation for Consistent Fair Classication, arXiv:1906.00066v3 [cs.LG] Oct. 29, 2021, 78 pages <URL: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://arxiv.org/pdf/1906.00066.p… [cited by applicant]
Tosifidis et al. FAE: A Fairness-Aware Ensemble Framework, IEEE, arXiv:2002.00695v1 [cs.AI] Feb. 3, 2020, 6 pages. <URL:https://arxiv.org/abs/2002.00695>. [cited by applicant]
Abduljabbar et al., “Applications of Artificial Intelligence in Transport: An Overview,” Sustainability (2019). [cited by applicant]
Albizuri et al. On Coalitional Semivalues. Games and Economic Behavior 49, Nov. 29, 2002, 37 pages. [cited by applicant]
Alonso-Meijide et al. Modification of the Banzhaf Value for Games with a Coalition Structure. Annals of Operations Research, Jan. 2002, 17 pages. [cited by applicant]
Amer et al. The modified Banzhaf value for games with coalition structure: an axiomatic characterization. Mathematical Social Sciences 43, (2002) 45-54, 10 pages. [cited by applicant]
Apley et al., “Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models,” Ar Xiv (2016). [cited by applicant]
Aumann et al. Cooperative Games with Coalition Structure. International journal of Game Theory, vol. 3, Issue 4, Jul. 1974, pp. 217-237. [cited by applicant]
Aurenhammer et al., Voronoi Diagrams And Delaunay Triangulations, pp. 7-13, 41-45, New Jersey, World Scientific Publishing (2013). [cited by applicant]
Breiman et al. Estimating Optimal Transformations for Multiple Regression and Correlation. Journal of the American Statistical Association. vol. 80, No. 391, Sep. 1985, 19 pages. [cited by applicant]
Bzdok et al., “Statistics Versus Machine Learning,” Nature Methods, vol. 15, 233-234 (Apr. 2018). [cited by applicant]
Casas-Mendez et al. An extension of the tau-value to games with coalition structures. European Journal of Operational Research, vol. 148, 2003, pp. 494-513. [cited by applicant]
Dubey et al. Value Theory Without Efficiency. Mathematics of Operations Research. vol. 6, No. 1. 1981, pp. 122-128. [cited by applicant]
Eggleston, H.G., Convexity, pp. 33-38, Cambridge University Press (1958). [cited by applicant]
Elshawi et al. On the interpretability of machine learning-based model for predicting hypertension. BMC Medical Informatics and Decision Making. vol. 19, No. 146. 2019, 32 pages. [cited by applicant]
Explainable Artificial Intelligence (xAI), Defense Advanced Research Projects Agency, DARPA-BAA-16-53 (2016). [cited by applicant]
FDIC Title VII-Equal Credit Opportunity, FDIC Law, Regulations, Related Acts—Consumer Protection (Dec. 31, 2019). [cited by applicant]
Gretton et al. A Kernel Statistical Test of Independence. In Advances in Neural Information Processing Systems. 2007, 8 pages. [cited by applicant]
Gretton et al. A Kernel Two-Sample Test. The Journal of Machine Learning Research. vol. 13, No. 1. Mar. 2012, pp. 723-773. [cited by applicant]
Gretton et al. Measuring Statistical Dependence with Hilbert-Schmidt Norms. In Algorithmic Learning Theory. Springer-Verlag Berlin Heidelberg. 2005, pp. 63-77. [cited by applicant]
Hall et al. An Introduction to Machine Learning Interpretability, O'Reilly, Second Edition, Aug. 2019, 62 pages. [cited by applicant]
Hoeffding, Wassily. A Non-Parametric Test of Independence. The Annals of Mathematical Statistics. 1948, pp. 546-557. [cited by applicant]
Ji et al. Post-Radiotherapy PET Image Outcome Prediction by Deep Learning Under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application. Duke University Medical Center. arXiv preprint. 2021, 2… [cited by applicant]
Jiang et al. Nonparametric K-Sample Tests via Dynamic Slicing. Journal of the American Statistical Association. Harvard Library. Sep. 11, 2015. pp. 642-653. [cited by applicant]
Jollife et al. Principal Component Analysis. Springer Series In Statistics. Second Edition, 2002, 518 pages. [cited by applicant]
Kamijo, Yoshio. A two-step Shapley value in a cooperative game with a coalition structure. International Game Theory Review. 2009, 11 pages. [cited by applicant]
Kraskov et al. Estimating mutual information. The American Physical Society. Physical Review E vol. 69. Jun. 23, 2004, 16 pages. [cited by applicant]
Leo Breiman. Statistical Modeling: The Two Cultures. Statistical Science, vol. 16, No. 3, 2001, pp. 199-231. [cited by applicant]
Leo et al., “Machine Learning in Banking Risk Management: A Literature Review,” Risks (2019). [cited by applicant]
Lopez-Paz et al. The Randomized Dependence Coefficient. In Advances in Neural Information Processing Systems. 2013, pp. 1-9. [cited by applicant]
Lundberg, Scott. Welcome to SHAP Documentation. 2018, 46 pages [online]. Retrieved online URL:< https://shap.readthedocs.io/en/latest/>. [cited by applicant]
Miroshinikov et al. Wasserstein-based faireness interpretability framework for machine learning models. arXiv:2011.03156v1. Nov. 6, 2020, 34 pages. [cited by applicant]
Modarres et al. Towards Explainable Deep Learning for Credit Lending: A Case Study. arxiv:1811.06471v2, Nov. 11, 2018, 8 pages. [cited by applicant]
Paninski, Liam. Estimation of Entropy and Mutual Information. Neural Computation, vol. 15, No. 6. 2003, pp. 1191-1253. [cited by applicant]
Renyi. On Measures of Dependence. Acta mathematica hungarica, vol. 10, No. 3. 1959, pp. 441-451. [cited by applicant]
SAS/STAT User's Guide, Version 8, Chapter 68 “The Varclus Procedure” (1999). [cited by applicant]
Shiryaev, A.N., Graduate Texts in Mathematics: Probability. 2nd ed., pp. 23-43, Springer (1980). [cited by applicant]
Szekely et al. Brownian Distance Covariance. The Annals of Applied Statistics, vol. 3, No. 4. 2009. 31 pages. [cited by applicant]
Tibshirani et al. Estimating the No. of clusters in a dataset via the gap statistic. Journal of the Royal Statistical Society, Series B. vol. 63, Part 2. 2001. pp. 411-423. [cited by applicant]
Tijs. Bounds for the Core and The Tau-Value. Game Theory and Mathematical Economics. 1981, pp. 123-132. [cited by applicant]
Vanderbei, Robert J. Linear Programming: Foundations and Extensions, 4th ed., pp. 11-16, 141-144, New York, NY, Springer Science and Business Media (2014). [cited by applicant]
Vaughan et al., “Explainable Neural Networks based on Additive Index Models,” Corporate Model Risk, Wells Fargo, USA (Jun. 2018). [cited by applicant]
Vidal-Puga, Juan. The Harsanyi paradox and the “right to talk” in bargaining among coalitions. Mathematical Social Sciences. vol. 64. Mar. 27, 2012. 32 pages. [cited by applicant]
Wang et al. Shapley Flow: A Graph-based Approach to Interpreting Model Predictions arXiv preprint arXiv:2010.14592, 2020, 11 pages. [cited by applicant]