IP Library › Granted Patent US 12,475,132
Granted Patent B2
US 12,475,132 · App. 18/111,823 · Granted Nov 18, 2025

Computing system and method for applying monte carlo estimation to determine the contribution of dependent input variable groups on the output of a data science model

Inventors: Konstandinos Kotsiopoulos (Easthampton, MA); Alexey Miroshnikov (Evanston, IL); Arjun Ravi Kannan (Buffalo Grove, IL)
Assignee: Capital One Financial Corporation
G06F16/2465
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,475,132
App. No.
18/111,823
Granted
Nov 18, 2025
Kind
B2
Abstract

A computing platform is configured to (i) train a data science model object to receive an input data record including a set of input variables and output a score for the input data record, (ii) arrange the set of input variables into variable groups based on dependencies between input variables, (iii) identify an input data record to be scored by the model object, (iv) for each respective variable group, iterate the following: (a) identify a sample historical data record from a set of historical data records, (b) select a random group coalition including the respective variable group and zero or more other variable groups, and (c) use the input data record, the sample historical data record, and the randomly-selected group coalition to compute an iteration-specific contribution value, and (v) for each respective variable group, aggregate the iteration-specific contribution values for each iteration and thereby determine an aggregated contribution value for the respective variable group.

Claims (97)

1 . A computing platform comprising:

at least one processor;

non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to:

train a model object for a data science model using a machine learning process, wherein the model object is trained to (i) receive an input data record comprising a set of input variables and (ii) output a score for the input data record;

arrange the set of input variables into two or more variable groups based on dependencies between respective input variables, where each variable group comprises at least one input variable;

after training the model object and arranging the input variables into the two or more variable groups:

identify a given input data record to be scored by the model object;

use the model object to output a score for the given input data record; and

for each respective variable group of the model object, determine a respective contribution value that indicates a respective contribution of the respective variable group to the score for the given input data record by:

(a) applying an iterative Monte Carlo process to the respective variable group, wherein each given iteration of the iterative Monte Carlo process involves:

identifying a sample historical data record from a set of historical data records;

selecting a random group coalition of variable groups comprising the respective variable group and zero or more other variable groups;

using the randomly-selected group coalition as a basis for producing first and second synthetic data records that comprise two different mixes of data values from the given input data record and the sample historical data record, wherein (i) the first synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition and a data value from the sample historical data record for any other data variable in the model object's set of input variables and (ii) the second synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition but not included in the respective variable group being evaluated and a data value from the sample historical data record for any other data variable in the model object's set of input variables; and

using the first and second synthetic data records to compute an iteration-specific contribution value for the respective variable group; and

(b) aggregating the iteration-specific contribution values computed for each given iteration of the iterative Monte Carlo process and thereby determine the respective contribution value for the respective variable group that indicates the respective contribution of the respective variable group to the score for the given input data record.

2 . The computing platform of claim 1 , wherein selecting the random group coalition of variable groups comprising the respective variable group and zero or more other variable groups comprises:

generating a random ordering of the two or more variable groups; and

defining the randomly-selected group coalition as including the respective variable group and all other variable groups that precede the respective variable group in the generated random ordering.

3 . The computing platform of claim 1 , wherein using the first and second synthetic data records to compute an iteration-specific contribution value for the respective variable group comprises:

using the trained model object to determine a first score for the first synthetic data record and a second score for the second synthetic data record; and

calculating a difference between the first score and the second score, wherein the difference is the iteration-specific contribution value.

4 . The computing platform of claim 1 , wherein:

using the randomly-selected group coalition as a basis for producing the first synthetic data record comprises:

identifying a subset of input variables that are included in the randomly-selected group coalition;

for the identified subset of input variables, using values from the given input data record for the first synthetic data record; and

for each other input variable, using values from the sample historical data record for the first synthetic data record; and

using the randomly-selected group coalition as a basis for producing the second synthetic data record comprises:

for the identified subset of input variables, excluding input variables included in the respective group, using values from the given input data record for the second synthetic data record; and

for each other input variable, including input variables included in the respective group, using values from the sample historical data record for the second synthetic data record.

5 . The computing platform of claim 1 , wherein aggregating the iteration-specific contribution values computed for each given iteration of the iterative Monte Carlo process comprises:

determining an average of the iteration-specific contribution values for each given iteration of the iterative Monte Carlo process.

6 . The computing platform of claim 1 , wherein the iterative Monte Carlo process involves 1,000 or more iterations.

7 . The computing platform of claim 1 , wherein identifying the sample historical data record from the set of historical data records comprises:

identifying a randomly-sampled historical data record from the set of historical data records.

8 . A non-transitory computer-readable medium, wherein the non-transitory computer-readable medium is provisioned with program instructions that, when executed by at least one processor, cause a computing platform to:

train a model object for a data science model using a machine learning process, wherein the model object is trained to (i) receive an input data record comprising a set of input variables and (ii) output a score for the input data record;

arrange the set of input variables into two or more variable groups based on dependencies between respective input variables, where each variable group comprises at least one input variable;

after training the model object and arranging the input variables into the two or more variable groups:

identify a given input data record to be scored by the model object;

use the model object to output a score for the given input data record; and

for each respective variable group of the model object, determine a respective contribution value that indicates a respective contribution of the respective variable group to the score for the given input data record by:

(a) applying an iterative Monte Carlo process to the respective variable group, wherein each given iteration of the iterative Monte Carlo process involves:

identifying a sample historical data record from a set of historical data records;

selecting a random group coalition of variable groups comprising the respective variable group and zero or more other variable groups;

using the randomly-selected group coalition as a basis for producing first and second synthetic data records that comprise two different mixes of data values from the given input data record and the sample historical data record, wherein (i) the first synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition and a data value from the sample historical data record for any other data variable in the model object's set of input variables and (ii) the second synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition but not included in the respective variable group being evaluated and a data value from the sample historical data record for any other data variable in the model object's set of input variables; and

using the first and second synthetic data records to compute an iteration-specific contribution value for the respective variable group; and

(b) aggregating the iteration-specific contribution values computed for each given iteration of the iterative Monte Carlo process and thereby determine the respective contribution value for the respective variable group that indicates the respective contribution of the respective variable group to the score for the given input data record.

9 . The non-transitory computer-readable medium of claim 8 ,

selecting the random group coalition of variable groups comprising the respective group and zero or more other variable groups comprises:

generating a random ordering of the two or more variable groups; and

defining the randomly-selected group coalition as including the respective variable group and all other variable groups that precede the respective variable group in the generated random ordering.

10 . The non-transitory computer-readable medium of claim 8 ,

wherein using the first and second synthetic data records to compute an iteration-specific contribution value for the respective variable group comprises:

using the trained model object to determine a first score for the first synthetic data record and a second score for the second synthetic data record; and

calculating a difference between the first score and the second score, wherein the difference is the iteration-specific contribution value.

11 . The non-transitory computer-readable medium of claim 8 , wherein:

using the randomly-selected group coalition as a basis for producing the first synthetic data record comprises:

identifying a subset of input variables that are included in the randomly-selected group coalition;

for the identified subset of input variables, using values from the given input data record for the first synthetic data record; and

for each other input variable, using values from the sample historical data record for the first synthetic data record; and

using the randomly-selected group coalition as a basis for producing the second synthetic data record comprises:

for the identified subset of input variables, excluding input variables included in the respective group, using values from the given input data record for the second synthetic data record; and

for each other input variable, including input variables included in the respective group, using values from the sample historical data record for the second synthetic data record.

12 . The non-transitory computer-readable medium of claim 8 , aggregating the iteration-specific contribution values computed for each given iteration of the iterative Monte Carlo process comprises:

determining an average of the iteration-specific contribution values for each given iteration of the iterative Monte Carlo process.

13 . The non-transitory computer-readable medium of claim 8 , wherein the iterative Monte Carlo process involves 1,000 or more iterations.

14 . The non-transitory computer-readable medium of claim 8 , wherein identifying the sample historical data record from the set of historical data records comprises:

identifying a randomly-sampled historical data record from the set of historical data records.

15 . A method carried out by a computing platform, the method comprising:

training a model object for a data science model using a machine learning process, wherein the model object is trained to (i) receive an input data record comprising a set of input variables and (ii) output a score for the input data record;

arranging the set of input variables into two or more variable groups based on dependencies between respective input variables, where each variable group comprises at least one input variable;

after training the model object and arranging the input variables into the two or more variable groups:

identifying a given input data record to be scored by the model object;

using the model object to output a score for the given input data record; and

for each respective variable group of the model object, determining a respective contribution value that indicates a respective contribution of the respective variable group to the score for the given input data record by:

(a) applying an iterative Monte Carlo process to the respective variable group, wherein each given iteration of the iterative Monte Carlo process involves:

identifying a sample historical data record from a set of historical data records;

selecting a random group coalition of variable groups comprising the respective variable group and zero or more other variable groups; using the randomly-selected group coalition as a basis for producing first and second synthetic data records that comprise two different mixes of data values from the given input data record and the sample historical data record, wherein (i) the first synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition and a data value from the sample historical data record for any other data variable in the model object's set of input variables and (ii) the second synthetic data record comprises a data value from the given input data record for any data variable that is included in the randomly-selected group coalition but not included in the respective variable group being evaluated and a data value from the sample historical data record for any other data variable in the model object's set of input variables; and

using the first and second synthetic data records to compute an iteration-specific contribution value for the respective variable group; and

(b) aggregating the iteration-specific contribution values computed for each given iteration of the iterative Monte Carlo process and thereby determine the respective contribution value for the respective variable group that indicates the respective contribution of the respective variable group to the score for the given input data record.

16 . The method of claim 15 , wherein selecting the random group coalition of variable groups comprising the respective group and zero or more other variable groups comprises:

generating a random ordering of the two or more variable groups; and

defining the randomly-selected group coalition as including the respective variable group and all other variable groups that precede the respective variable group in the generated random ordering.

17 . The method of claim 15 , wherein using the first and second synthetic data records to compute an iteration-specific contribution value comprises:

using the trained model object to determine a first score for the first synthetic data record and a second score for the second synthetic data record; and

calculating a difference between the first score and the second score, wherein the difference is the iteration-specific contribution value.

18 . The method of claim 15 , wherein;

using the randomly-selected group coalition as a basis for producing the first synthetic data record comprises:

identifying a subset of input variables that are included in the randomly-selected group coalition;

for the identified subset of input variables, using values from the given input data record for the first synthetic data record; and

for each other input variable, using values from the sample historical data record for the first synthetic data record; and

using the randomly-selected group coalition as a basis for producing the second synthetic data record comprises:

for the identified subset of input variables, excluding input variables included in the respective group, using values from the given input data record for the second synthetic data record; and

for each other input variable, including input variables included in the respective group, using values from the sample historical data record for the second synthetic data record.

19 . The computing platform of claim 1 , wherein the iterative Monte Carlo process is applied in parallel for each respective variable group of the model object.

20 . The method of claim 15 , wherein the iterative Monte Carlo process is applied in parallel for each respective variable group of the model object.

Assignments (2)
MERGER Recorded Jul 2, 2025
From: DISCOVER FINANCIAL SERVICES
To: CAPITAL ONE FINANCIAL CORPORATION
Reel/Frame 071784/0903 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2023
From: KOTSIOPOULOS, KONSTANDINOS; MIROSHNIKOV, ALEXEY; KANNAN, ARJUN RAVI
To: DISCOVER FINANCIAL SERVICES
Reel/Frame 062961/0716 →
Continuity (1)
Related Publication 20240281450A1 · Aug 22, 2024
References Cited (178)
US 8370253B1 · Grossman · 2013 [cited by applicant]
US 11030583B1 · Garg et al. · 2021 [cited by applicant]
US 11636386B2 · Lohia et al. · 2023 [cited by applicant]
US 12020133B2 · Goldszmidt et al. · 2024 [cited by applicant]
US 12277455B2 · Walters et al. · 2025 [cited by applicant]
US 20110270782A1 · Trenner · 2011 [cited by examiner]
US 20140229415A1 · Martineau et al. · 2014 [cited by applicant]
US 20180285685A1 · Singh et al. · 2018 [cited by applicant]
US 20190087744A1 · Schiemenz · 2019 [cited by examiner]
US 20190188605A1 · Zavesky et al. · 2019 [cited by applicant]
US 20190279111A1 · Merrill et al. · 2019 [cited by applicant]
US 20190378210A1 · Merrill et al. · 2019 [cited by applicant]
US 20200081865A1 · Farrar et al. · 2020 [cited by applicant]
US 20200082299A1 · Affonso et al. · 2020 [cited by applicant]
US 20200184350A1 · Bhide et al. · 2020 [cited by applicant]
US 20200218987A1 · Jiang et al. · 2020 [cited by applicant]
US 20200302309A1 · Golding · 2020 [cited by applicant]
US 20200302524A1 · Kamkar et al. · 2020 [cited by applicant]
US 20200311486A1 · Dey et al. · 2020 [cited by applicant]
US 20200320428A1 · Chaloulos et al. · 2020 [cited by applicant]
US 20200372035A1 · Tristan et al. · 2020 [cited by applicant]
US 20200372304A1 · Kenthapadi et al. · 2020 [cited by applicant]
US 20200372406A1 · Wick · 2020 [cited by applicant]
US 20200380398A1 · Weirder et al. · 2020 [cited by applicant]
US 20200410129A1 · Nadler et al. · 2020 [cited by applicant]
US 20210081828A1 · Lopez De Prado · 2021 [cited by examiner]
US 20210133870A1 · Kamkar et al. · 2021 [cited by applicant]
US 20210158102A1 · Lohia et al. · 2021 [cited by applicant]
US 20210173402A1 · Chang et al. · 2021 [cited by applicant]
US 20210174222A1 · Dodwell et al. · 2021 [cited by applicant]
US 20210224605A1 · Zhang et al. · 2021 [cited by applicant]
US 20210224687A1 · Goldszmidt et al. · 2021 [cited by applicant]
US 20210241033A1 · Yang · 2021 [cited by applicant]
US 20210248503A1 · Hickety et al. · 2021 [cited by applicant]
US 20210256832A1 · Weisz et al. · 2021 [cited by applicant]
US 20210304039A1 · Tang et al. · 2021 [cited by applicant]
US 20210334654A1 · Himanshi et al. · 2021 [cited by applicant]
US 20210350272A1 · Miroshnikov et al. · 2021 [cited by applicant]
US 20210383268A1 · Miroshnikov · 2021 [cited by examiner]
US 20210383275A1 · Miroshnikov · 2021 [cited by examiner]
US 20220036203A1 · Nachum et al. · 2022 [cited by applicant]
US 20240095605A1 · Fylstra · 2024 [cited by examiner]
WO 2021074491A1 · 2021 [cited by applicant]
Blum et al. Recovering from Biased Data: Can Fairness Constraints Improve Accuracy? Toyota Technological Institute at Chicago arXiv:1912.01094v1 [cs.LG] Dec. 4, 2019, 20 pages. [cited by applicant]
Varley et al. Fairness in Machine Learning with Tractable Models. School of Informatics, University of Edinburgh, UK arXiv:1905.07026v2 [cs.LG] Jan. 13, 2020, 26 pages. [cited by applicant]
Hickey et al. Fairness by Explicability and Adversarial SHAP Learning. Experian UK&I and EMEA DataLabs, London, UK arXiv:2003.05330v3 [cs.LG] Jun. 26, 2020, 17 pages. [cited by applicant]
Manu Joseph. Interpretability Cracking open the black box—Part 11. Machine Learning. Deep & Shallow, Nov. 16, 2019-Nov. 28, 2019, 23 pages. [cited by applicant]
Manu Joseph. Interpretability Cracking open the black box—Part 111. Machine Learning. Deep & Shallow, Nov. 24, 2019-Dec. 30, 2020, 29 pages. [cited by applicant]
Krivoruchko et al. Using Multivariate Interpolation for Estimating Well Performance, Esri.com, Arcuser. Esri Technology Summer 2014, 13 pages. URL:<URL: https://www.esri.com/abouUnewsroom/arcuser/using-multivariate-inte… [cited by applicant]
Chen et al. Comparison and Improvement of the predictability and interpretability with ensemble learning models in QSPR applications. Journal of Cheminformatics, Mar. 30, 2020, 50 pages. <URL:https://jcheminf.biomedcent… [cited by applicant]
Ding et al. Comparisons of Two Ensemble Mean Methods in Measuring the Average Error Growth and the Predictability. The Chinese Meteorological Society and Springer-Verlag Berlin Heidelberg, No. 4, vol. 25, Mar. 21, 2011,… [cited by applicant]
Hu et al. Locally Interpretable Models and Effects based on Supervised Partitioning, Wells Fargo, Jun. 1, 2018, 15 pages <URL: https://arxiv.org/pdf/1806.00663>. [cited by applicant]
Jiang et al. Wasserstein Fair Classification. Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, arXiv:1907.12059v1 [stat.ML] Jul. 28, 2019, 15 pages. <URL: chrome-extension://efaidnbm… [cited by applicant]
Wei et al. Optimized Score Transformation for Consistent Fair Classication, arXiv: 1906.00066v3 [cs.LG] Oct. 29, 2021, 78 pages <URL: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://arxiv.org/pdf/1906.00066.… [cited by applicant]
Iosifidis et al. FAE: A Fairness-Aware Ensemble Framework, IEEE, arXiv:2002.00695v1 [cs.Al] Feb. 3, 2020, 6 pages. <URL:https://arxiv.org/abs/2002.00695>. [cited by applicant]
Abduljabbar et al., “Applications of Artificial Intelligence in Transport: An Overview,” Sustainability (2019). [cited by applicant]
Albizuri et al. On Coalitional Semivalues. Games and Economic Behavior 49, Nov. 29, 2002, 37 pages. [cited by applicant]
Alonso-Meijide et al. Modification of the Banzhaf Value for Games with a Coalition Structure. Annals of Operations Research, Jan. 2002, 17 pages. [cited by applicant]
Amer et al. The modified Banzhaf value for games with coalition structure: an axiomatic characterization. Mathematical Social Sciences 43, (2002) 45-54, 10 pages. [cited by applicant]
Apley et al., “Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models,” Ar Xiv (2016). [cited by applicant]
Aumann et al. Cooperative Games with Coalition Structure. International journal of Game Theory, vol. 3, Issue 4, Jul. 1974, pp. 217-237. [cited by applicant]
Aurenhammer et al., Voronoi Diagrams And Delaunay Triangulations, pp. 7-13, 41-45, New Jersey, World Scientific Publishing (2013). [cited by applicant]
Breiman et al. Estimating Optimal Transformations for Multiple Regression and Correlation. Journal of the American Statistical Association. vol. 80, No. 391, Sep. 1985, 19 pages. [cited by applicant]
Bzdok et al., “Statistics Versus Machine Learning,” Nature Methods, vol. 15, 233-234 (Apr. 2018). [cited by applicant]
Casas-Mendez et al. An extension of the tau—value to games with coalition structures. European Journal of Operational Research, vol. 148, 2003, pp. 494-513. [cited by applicant]
Dubey et al. Value Theory Without Efficiency. Mathematics of Operations Research. Volume 6, No. 1. 1981, pp. 122-128. [cited by applicant]
Eggleston, H.G., Convexity, pp. 33-38, Cambridge University Press (1958). [cited by applicant]
Elshawi et al. On the interpretability of machine learning-based model for predicting hypertension. BMC Medical Informatics and Decision Making. vol. 19, No. 146. 2019, 32 pages. [cited by applicant]
Explainable Artificial Intelligence (xAI), Defense Advanced Research Projects Agency, DARPA-BAA-16-53 (2016). [cited by applicant]
FDIC Title VII-Equal Credit Opportunity, FDIC Law, Regulations, Related Acts—Consumer Protection (Dec. 31, 2019). [cited by applicant]
Gretton et al. A Kernel Statistical Test of Independence. In Advances in Neural Information Processing Systems. 2007, 8 pages. [cited by applicant]
Gretton et al. A Kernel Two-Sample Test. The Journal of Machine Learning Research. Volume 13, No. 1. Mar. 2012, pp. 723-773. [cited by applicant]
Gretton et al. Measuring Statistical Dependence with Hilbert-Schmidt Norms. In Algorithmic Learning Theory. Springer-Verlag Berlin Heidelberg. 2005, pp. 63-77. [cited by applicant]
Hall et al. An Introduction to Machine Learning Interpretability, O'Reilly, Second Edition, Aug. 2019, 62 pages. [cited by applicant]
Hoeffding, Wassily. A Non-Parametric Test of Independence. The Annals of Mathematical Statistics. 1948, pp. 546-557. [cited by applicant]
Ji et al. Post-Radiotherapy PET Image Outcome Prediction by Deep Learning Under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application. Duke University Medical Center. arXiv preprint. 2021, 2… [cited by applicant]
Jiang et al. Nonparametric K-Sample Tests via Dynamic Slicing. Journal of the American Statistical Association. Harvard Library. Sep. 11, 2015. pp. 642-653. [cited by applicant]
Jollife et al. Principal Component Analysis. Springer Series In Statistics. Second Edition, 2002, 518 pages. [cited by applicant]
Kamijo, Yoshio. A two-step Shapley value in a cooperative game with a coalition structure. International Game Theory Review. 2009, 11 pages. [cited by applicant]
Kraskov et al. Estimating mutual information. The American Physical Society. Physical Review E vol. 69. Jun. 23, 2004, 16 pages. [cited by applicant]
Leo Breiman. Statistical Modeling: The Two Cultures. Statistical Science, vol. 16, No. 3, 2001, pp. 199-231. [cited by applicant]
Leo et al., “Machine Learning in Banking Risk Management: A Literature Review,” Risks (2019). [cited by applicant]
Lopez-Paz et al. The Randomized Dependence Coefficient. In Advances in Neural Information Processing Systems. 2013, pp. 1-9. [cited by applicant]
Lundberg, Scott. Welcome to SHAP Documentation. 2018, 46 pages [online]. Retrieved online URL:<https://shap.readthedocs.io/en/latest/>. [cited by applicant]
Miroshinikov et al. Wasserstein-based faireness interpretability framework for machine learning models. arXiv:2011.03156v1. Nov. 6, 2020, 34 pages. [cited by applicant]
Modarres et al. Towards Explainable Deep Learning for Credit Lending: A Case Study. arxiv:1811.06471v2, Nov. 11, 2018, 8 pages. [cited by applicant]
Paninski, Liam. Estimation of Entropy and Mutual Information. Neural Computation, vol. 15, No. 6. 2003, pp. 1191-1253. [cited by applicant]
Renyi. On Measures of Dependence. Acta mathematica hungarica, vol. 10, No. 3.1959, pp. 441-451. [cited by applicant]
SAS/STAT User's Guide, Version 8, Chapter 68 “The VARCLUS Procedure” (1999). [cited by applicant]
Shiryaev, A.N., Graduate Texts in Mathematics: Probability. 2nd ed., pp. 23-43, Springer (1980). [cited by applicant]
Szekely et al. Brownian Distance Covariance. The Annals of Applied Statistics, vol. 3, No. 4. 2009. 31 pages. [cited by applicant]
Tibshirani et al. Estimating the No. of clusters in a dataset via the gap statistic. Journal of the Royal Statistical Society, Series B. vol. 63, Part 2. 2001. pp. 411-423. [cited by applicant]
Tijs. Bounds for the Core and The Tau-Value. Game Theory and Mathematical Economics. 1981, pp. 123-132. [cited by applicant]
Vanderbei, Robert J. Linear Programming: Foundations and Extensions, 4th ed., pp. 11-16, 141-144, New York, NY, Springer Science and Business Media (2014). [cited by applicant]
Vaughan et al., “Explainable Neural Networks based on Additive Index Models,” Corporate Model Risk, Wells Fargo, USA (Jun. 2018). [cited by applicant]
Vidal-Puga, Juan. The Harsanyi paradox and the “right to talk” in bargaining among coalitions. Mathematical Social Sciences. vol. 64. Mar. 27, 2012. 32 pages. [cited by applicant]
Wang et al. Shapley Flow: A Graph-based Approach to Interpreting Model Predictions arXiv preprint arXiv:2010.14592, 2020, 11 pages. [cited by applicant]
Aas et al. Explaining Individual Predictions When Features are Dependent: More Accurate Approximations to Shapley Values. arXiv preprint arXiv:1903.10464v3. Feb. 6, 2020. 28 pages. [cited by applicant]
Abduljabbar et al. Applications of Artificial Intelligence in Transport: An Overview. Sustainability. 2019. 24 pages. [cited by applicant]
Wehle et al., “Machine Learning, Deep Learning, and AI: What's the Difference?” Data Scientist Innovation Day (2017). [cited by applicant]
Wuest et al., “Machine Learning in Manufacturing: Advantages, Challenges and Applications, ” Production and Manufacturing Research, vol. 4, 23-45 (2016). [cited by applicant]
Zeng et al. Jackknife approach to the estimation of mutual information. PNAS, vol. 115, No. 40. Oct. 18, 2018. 6 pages. [cited by applicant]
Zhao et al. Causal Interpretations of Black-Box Models. J Bus., Econ. Stat., DOI:10.1080/07350015.2019.1624293, 2019, 16 pages. [cited by applicant]
International Search Report and Written Opinion issued in International Application No. PCT/US2022/029670, mailed on Sep. 7, 2022, 10 pages. [cited by applicant]
Perrone et al. Fair Bayesian Optimization. AIES '21, May 19-21, 2021, Virtual Event, USA. pp. 854-863. [cited by applicant]
Rawls, John. Justice as Fairness: A Restatement. Harvard University Press. 2001, 32 pages. [cited by applicant]
Reshef et al. An Empirical Study of Leading Measures of Dependence. arXiv preprint arXiv:1505.02214. May 12, 2015, 42 pages. [cited by applicant]
Reshef et al. Detecting Novel Associations in Large Data Sets. Science, vol. 334. Dec. 2011, pp. 1518-1524. [cited by applicant]
Reshef et al. Measuring Dependence Powerfully and Equitably. Journal of Machine Learning Research, vol. 17. 2016, 63 pages. [cited by applicant]
Ribeiro et al. “Why Should I Trust You?” Explaining the Predictions of Any Classifier. 22nd Conference on Knowledge Discovery and Data Mining, San Francisco, CA. arXiv:1602.04938v3. Aug. 9, 2016, 10 pages. [cited by applicant]
Royden et al. Real Analysis. Boston: Prentice Hall, 4th Edition. 2010, 516 pages. [cited by applicant]
Santambrogio, Filippo. Optimal Transport for Applied Mathematicians. Calculus of Variations, PDEs and Modeling. Springer. May 2015, 356 pages. [cited by applicant]
Schmidt et al. An Introduction to Artificial Intelligence and Solutions to the Problems of Algorithmic Discrimination. arXiv preprint arXiv:1911.05755. vol. 73, No. 2. 2019, pp. 130-144. [cited by applicant]
Shapley, L.S. A Value for n-Person Games. Annals of Mathematics Studies. 1953, 13 pages. [cited by applicant]
Shiryaev, A.N. Probability. Springer, Second Edition. 1980, 22 pages. [cited by applicant]
Shorack et al. Empirical Processes with Applications to Statistics. Wiley, New York. 1986, 36 pages. [cited by applicant]
Strumbelji et al. Explaining Prediction Models and Individual Predictions with Feature Contributions. Springer. Knowledge Information System, vol. 41, No. 3. 2014, pp. 647-665. [cited by applicant]
Sundararajan et al. The Many Shapley Values for Model Explanation. arXiv preprint arXiv:1908.08474v1, Aug. 22, 2019, 9 pages. [cited by applicant]
Szekely et al. Measuring and Testing Dependence by Correlation of Distances. The Annals of Statistics, vol. 35, No. 6. ArXiv:0803.4101v1, Mar. 28, 2008, 26 pages. [cited by applicant]
Thorpe, Matthew, Introduction to Optimal Transport, University of Cambridge, 2018, 56 pages. [cited by applicant]
Villani, Cedric. Optimal Transport, Old and New. Springer. Jun. 13, 2008, 998 pages. [cited by applicant]
Wasserman et al. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. 2004, 14 pages. [cited by applicant]
Woodworth et al. Learning Non-Discriminatory Predictors. Proceedings of Machine Learning Research, vol. 65, No. 1-34. 2017, 34 pages. [cited by applicant]
Young. Monotonic Solutions of Cooperative Games International Journal of Game Theory, vol. 14, Issue 2. 1985. 8 pages. [cited by applicant]
Zemel et al. Learning Fair Representations. Proceedings of the 30th International Conference on Machine Learning, PMLR vol. 28, No. 3. 2013, pp. 325-333. [cited by applicant]
Zhang et al. Mitigating Unwanted Biases with Adversarial Learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Feb. 2-3, 2018, pp. 335-340. [cited by applicant]
Kwak, Nojun et al. Input Feature Selection for Classification Problems. IEEE Transactions on Neural Networks. Volume 13, No. 1. pp. 143-159. 2002. 17 pages. [cited by applicant]
Lundberg, Scott M., et al. Consistent Individualized Feature Attribution for Tree Ensembles. arXiv preprint arxiv:1802.03888v2. Jun. 18, 2018. 9 pages. [cited by applicant]
Balashankar et al. Pareto-Efficient Fairness for Skewed Subgroup Data. In the International Conference on Machine earning AI for Social Good Workshop. vol. 8. Long Beach, United States, 2019, 8 pages. [cited by applicant]
Banzhaf III. Weighted Voting Doesn't Work: A Mathematical Analysis. Rutgers Law Review, vol. 19, No. 2, 1965, 28 pages. [cited by applicant]
Barocas et al. Fairness and Machine Learning: Limitations and Opportunities. [online], 253 p. 1999 [retrieved online on May 26, 2022]. Retrieved from the Internet:<URL:https://fairmlbook.org/>. [cited by applicant]
Bergstra et al. Algorithms for Hyper-Parameter Optimization. NIPS 11: Proceedings of the 24th International conference on Neural Information Processing Systems. Dec. 2011, 9 pages. [cited by applicant]
Bousquet et al. Stability and Generalization. Journal of Machine Learning Research 2. Mar. 2002, pp. 499-526. [cited by applicant]
Chen et al. True to Model or True to the Data? arXiv preprint arXiv:2006.1623v1, Jun. 29, 2020, 7 pages. [cited by applicant]
Cover et al. Elements of Information Theory. A John Wiley & Sons, Inc., Publication. Second Edition, 2006, 774 pages. [cited by applicant]
Del Barrio et al. Obtaining Fairness Using Optimal Transport Theory. arXiv preprint arXiv:1806.03195v2. Jul. 19, 2018, 25 pages. [cited by applicant]
Del Barrio et al. On Approximate Validation of Models: A Kolmogorov-Smirnov Based Approach. arXiv:1903.08687v1, Mar. 20, 2019, 32 pages. [cited by applicant]
Dickerson et al. Machine Learning. Considerations for fairly and transparently expanding access to credit. 2020, 29 pages [online]. Retrieved online: URL<https://info.h2o.ai/rs/644-PKX-778/images/Machine%20Learning%20-%… [cited by applicant]
Dwork et al. Generalization in Adaptive Data Analysis and Holdout Reuse. NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, vol. 2, Dec. 2015, 9 pages. [cited by applicant]
Dwork et al. Fairness Through Awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 2012, 214-226. [cited by applicant]
Equal Credit Opportunity Act (Ecoa). Fdic Law, Regulations, Related Actions, 9 pages [online]. Retrieved online: URL<https://www.fdic.gov/regulations/laws/rules/6000-1200.html>. [cited by applicant]
Equal Employment Opportunity Act. 2 pages [online]. Retrieved online: URL<https://www.dol.gov/sites/dolgov/files/ofccp/regs/compliance/posters/pdf/eeopost.pdf>. [cited by applicant]
Fair Housing Act (FHA). FDIC Law, Regulations, Related Actions, 2022, 3 pages [online]. Retrieved online: URL<https://www.ecfr.gov/current/title-12/chapter-III/subchapter-B/part-338. [cited by applicant]
Feldman et al. Certifying and Removing Disparate Impact. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, 259-268. [cited by applicant]
Friedman, Jerome. 1999 Reitz Lecture. Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, vol. 29, No. 6. 2001, pp. 1189-1232. [cited by applicant]
G. Owen. Modification of the Banzhaf-Coleman index for Games with a Priori Unions. Power, Voting and Voting Power. 1981, pp. 232-238. [cited by applicant]
Goldstein et al. Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation. arXiv:1309.6392v2J. Mar. 20, 2014, 22 pages. [cited by applicant]
Hall et al. A United States Fair Lending Perspective on Machine Learning. Frontiers in Artificial Intelligence. doi: 10.3389/frai.2021.695301, Jun. 7, 2021, 9 pages. [cited by applicant]
Hall, Patrick. On the Art and Science of Explainable Machine Learning: Techniques, Recommendations, and Responsibilities. KDD'19 XAI Workshop. arXiv preprint arxiv:1810.02909, Aug. 2, 2019, 10 pages. [cited by applicant]
Hardt et al. Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems, 2015, 3315-3323. [cited by applicant]
Hashimoto et al. Fairness Without Demographics in Repeated Loss Minimization. In ICML, 2018, 10 pages. [cited by applicant]
Hastie et al. The Elements of Statistical Learning: Data Mining, Interference, and Prediction. Springer. Second edition, Jan. 13, 2017, 764 pages. [cited by applicant]
Heller et al. A consistent multivariate test of association based on ranks of distances. Biometrika. arXiv:1201.3522v1. Jan. 17, 2012, 14 pages. [cited by applicant]
Jeller et al. Consistent Distribution-Free K-Sample and Independence Tests for Univariate Random Variables. Journal of Machine Learning Research. vol. 17, No. 29. Feb. 2016, 54 pages. [cited by applicant]
Janzing et al. Feature relevance quantification in explainable AI:A causal problem. arXiv preprint arXiv:1910.13413v2. Nov. 25, 2019, 11 pages. [cited by applicant]
Jiang et al. Identifying and Correcting Label Bias in Machine Learning. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS). 2020, 10 pages. [cited by applicant]
Jiang et al. Smooth Isotonic Regression: A New Method to Calibrate Predictive Models. AMIA Joint Summits on Translational Science proceedings. 2011, pp. 16-20. [cited by applicant]
Kamiran et al. Classifying without Discriminating. 2009 2nd International Conference on Computer, Control and Communication. doi: 10.1109/IC4.2009.4909197. 2009, pp. 1-6. [cited by applicant]
Kamiran et al. Data Preprocessing Techniques for Classification Without Discrimination. Knowl. Inf. Syst. DOI 10.1007/s10115-011-0463-8. Dec. 3, 2011, 33 pages. [cited by applicant]
Kamiran et al. Discrimination Aware Decision Tree Learning. 2010 IEEE International Conference on Data Mining. doi: 10.1109/ICDM.2010.50. 2010, pp. 869-874. [cited by applicant]
Kamishima et al. Fairness-Aware Classifier with Prejudice Remover Regularizer. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), Part … [cited by applicant]
Karush, William. Minima of Functions of Several Variables with Inequalities as Side Constraints. A Dissertation Submitted to the Faculty of the Division of the Physical Sciences in Candidacy for the Degree of Master in … [cited by applicant]
Kearns et al. Algorithmic Stability and Sanity-Check Bounds for Leave-One-Out Cross-Validation. Neural Computation, vol. 11, No. 6. 1999, pp. 1427-1453. [cited by applicant]
Koralov et al. Theory of Probability and Random Processes. Second Edition. Springer. 1998, 349 pages. [cited by applicant]
Kovalev et al. A Robust Algorithm for Explaining Unreliable Machine Learning Survival Models Using the Kolmogorov-Smirnov Bounds. Neural Networks. arXiv:2005.02249v1, May 5, 2020. pp. 1-39. [cited by applicant]
Kuhn et al. Nonlinear programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, Calif. University of California Press. 90. 1951, pp. 481-492. [cited by applicant]
Lahoti et al. Fairness without Demographics through Adversarially Reweighted Learning. arXiv preprint arXiv:2006.13114. 2020, 13 pages. [cited by applicant]
Lipovetsky et al. Analysis of Regression in Game Theory Approach. Applied Stochastic Models Business and Industry. vol. 17. Apr. 26, 2001, pp. 319-330. [cited by applicant]
Lorenzo-Freire, Silvia. New characterizations of the Owen and Banzhaf-Owen values using the intracoalitional balanced contributions property. Department of Mathematics. 2017, 23 pages. [cited by applicant]
Lundberg et al. Consistent Individualized Feature Attribution for Tree Ensembles. arXiv preprint arxiv:1802.03888v3. Mar. 7, 2019, 9 pages. [cited by applicant]
Lundberg et al. A Unified Approach to Interpreting Model Predictions. 31st Conference on Neural Information Processing Systems, 2017, 10 pages. [cited by applicant]
Markowitz, Harry. Portfolio Selection. The Journal of Finance. vol. 7, No. 1. 1952, pp. 77-91. [cited by applicant]
Miroshinikov et al. Model-Agnostic Bias Mitigation Methods with Regressor Distribution Control for Wasserstein-Based Fairness Metrics. arXiv:2111.11259v1. Nov. 19, 2021, 29 pages. [cited by applicant]
Miroshinikov et al. Mutual Information-Based Group Explainers with Coalition Structure for Machine Learning Model Explanations. Computer Science and Game Theory. arXiv:2102.10878v1. Feb. 22, 2021, 46 pages. [cited by applicant]
Miroshinikov et al. Wasserstein-Based Fairness Interpretability Framework for Machine Learning Models. arXiv:2011.03156v1. Nov. 6, 2020, 34 pages. [cited by applicant]
Owen, Guillermo. Values of Games with a Priori Unions. In: Essays in Mathematical Economics and Game Theory. Springer, 1977, pp. 76-88. [cited by applicant]
Pareto. Optimality. In: Multiobjective Linear Programming. Springer International Publishing. DOI 10.1007/978-3-319-21091-9_4. 2016, pp. 85-118. [cited by applicant]
Casalicchio, Giuseppe. Visualizing the Feature Importance for Black Box Models. Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer Science. Vol. 11051. Springer. pp. 655-670… [cited by applicant]