IP Library › Granted Patent US 12,443,875
Granted Patent B2
US 12,443,875 · App. 17/093,978 · Granted Oct 14, 2025

Explanatory confusion matrices for machine learning

Inventors: Alex Swain (Cedar Park, TX); Stefan A. G. Van Der Stockt (Austin, TX); Edward James Biddle (Winchester, GB); Daniel Kuehn (Austin, TX)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,443,875
App. No.
17/093,978
Granted
Oct 14, 2025
Kind
B2
Abstract

In an approach to creating explanatory confusion matrices, responsive to receiving a machine learning model for analysis, a confusion matrix is calculated for the machine learning model, where each cell in the confusion matrix has a corresponding set of data. A link is created from each cell in the confusion matrix to the corresponding set of data. Responsive to a user selecting in a user interface a specific cell of the confusion matrix, the corresponding set of data to the specific cell is displayed on the user interface.

Claims (33)

1. A computer implemented method for creating explanatory confusion matrices, comprising:

building, by one or more computer processors with access to classification results from a machine learning model, a user interface including a confusion matrix having a plurality of cells for visualizing the classification results, wherein the plurality of cells includes respective electronic links to a corresponding set of data that produced the classification results visualized in the plurality of cells, and wherein each cell of the plurality of cells that is associated with a respective incorrect classification by the machine learning model is selectable to execute an explanatory application that is configured to identify at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and at least one recommendation to correct the machine learning model;

executing, based on a user manipulation of the user interface to select a specific cell in the confusion matrix that is associated with the respective incorrect classification by the machine learning model, the explanatory application to identify the at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and the at least one recommendation to correct the machine learning model;

visualizing, by the one or more computer processors on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model;

causing, based on the user manipulation of the user interface to correct the respective incorrect classification associated with the specific cell based on the at least one recommendation displayed on the user interface, at least one correction to each misclassified data associated with the specific cell; and

responsive to the user manipulation of the user interface to correct each misclassified data associated with the specific cell, generating, by the one or more computer processors, correctly classified data and retraining, by the one or more computer processors, the machine learning model with the correctly classified data to adjust at least one feature of the machine learning model such that the machine learning model retrained to include the at least one adjusted feature correctly classifies each misclassified data associated with the specific cell and improves future predictions on unseen data.

2. The computer implemented method of claim 1 , wherein the visualizing the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model further comprises:

displaying, by the one or more computer processors, one or more explainability metrics of the specific cell of the plurality of cells that was misclassified.

3. The computer implemented method of claim 1 , wherein the visualizing the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model further comprises:

exporting, by the one or more computer processors, the corresponding set of data as a graphical representation on the user interface.

4. A computer program product for creating explanatory confusion matrices, comprising:

one or more computer readable storage media; and program instructions stored on the one or more computer readable storage media to perform operations comprising:

building, based on classification results from a machine learning model, a user interface including a confusion matrix having a plurality of cells for visualizing the classification results, wherein the plurality of cells includes respective electronic links to a corresponding set of data that produced the classification results visualized in the plurality of cells, and wherein each cell of the plurality of cells that is associated with a respective incorrect classification by the machine learning model is selectable to execute an explanatory application that is configured to identify at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and at least one recommendation to correct the machine learning model;

executing, based on a user manipulation of the user interface to select a specific cell in the confusion matrix that is associated with the respective incorrect classification by the machine learning model, the explanatory application to identify the at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and the at least one recommendation to correct the machine learning model;

visualizing, on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model;

causing, based on the user manipulation of the user interface to correct the respective incorrect classification associated with the specific cell based on the at least one recommendation displayed on the user interface, at least one correction to each misclassified data associated with the specific cell; and

responsive to the user manipulation of the user interface to correct each misclassified data associated with the specific cell, generating correctly classified data and retraining the machine learning model with the correctly classified data to adjust at least one feature of the machine learning model such that the machine learning model retrained to include the at least one adjusted feature correctly classifies each misclassified data associated with the specific cell and improves future predictions on unseen data.

5. The computer program product of claim 4 , wherein the visualizing, on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model further comprises:

displaying one or more explainability metrics of the specific cell of the plurality of cells that was misclassified.

6. The computer program product of claim 4 , wherein the visualizing, on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model further comprises:

exporting the corresponding set of data as a graphical representation on the user interface.

7. A computer system for creating explanatory confusion matrices, the computer system comprising:

a processor set;

one or more computer readable storage media; and

program instructions stored on the one or more computer readable storage media to cause the processor set to perform operations comprising:

building, by the processor set with access to classification results from a machine learning model, a user interface including a confusion matrix having a plurality of cells for visualizing the classification results, wherein the plurality of cells includes respective electronic links to a corresponding set of data that produced the classification results visualized in the plurality of cells, and wherein each cell of the plurality of cells that is associated with a respective incorrect classification by the machine learning model is selectable to execute an explanatory application that is configured to identify at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and at least one recommendation to correct the machine learning model;

executing, based on a user manipulation of the user interface to select a specific cell in the confusion matrix that is associated with the respective incorrect classification by the machine learning model, the explanatory application to identify the at least one data-specific reason that each misclassified data resulting in the respective incorrect classification is incorrectly classified and the at least one recommendation to correct the machine learning model;

visualizing, on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model;

causing, based on the user manipulation of the user interface to correct the respective incorrect classification associated with the specific cell based on the at least one recommendation displayed on the user interface, at least one correction to each misclassified data associated with the specific cell; and

responsive to the user manipulation of the user interface to correct each misclassified data associated with the specific cell, generating correctly classified data and retraining the machine learning model with the correctly classified data to adjust at least one feature of the machine learning model such that the machine learning model retrained to include the at least one adjusted feature correctly classifies each misclassified data associated with the specific cell and improves future predictions on unseen data.

8. The computer system of claim 7 , wherein the visualizing, on the user interface, the at least one data-specific reason that each misclassified data associated with the specific cell is incorrectly classified and the at least one recommendation to correct the machine learning model further comprises:

displaying one or more explainability metrics of the specific cell of the plurality of cells that was misclassified.

9. The computer implemented method of claim 1 , further comprising: generating a database including a record for the each cell of the plurality of cells, wherein the record includes the corresponding set of data accessible by the respective electronic links from the plurality of cells.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2020
From: SWAIN, ALEX; VAN DER STOCKT, STEFAN A. G.; BIDDLE, EDWARD JAMES; KUEHN, DANIEL
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 054323/0400 →
Continuity (1)
Related Publication 20220147862A1 · May 12, 2022
References Cited (33)
US 8611675B2 · Chow · 2013 [cited by examiner]
US 10354205B1 · Pham · 2019 [cited by applicant]
US 10719301B1 · Dasgupta · 2020 [cited by examiner]
US 20080154807A1 · Chow · 2008 [cited by examiner]
US 20150254555A1 · Williams, Jr. · 2015 [cited by applicant]
US 20200293940A1 · Chopra · 2020 [cited by examiner]
US 20210256420A1 · Elisha · 2021 [cited by examiner]
CN 107229976A · 2017 [cited by applicant]
Ashish Kapoor et al., “Interactive Optimization for Steering Machine Classification”, Apr. 10, 2010 (Year: 2010). [cited by examiner]
Marco Tulio Ribeiro et al., “‘Why Should I Trust You?’ Explaining the Predictions of Any Classifier”, Feb. 16, 2016 (Year: 2016). [cited by examiner]
Anonymous et al., “A visual introduction to machine learning”, R2D3, United States, Accessed on Jul. 31, 2020, 2 Pages. [cited by applicant]
Anonymous et al., “A visual introduction to machine learning—Part II, Model Tuning and the Bias-Variance Tradeoff”, R2D3, United States, Accessed on Jul. 31, 2020, 1 Page. [cited by applicant]
Anonymous et al., “BLOKK is a font for quick mock-ups and wireframing for clients who do not understand latin”, Blokk Font, Accessed on Jul. 31, 2020, 1 Page. [cited by applicant]
Anonymous et al., “Global Warming”, Contropedia, Revision 893815787, Apr. 23, 2019, 22 Pages. [cited by applicant]
Anonymous et al., “Introducing the Create ML App”, Apple Developer, WWDC, 2019, 2 Pages. [cited by applicant]
Anonymous et al., “Recognizing hand-written digits”, Scikit-learn Developers, 2007-2009, 2 Pages. [cited by applicant]
Anonymous et al., “The Preservation of Favoured Traces”, Fathom Information Design, Accessed on Jul. 31, 2020, 5 Pages. [cited by applicant]
Anonymous et al., “Visually probe the behavior of trained machine learning models, with minimal coding”, What-If Tool, GitHub, Inc., Accessed on Jul. 31, 2020, 1 Page. [cited by applicant]
Anonymous et al., “Word-mesh”, GitHub, Inc., Accessed on Jul. 31, 2020, 4 Pages. [cited by applicant]
Authors et. al. Disclosed Anonymously, “Automatic analysis and remediation of dynamic perturbations in resolver classes for retrospective learning and assignment of email tickets”, An IP.com Prior Art Database Technical… [cited by applicant]
Distill, Accessed on Jul. 31, 2020, 4 Pages. [cited by applicant]
Hu et al., “Method and System for Gender Classification of Names using Character Based Machine Learning Models”, An IP.com Prior Art Database Technical Disclosure, IP.com No. IPCOM000259835D, IP.com Electronic Publicati… [cited by applicant]
Kessler et al., “Scattertext 0.0.2.67”, GitHub, Inc., Accessed on Jul. 31, 2020, 40 Pages. [cited by applicant]
Krause et al., “Using Visual Analytics to Interpret Predictive Machine Learning Models”, 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY, USA, arXiv:1606.05685v2, [stat.ML], Jun… [cited by applicant]
Kukkar et al., “A Novel Deep-Learning-Based Bug Severity Classification Technique Using Convolutional Neural Networks and Random Forest with Boosting”, Sensors 2019, 19, 2964; doi:10.3390/s19132964, Jul. 5, 2019, 22 Pag… [cited by applicant]
Lundberg et al., “SHAP (SHapley Additive exPlanations)”, GitHub, Inc., Accessed on Jul. 31, 2020, 7 Pages. [cited by applicant]
Panickar et al., “Method and System for Classifying Large Data Using Machine Learning Algorithms”, An IP.com Prior Art Database Technical Disclosure, IP.com No. IPCOM000233324D, IP.com Electronic Publication Date: Dec. … [cited by applicant]
Posavec, Stefanie, “Writing Without Words”, Accessed on Jul. 31, 2020, 1 Page. [cited by applicant]
Powell, Victor, “CSV Fingerprints”, Setosa Blog, Aug. 3, 2014, 2 Pages. [cited by applicant]
Ribeiro, Marco Tulio Correia et al., “Lime”, GitHub, Inc., Accessed on Jul. 31, 2020, 3 Pages. [cited by applicant]
Schapire, Rob, “Machine Learning Algorithms for Classification”, Princeton University, United States, 2017, 92 Pages. [cited by applicant]
Turesson et al., “Machine Learning Algorithms for Automatic Classification of Marmoset Vocalizations”, CrossMark, PLoS One 11(9):e0163041, doi:10.1371/journal.pone.0163041, Sep. 21, 2016, 14 Pages. [cited by applicant]
Yau, Nathan, “What Makes People the Most Happy”, FlowingData, Jun. 21, 2018, 6 Pages. [cited by applicant]