IP Library › Granted Patent US 12,737,585
Granted Patent B2
US 12,737,585 · App. 18/300,699 · Granted Sep 15, 2026

Explainable classifications with abstention using client agnostic machine learning models

Inventors: Arun A. Ayachitula (Dobbs Ferry, NY); Rohit Khandekar (Jersey City, NJ); Upendra Sharma (Hartsdale, NY)
Assignee: Kyndryl, Inc.
G06N3/04G06N3/09
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,737,585
App. No.
18/300,699
Granted
Sep 15, 2026
Kind
B2
Abstract

Embodiments relate to providing explainable classifications with abstention using client agnostic machine learning models. A technique includes classifying, by a processor, a record with a label using a machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of an information technology (IT) domain. The processor generates an explanation of a decision by the machine learning model to classify the record with the label and displays the explanation in a human readable form.

Claims (44)

1 . A computer-implemented method comprising:

classifying, by a processor, a record with a label using a machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of an information technology (IT) domain that is associated with computer systems operating in an IT computing environment;

generating, by the processor, an explanation of a decision by the machine learning model to classify the record with the label associated with the computer systems operating in the IT computing environment;

displaying the explanation in a human readable form; and

preventing the given record from being input to an automated resolution system based on the given record being outside of the scope of the IT domain associated with the computer systems operating in the IT computing environment.

2 . The computer-implemented method of claim 1 , wherein the human readable form comprises a disjunctive normal form.

3 . The computer-implemented method of claim 1 , wherein the explanation of the decision by the machine learning model is based on a linear classifier formula utilized by the machine learning model.

4 . The computer-implemented method of claim 1 , wherein:

the explanation of the decision by the machine learning model is based on features and respective coefficients corresponding to the features, the features and the respective coefficients being derived from a linear classifier formula of the machine learning model; and

the features are extracted from text of the record.

5 . The computer-implemented method of claim 1 , wherein the human readable form comprises a display of pertinent positive features with respective contributions for each of the pertinent positive features to the decision by the machine learning model, the pertinent positive features being associated with the computer systems operating in the IT computing environment.

6 . The computer-implemented method of claim 1 , wherein the machine learning model is trained on training data in the IT domain.

7 . The computer-implemented method of claim 1 , wherein:

the machine learning model comprises a linear classifier algorithm; and

the record is a ticket of technical problems in an IT environment.

8 . A system comprising:

a memory having computer readable instructions; and

one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:

classifying a record with a label using a machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of an information technology (IT) domain that is associated with computer systems operating in an IT computing environment;

generating an explanation of a decision by the machine learning model to classify the record with the label associated with the computer systems operating in the IT computing environment;

displaying the explanation in a human readable form; and

preventing the given record from being input to an automated resolution system based on the given record being outside of the scope of the IT domain associated with the computer systems operating in the IT computing environment.

9 . The system of claim 8 , wherein the human readable form comprises a disjunctive normal form.

10 . The system of claim 8 , wherein the explanation of the decision by the machine learning model is based on a linear classifier formula utilized by the machine learning model.

11 . The system of claim 8 , wherein:

the explanation of the decision by the machine learning model is based on features and respective coefficients corresponding to the features, the features and the respective coefficients being derived from a linear classifier formula of the machine learning model; and

the features are extracted from text of the record.

12 . The system of claim 8 , wherein the human readable form comprises a display of pertinent positive features with a measure of respective contributions for each of the pertinent positive features to the decision by the machine learning model.

13 . The system of claim 8 , wherein the machine learning model is trained on training data in the IT domain.

14 . The system of claim 8 , wherein:

the machine learning model comprises a linear classifier algorithm; and

the record is a ticket of technical problems in an IT environment.

15 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

classifying a record with a label using a machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of an information technology (IT) domain that is associated with computer systems operating in an IT computing environment;

generating an explanation of a decision by the machine learning model to classify the record with the label associated with the computer systems operating in the IT computing environment;

displaying the explanation in a human readable form; and

preventing the given record from being input to an automated resolution system based on the given record being outside of the scope of the IT domain associated with the computer systems operating in the IT computing environment.

16 . The computer program product of claim 14 , wherein the human readable form comprises a disjunctive normal form.

17 . The computer program product of claim 15 , wherein the explanation of the decision by the machine learning model is based on a linear classifier formula utilized by the machine learning model.

18 . The computer program product of claim 15 , wherein:

the explanation of the decision by the machine learning model is based on features and respective coefficients corresponding to the features, the features and the respective coefficients being derived from a linear classifier formula of the machine learning model; and

the features are extracted from text of the record.

19 . The computer program product of claim 15 , wherein the human readable form comprises a display of pertinent positive features with a measure of respective contributions for each of the pertinent positive features to the decision by the machine learning model.

20 . The computer program product of claim 15 , wherein the machine learning model is trained on training data in the IT domain.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2023
From: AYACHITULA, ARUN A.; KHANDEKAR, ROHIT; SHARMA, UPENDRA
To: KYNDRYL, INC.
Reel/Frame 063327/0369 →
Continuity (1)
Related Publication 20240346283A1 · Oct 17, 2024
References Cited (60)
US 10438212B1 · Jilani · 2019 [cited by examiner]
US 10621550B2 · Carey et al. · 2020 [cited by applicant]
US 10824959B1 · Chatterjee et al. · 2020 [cited by applicant]
US 11270226B2 · Meng et al. · 2022 [cited by applicant]
US 11343146B1 · Mohanty · 2022 [cited by examiner]
US 11556843B2 · Backas et al. · 2023 [cited by applicant]
US 11568286B2 · Nourian · 2023 [cited by examiner]
US 11861493B2 · Vengertsev · 2024 [cited by examiner]
US 11886963B2 · Welsh · 2024 [cited by examiner]
US 11928182B1 · Arici et al. · 2024 [cited by applicant]
US 12265461B2 · Kozhaya · 2025 [cited by examiner]
US 20170032035A1 · Gao et al. · 2017 [cited by applicant]
US 20170178038A1 · Guven et al. · 2017 [cited by applicant]
US 20190132191A1 · Mann · 2019 [cited by examiner]
US 20190354810A1 · Samel · 2019 [cited by examiner]
US 20200184272A1 · Zhang · 2020 [cited by examiner]
US 20200193243A1 · Dhurandhar et al. · 2020 [cited by applicant]
US 20210049512A1 · Chatterjee et al. · 2021 [cited by applicant]
US 20210073627A1 · Sarferaz · 2021 [cited by examiner]
US 20210097433A1 · Olgiati · 2021 [cited by examiner]
US 20210133610A1 · Natesan Ramamurthy · 2021 [cited by examiner]
US 20210174168A1 · Dalli · 2021 [cited by examiner]
US 20210304057A1 · Kundu · 2021 [cited by examiner]
US 20210312336A1 · Sinn · 2021 [cited by examiner]
US 20220050918A1 · Macedo · 2022 [cited by examiner]
US 20220092469A1 · Bremer · 2022 [cited by examiner]
US 20220107858A1 · Jain et al. · 2022 [cited by applicant]
US 20220156134A1 · Lehmann et al. · 2022 [cited by applicant]
US 20220198295A1 · Singla · 2022 [cited by examiner]
US 20220245461A1 · Sern et al. · 2022 [cited by applicant]
US 20220335315A1 · Zonca · 2022 [cited by examiner]
US 20230222358A1 · Wu · 2023 [cited by examiner]
US 20230236960A1 · Dzhuraeva · 2023 [cited by examiner]
US 20230259807A1 · Rawat · 2023 [cited by examiner]
US 20230281518A1 · Verma · 2023 [cited by examiner]
US 20230316196A1 · Sewak et al. · 2023 [cited by applicant]
US 20230319099A1 · Karimibiuki · 2023 [cited by examiner]
US 20230368028A1 · Lupesko · 2023 [cited by examiner]
US 20230401829A1 · Liu · 2023 [cited by examiner]
US 20240169271A1 · Noppen · 2024 [cited by examiner]
US 20240212795A1 · Freer · 2024 [cited by examiner]
US 20240220579A1 · Mhatre · 2024 [cited by examiner]
US 20240255939A1 · Kanazawa · 2024 [cited by examiner]
US 20240345905A1 · Ayachitula · 2024 [cited by examiner]
US 20240346283A1 · Ayachitula · 2024 [cited by examiner]
US 20240414064A1 · Ayachitula · 2024 [cited by examiner]
EP 3447696A1 · 2019 [cited by applicant]
GB 2603279A · 2022 [cited by applicant]
JP 2018180759A · 2018 [cited by applicant]
JP 2019101495A · 2019 [cited by applicant]
JP 2020091836A · 2020 [cited by applicant]
WO 2022057425A1 · 2022 [cited by applicant]
Anonymous, “Natural language processing—Wikipedia,” URL: https://en.wikipedia.org/wiki/Natural_language_processing; Retrieved: Feb. 1, 2024; 13 pages. [cited by applicant]
International Search Report and Written Opinion; Application No. PCT/EP2023/071905; Filed: Aug. 8, 2023; Date of mailing: Feb. 6, 2024; 11 pages. [cited by applicant]
Ayachitula, et al. “Explainable Classifications With Abstention Using Client Agnostic Machine Learning Models,” U.S. Appl. No. 18/300,688, filed Apr. 14, 2023. [cited by applicant]
Kyndryl: List of Kyndryl Patents or Patent Applications Treated as Related (Appendix P); Date Filed: Apr. 17, 2023; 2 pages. [cited by applicant]
International Search Report and Written Opinion; Application No. PCT/EP2023/063772; Filed: May 23, 2023; Date of mailing; Jan. 15, 2024; 15 pages. [cited by applicant]
Chatterjee et al., “A Pipeline for Automating Labeling to Prediction in Classification of NFRs”, IEEE 29th International Requirements Engineering Conference, 2021, pp. 323-333. [cited by applicant]
Ayachitula, et al. “Self-Learning Automated Information Technology Change Risk Prediction,” U.S. Appl. No. 18/331,361, filed Jun. 9, 2023. [cited by applicant]
Kyndryl: List of Kyndryl Patents or Patent Applications Treated as Related (Appendix P); Date Filed: Jun. 9, 2023; 2 pages. [cited by applicant]