Granted Patent B2

US 12,591,795 · App. 17/533,285 · Granted Mar 31, 2026

Inventors: Young Jun Kim (Seoul, KR); Jae Sun Shin (Seoul, KR)

Assignee: SAMSUNG SDS CO., LTD.

G06N5/045G06F18/2413G06F18/28

Abstract

Claims (52)

1 . A method performed on a computing device for providing explainable artificial intelligence (XAI), the method comprising:

receive, from a user terminal, a rationale data request;

inputting, based on the rationale data request, an entire dataset into a machine-learned model;

configuring a first set of prototypes representing each instance of data instances of the entire dataset input to the machine-learned model;

configuring a second set of criticisms that are samples of other data instances not expressed by the first set of prototypes from among the data instances of the entire dataset;

calculating a first feature based rationale of output data for a first data instance of the machine-learned model by considering all prototypes included in the first set of prototypes and all criticisms included in the second set of criticisms;

generating determination result content for the first data instance by using the output data for the first data instance by the machine-learned model;

generating rationale content for the determination result content by using the first feature based rationale of the output data, wherein the rationale content indicates at least one top feature having a first influence on the determination result content and at least one low feature having a second influence on the determination result content lower than the first influence, the at least one top feature being selected from a top feature of a list of first feature based rationales in descending order of rationale values of all features of the entire dataset, and the at least one low feature being selected from a bottom feature of the list of first feature based rationales in descending order of rationale values of all features of the entire dataset;

generating output data to be output in an output screen of the user terminal, the output data including the determination result content and the rationale content; and

providing, based on the rationale data request, the output data to the user terminal.

2 . The method of claim 1 , wherein the configuring of the first set of prototypes comprises:

configuring the first set of prototypes to include a first number of prototypes corresponding to a first ratio of the first number of prototypes to a third number of the data instances of the entire dataset; and

wherein the configuring of the second set of criticisms comprises:

repeating adding a criticism to the second set of criticisms until a second number of criticisms in the second set of criticisms corresponds to at least one of a second ratio of the first number of the prototypes in the first set of prototypes to the second number of criticisms of the second set of criticisms, or an evaluation value of the second set of criticisms calculated using a kernel matrix of the entire dataset is less than or equal to a threshold.

3 . The method of claim 1 , wherein the calculating of the first feature based rationale comprises:

calculating a relative distance between data points that all the prototypes of the first set of prototypes and all the criticisms of the second set of criticisms form on a feature space, and a data point of a modified data instance, in which a value of a first feature among features of the first data instance is increased by a modified distance;

calculating a difference between the output data for the first data instance of the machine-learned model and output data for the modified data instance of the machine-learned model;

calculating an evaluation value for the modified data instance by aggregating the relative distance and the difference;

calculating a plurality of evaluation values by repeating calculating the relative distance while changing the modified distance, calculating the difference, and calculating the evaluation value; and

calculating the first influence of the first feature on the output data for the first data instance of the machine-learned model by aggregating the plurality of evaluation values.

4 . The method of claim 3 , wherein the calculating of the first feature based rationale further comprises:

calculating the first feature based rationale by scaling the first influence of the first feature using influence of all features.

5 . The method of claim 3 , wherein the calculating of the plurality of evaluation values comprises:

calculating the plurality of evaluation values by repeating the calculating of the relative distance while changing the modified distance in a range between (MIN−(the value of the first feature of the first data instance))/(MAX−MIN) and (MAX−(the value of the first feature of the first data instance))/(MAX−MIN), the calculating of the difference, and the calculating of the evaluation value,

wherein the MAX is a maximum value of values of the first feature of the data instances of the entire dataset, and the MIN is a minimum value of values of the first feature of the data instances of the entire dataset.

6 . The method of claim 3 , wherein the calculating of the evaluation value for the modified data instance comprises:

calculating an adjusted evaluation value by adjusting the calculated evaluation value so that the calculated evaluation value decreases as the modified distance increases; and

wherein the calculating of the plurality of evaluation values comprises:

calculating a plurality of adjusted evaluation values by repeating the calculating of the relative distance while changing the modified distance, the calculating of the difference, and the calculating of the adjusted evaluation value.

7 . The method of claim 1 , wherein the calculating of the first feature based rationale comprises:

calculating a relative distance between data points that all the prototypes of the first set of prototypes and all the criticisms of the second set of criticisms form on a feature space, and a data point of a modified data instance, in which a value of a first feature among features of the first data instance is increased by a modified distance;

calculating a difference between the output data for the first data instance of the machine-learned model and output data for the modified data instance of the machine-learned model; and

calculating an integral value for the modified distance of an evaluation value for the modified data instance calculated using the relative distance and the difference as the first influence of the first feature on the output data for the first data instance of the machine-learned model.

8 . The method of claim 7 , wherein the calculating of the integral value for the modified distance comprises:

calculating the integral value for the modified distance of the evaluation value for the modified data instance in a range between (MIN−(the value of the first feature of the first data instance))/(MAX−MIN) and (MAX−(the value of the first feature of the first data instance)/(MAX−MIN), and

wherein the MAX is a maximum value of values of the first feature of the data instances of the entire dataset, and the MIN is a minimum value of values of the first feature of the data instances of the entire dataset.

9 . The method of claim 7 , wherein the calculating of the integral value for the modified distance comprises:

calculating the integral value for the modified distance of an adjusted evaluation value adjusted so that the evaluation value for the modified data instance decreases as the modified distance increases.

10 . An apparatus for providing explainable artificial intelligence, the apparatus comprising:

one or more processors;

a memory for loading a computer program executed by the one or more processors and data of a machine-learned model; and

a storage for storing the computer program;

wherein the computer program, when executed by the one or more processors, cause the apparatus to:

receive, from a user terminal, a rationale data request;

input, based on the rationale data request, an entire dataset into the machined-learned model;

configure a first set of prototypes representing each instance of data instances of the entire dataset input to the machine-learned model;

configure a second set of criticisms that are samples of other data instances not expressed by the first set of prototypes from among the data instances of the entire dataset;

calculate a first feature based rationale for output data for a first data instance of the machine-learned model by considering all prototypes included in the first set of prototypes and all criticisms included in the second set of criticisms;

generate determination result content for the first data instance by using the output data for the first data instance by the machine-learned model;

generate rationale content for the determination result content by using the first feature based rationale of the output data, wherein the rationale content indicates at least one top feature having a first influence on the determination result content exceeding a first threshold and at least one low feature having a second influence on the determination result content lower than the first influence, the at least one top feature being selected from a top feature of a list of first feature based rationales in descending order of rationale values of all features of the entire dataset, and the at least one low feature being selected from a bottom feature of the list of first feature based rationales in descending order of rationale values of all features of the entire dataset;

generate output data to be output in an output screen of the user terminal, the output data including the determination result content and the rationale content; and

provide, based on the rationale data request, the output data to the user terminal.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (27)