Granted Patent B2

US 12,737,643 · App. 17/931,479 · Granted Sep 15, 2026

Inventors: Aaron K. Baughman (Cary, NC); Jeremy R. Fox (Georgetown, TX); Zachary A. Silverstein (Georgetown, TX); Sarbajit K. Rakshit (Kolkata, IN)

Assignee: International Business Machines Corporation

G06N5/022

Abstract

Claims (55)

1 . A computer-implemented method comprising:

identifying a user experience (UX) component containing information that conveys a trustworthy artificial intelligence (AI) factor, wherein the UX component is part of a front-end UX of an application that utilizes an AI model, and the information is related to an output of the AI model, and the trustworthy AI factor measures an aspect of trust in the output of the AI model;

evaluating the information contained in the UX component to determine a trust score for the UX component that indicates a degree to which the information contained in the UX component conveys the trustworthy AI factor;

determining based on the trust score for the UX component that the information contained in the UX component does not meet a threshold of disclosure of the trustworthy AI factor;

obtaining an alternative UX component containing additional information that meets the threshold of disclosure of the trustworthy AI factor; and

providing the alternative UX component for incorporation into the front-end UX of the application.

2 . The computer-implemented method of claim 1 , wherein identifying the UX component containing the information that conveys the trustworthy AI factor further comprises analyzing the front-end UX of the application using one or more machine learning techniques to identify the UX component which has a confidence interval representation of the trustworthy AI factor.

3 . The computer-implemented method of claim 1 , wherein the trustworthy AI factor is an accuracy trustworthy AI factor and evaluating the UX component further comprises analyzing a calibration of the AI model based on a reliability score generated for the AI model to determine an accuracy of the output of the AI model.

4 . The computer-implemented method of claim 1 , wherein the trustworthy AI factor is a transparency trustworthy AI factor and evaluating the UX component further comprises using predictor variables of the AI model to determine a degree to which transparency of the AI model is disclosed in the UX component.

5 . The computer-implemented method of claim 1 , further comprising analyzing source code of the AI model to determine a semantic trust score that provides a confidence level for semantic interpretation of terms in the UX component.

6 . The computer-implemented method of claim 1 , further comprising:

performing the steps of identifying the UX component and evaluating the information contained in the UX component for each of a plurality of trustworthy AI factors selected from a group consisting of: accuracy, explainability, transparency, and fairness; and

generating an overall trust score to indicate a degree to which the information in the front-end UX discloses the plurality of trustworthy AI factors.

7 . The computer-implemented method of claim 6 , further comprising:

performing a bin packing technique to increase the overall trust score of the information in the front-end UX to disclose the plurality of trustworthy AI factors and to decrease a number of UX components included in the front-end UX.

8 . The computer-implemented method of claim 7 , further comprising:

assigning weights to alternative UX components based at least in part on user-preferences, wherein the weights are considered when performing the bin packing technique.

9 . The computer-implemented method of claim 1 , further comprising:

receiving, at a discoverable service endpoint, a request to determine the trust score for the UX component; and

providing the trust score for the UX component in response to the request.

10 . The computer-implemented method of claim 9 , wherein the request to determine the trust score for the UX component is generated within an integrated development environment (IDE) in relation to development of the application that utilizes the AI model.

11 . A system, comprising:

one or more computer readable storage media storing program instructions and one or more processors which, in response to executing the program instructions, are configured to:

identify user experience (UX) components containing information, which is related to an output of an artificial intelligence (AI) model, that conveys one of a plurality of trustworthy AI factors, wherein the UX components are part of a front-end UX of an application that utilizes the AI model, and wherein the plurality of trustworthy AI factors measure different aspects of trust in the output by the AI model;

evaluate each of the UX components to determine trust scores for the UX components, wherein the trust scores indicate a degree to which the information contained in a UX component conveys a respective trustworthy AI factor;

determine based on a trust score for a UX component included in the UX components being evaluated that information contained in the UX component does not meet a threshold of disclosure of a trustworthy AI factor;

obtain an alternative UX component containing additional information that meets the threshold of disclosure of the trustworthy AI factor; and

provide the alternative UX component for incorporation into the front-end UX of the application.

12 . The system of claim 11 , wherein the program instructions configured to cause the one or more processors to identify the UX component further cause the one or more processors to:

analyze the front-end UX of the application using one or more machine learning techniques to identify the UX component as having a confidence interval representation of the trustworthy AI factor.

13 . The system of claim 11 , wherein the program instructions are further configured to cause the one or more processors to:

analyze a calibration of the AI model based on a reliability score generated for the AI model to determine an accuracy of the output of the AI model.

14 . The system of claim 11 , wherein the program instructions are further configured to cause the one or more processors to:

obtain predictor variables of the AI model; and

evaluate the UX component using the predictor variables of the AI model to determine a degree to which transparency of the AI model is disclosed in the UX component.

15 . The system of claim 11 , wherein the program instructions are further configured to cause the one or more processors to:

generate an overall trust score to indicate a degree to which the information in the front-end UX discloses the plurality of trustworthy AI factors; and

performing a bin packing technique to increase the overall trust score of the information in the front-end UX to disclose the plurality of trustworthy AI factors and to decrease a number of UX components included in the front-end UX.

16 . A computer program product, comprising:

one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions configured to cause one or more processors to:

identify user experience (UX) components containing information, which is related to an output of an artificial intelligence (AI) model, that conveys one of a plurality of trustworthy AI factors, wherein the UX components are part of a front-end UX of an application that utilizes the AI model, and wherein the plurality of trustworthy AI factors measure different aspects of trust in the output of the AI model;

evaluate each of the UX components to determine trust scores for the UX components, wherein the trust scores indicate a degree to which the information contained in a UX component conveys a respective trustworthy AI factor;

determine based on a trust score for a UX component included in the UX components being evaluated that information contained in the UX component does not meet a threshold of disclosure of a trustworthy AI factor;

obtain an alternative UX component containing additional information that meets the threshold of disclosure of the trustworthy AI factor; and

provide the alternative UX component for incorporation into the front-end UX of the application.

17 . The computer program product of claim 16 , wherein the program instructions configured to cause the one or more processors to identify the UX component further cause the one or more processors to:

analyze the front-end UX of the application using one or more machine learning techniques to identify the UX component as having a confidence interval representation of the trustworthy AI factor.

18 . The computer program product of claim 16 , wherein the program instructions are further configured to cause the one or more processors to:

analyze a calibration of the AI model based on a reliability score generated for the AI model to determine an accuracy of the output of the AI model.

19 . The computer program product of claim 16 , wherein the program instructions are further configured to cause the one or more processors to:

obtain predictor variables of the AI model; and

evaluate the UX component using the predictor variables of the AI model to determine a degree to which transparency of the AI model is disclosed in the UX component.

20 . The computer program product of claim 16 , wherein the program instructions are further configured to cause the one or more processors to:

generate an overall trust score to indicate a degree to which the information in the front-end UX discloses the plurality of trustworthy AI factors; and

performing a bin packing technique to increase the overall trust score of the information in the front-end UX to disclose the plurality of trustworthy AI factors and to decrease a number of UX components included in the front-end UX.

Assignments (1)

Continuity (1)

References Cited (28)