Granted Patent B2

US 12,223,063 · App. 18/739,111 · Granted Feb 11, 2025

Inventors: James Myers (New York, NY); William Franklin Cameron (Jacksonville, FL); Miriam Silver (Tel Aviv, IL); Prithvi Narayana Rao (Allen, TX); Pramod Goyal (Ahmedabad, IN); Manjit Rajaretnam (Irving, TX)

Assignee: CITIBANK, N.A.

G06F21/577G06F21/552

Abstract

Claims (114)

1. A method for grading a pre-trained Artificial Intelligence (AI) model via a graphical user interface (GUI), the method comprising:

obtaining a set of application domains of the pre-trained AI model in which a pre-trained AI model will be used,

wherein the pre-trained AI model is configured to generate, in response to a received input, a response;

using the set of application domains, determining a set of guidelines defining one or more operation boundaries of the pre-trained AI model by mapping each application domain of the set of application domains to one or more guidelines of the set of guidelines;

generating a set of test categories associated with the one or more guidelines of the set of guidelines,

wherein each of the set of test categories includes a set of benchmarks,

wherein each benchmark in the set of benchmarks is configured to indicate a degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with a corresponding test category

wherein the set of test categories include at least two of: quality of training data of the pre-trained AI model, security measures of the pre-trained AI model, software development practices of the pre-trained AI model, satisfaction with regulations of the pre-trained AI model, and explainability of the response of the pre-trained AI model;

for each test categories in the set of test categories, constructing a set of tests,

wherein each test comprises: (1) a prompt and (2) an expected response,

wherein each test is configured to test the degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with the corresponding test category;

for each test of the sets of tests, obtaining, from the pre-trained AI model, a set of case-specific responses by:

transmitting the prompt of the test into one or more nodes of an input layer of the pre-trained AI model, wherein the one or more nodes are associated with the corresponding test category, and

responsive to transmitting the prompt, receiving, from an output layer of the pre-trained AI model, a case-specific response;

using the obtained sets of case-specific responses, assigning a grade, for each test category, to the pre-trained AI model in accordance with the set of benchmarks for the corresponding test category by, for each test:

comparing the expected response of the test to the case-specific response received from the pre-trained AI model;

using the assigned grades, mapping the assigned grades for each test category to a particular degree of satisfaction corresponding to one or more application domains of the pre-trained AI model; and

generating for display at the GUI, a graphical layout indicating application-domain-specific grades, wherein the graphical layout includes a first graphical representation of each application domain of the pre-trained AI model and a second graphical representation of corresponding application-domain-specific grades.

2. The method of claim 1 , wherein the set of tests is a first set of tests, wherein the prompt is a first prompt, wherein the expected response is a first expected response, the method further comprising:

generating, for each of the test categories in the set of test categories, a second set of tests,

wherein each test in the second set of tests comprises: (1) a second prompt and (2) a second expected response,

wherein the second prompt is different from the first prompt,

wherein the second expected response is different from the first expected response, and

wherein each test in the second set of tests is configured to test the degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with the corresponding test category.

3. The method of claim 1 , further comprising:

determining the set of guidelines, via a machine learning (ML) model, using one or more of:

a location of the pre-trained AI model,

a use case of the pre-trained AI model, or

data sources used in the pre-trained AI model.

4. The method of claim 1 ,

wherein one or more of the test categories within the set of test categories relates to the quality of training data of the pre-trained AI model, and

wherein a corresponding set of tests relates to one or more of:

a presence of bias within the training data,

a presence of structured metadata in the training data, or

a presence of outliers in the training data.

5. The method of claim 1 , wherein the set of guidelines include one or more of: governmental regulations of a specific jurisdiction, organization-specific regulations, or AI application type-specific guidelines.

6. The method of claim 1 ,

wherein one or more of the test categories within the set of test categories relates to the security measures of the pre-trained AI model, and

wherein corresponding set of tests relates to one or more of:

data encryption in the pre-trained AI model,

access controls of the pre-trained AI model,

vulnerability management of the pre-trained AI model,

threat detection of the pre-trained AI model, or

remediation actions of the pre-trained AI model.

7. The method of claim 1 ,

wherein the case-specific response of the pre-trained AI model includes a case-specific outcome and a case-specific explanation of how the case-specific outcome was determined,

wherein the expected response of each test includes an expected outcome and an expected explanation of how the expected outcome was determined, and

wherein comparing the expected response of a particular test to the case-specific response received from the pre-trained AI model includes:

comparing the expected outcome of the particular test to the case-specific outcome received from the pre-trained AI model, and

responsive to the expected outcome of the particular test satisfying the case-specific outcome received from the pre-trained AI model, comparing the expected explanation of the particular test to a corresponding case-specific explanation of the case-specific outcome.

8. A non-transitory, computer-readable storage medium storing instructions for grading a pre-trained Artificial Intelligence (AI) model, wherein the instructions when executed by at least one data processor of a system, cause the system to:

obtain a set of application domains of the pre-trained AI model in which a pre-trained AI model will be used,

wherein the pre-trained AI model is configured to generate, in response to a received input, a response;

using the set of application domains, determine a set of guidelines defining one or more operation boundaries of the pre-trained AI model by mapping each application domain of the set of application domains to one or more guidelines of the set of guidelines;

accessing a set of assessment domains associated with the one or more guidelines of the set of guidelines,

wherein each of the set of assessment domains includes a set of benchmarks,

wherein each benchmark in the set of benchmarks is configured to indicate a degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with corresponding assessment domains;

for each assessment domains in the set of assessment domains, construct a set of assessments,

wherein each assessment is configured to test the degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with a corresponding assessment domain;

evaluate the pre-trained AI model against the set of assessments to determine the degree of satisfaction of the pre-trained AI model with the set of guidelines for the corresponding assessment domain by transmitting a command indicating one or more assessments into an input layer of the AI model;

using the evaluation, assign a grade, for each assessment domain, to the pre-trained AI model in accordance with the set of benchmarks for the corresponding assessment domain by;

using the assigned grades, map the assigned grades for each assessment domain to a particular degree of satisfaction corresponding to one or more application domains of the pre-trained AI model; and

generate a representation indicating application-domain-specific grades, wherein the representation includes each application domain of the pre-trained AI model and corresponding application-domain-specific grades.

9. The computer-readable storage medium of claim 8 , wherein one or more of the assigned grades for each assessment domain includes one or more of:

a binary indicator of a presence of adherence of the pre-trained AI model with the set of guidelines for the corresponding assessment domain,

a category indicating a corresponding assigned grade, or

a probability indicating the corresponding assigned grade.

10. The computer-readable storage medium of claim 8 , wherein the instructions further cause the system to:

receive a subset of application domains within the set of application domains; and

present a subset of the application-domain-specific grades using a particular view scope,

wherein the particular view scope filters the set of application-domain-specific grades using the subset of application domains.

11. The computer-readable storage medium of claim 8 , wherein the instructions further cause the system to:

receive an indicator of a type of application associated with the pre-trained AI model;

identify a relevant set of assessment domains associated with the type of the application defining the one or more operation boundaries of the pre-trained AI model; and

obtain the relevant set of assessment domains, via an Application Programming Interface (API).

12. The computer-readable storage medium of claim 8 ,

wherein the set of assessments of a particular assessment domain constructed by the pre-trained AI model includes a set of seed assessments,

wherein subsequently assessments of the set of assessments constructed subsequent to the set of seed assessments are dynamically generated using the degree of satisfaction of the pre-trained AI model with the one or more guidelines associated with the set of seed assessments.

13. The computer-readable storage medium of claim 8 , wherein the instructions further cause the system to:

using the set of application-domain-specific grades, generate a set of actions configured to adjust the set of application-domain-specific grades to a desired set of application-domain-specific grades.

14. The non-transitory, computer-readable storage medium of claim 8 , wherein the instructions further cause the system to:

obtain a new set of guidelines;

identify one or more new assessment domains associated with the new set of guidelines; and

iteratively update the set of assessment domains by adding the one or more new assessment domains to the set of assessment domains.

15. A system for grading an Artificial Intelligence (AI) model, comprising:

at least one processor; and

one or more non-transitory computer-readable media storing instructions, which when executed by at least one processor, perform operations comprising:

obtaining a set of application domains of the AI model in which an AI model will be used,

wherein the AI model is configured to generate, in response to a received input, a response;

using the set of application domains, determining a set of guidelines defining one or more operation boundaries of the AI model by mapping each application domain of the set of application domains to one or more guidelines of the set of guidelines;

accessing a set of assessment domains associated with the one or more guidelines of the set of guidelines,

wherein each of the set of assessment domains includes a set of benchmarks,

wherein each benchmark in the set of benchmarks is configured to indicate a degree of satisfaction of the AI model with the one or more guidelines associated with corresponding assessment domains;

for each assessment domains in the set of assessment domains, constructing a set of assessments,

wherein each assessment is configured to test the degree of satisfaction of the AI model with the one or more guidelines associated with a corresponding assessment domain;

evaluating the AI model against the set of assessments to determine the degree of satisfaction of the AI model with the set of guidelines for the corresponding assessment domain;

using the evaluation, assigning a grade, for each assessment domain, to the AI model in accordance with the set of benchmarks for the corresponding assessment domain by;

using the assigned grades, mapping the assigned grades for each assessment domain to a particular degree of satisfaction corresponding to one or more application domains of the AI model;

generating a representation indicating application-domain-specific grades, wherein the representation includes each application domain of the AI model and corresponding application-domain-specific grades; and

using the application-domain-specific grades, automatically generating a set of actions to adjust one or more parameters of the AI model to increase a degree of satisfaction of the AI model with corresponding operative boundaries of one or more application domains.

16. The system of claim 15 , the operations further comprising:

weighing the assigned grades of each assessment domain within the set of assessment domains of the AI model based on predetermined weights corresponding with each assessment domains,

wherein the set of application-domain-specific grades includes an overall score in accordance with the weighted application-domain-specific grades of each assessment domains.

17. The system of claim 15 , the operations further comprising:

generating confidence scores for each assigned grade,

wherein the confidence scores are configured to represent a reliability of the assigned grade.

18. The system of claim 15 , wherein evaluating the AI model against the set of assessments further causes the system to:

in response to reaching a non-compliance threshold indicating a low level of satisfaction of the AI model with the set of guidelines, prevent assigning additional grades to the assessment domains.

19. The system of claim 15 , wherein the application-domain-specific grades is a first set of application-domain-specific grades, the operations further comprising:

receiving an updated set of benchmarks for one or more of the set of assessment domains;

using the updated set of benchmarks, generating a second set of application-domain-specific grades of the AI model; and

responsive to the second set of application-domain-specific grades indicating a lower degree of satisfaction than the first set of application-domain-specific grades, updating a training data that the AI model is trained on to increase the degree of satisfaction of the AI model with the updated set of benchmarks.

20. The system of claim 15 , wherein the set of actions is a set of programmatic workflows, the operations further comprising:

executing the set of actions to generate a document configured to satisfy the one or more operation boundaries of the AI model.

Assignments (1)

Continuity (6)

References Cited (116)

Cited By (2)

US 12,524,412 US 12,621,253