Granted Patent B1

US 12,361,334 · App. 19/015,646 · Granted Jul 15, 2025

Inventors: Vishal Mysore (Mississauga, CA); Ramkumar Ayyadurai (Jersey City, NJ); Chamindra Desilva (London, GB)

Assignee: CITIBANK, N.A.

G06N20/20

Abstract

Claims (92)

1. A non-transitory, computer-readable storage medium storing instructions for evaluating responses generated by one or more artificial intelligence (AI) models, wherein the instructions when executed by at least one data processor of a system, cause the system to:

train a machine learning (ML) model on a training dataset including a set of attributes to, in response to an input, generate an output that identifies a presence of one or more certain patterns of the set of attributes in the input,

wherein each certain pattern represents a disproportionate association of one or more attributes of the set of attributes within the input;

for each certain pattern, using the trained ML model, construct a set of validation actions for a set of responses generated by an AI model,

wherein each validation action includes: (1) a test command set, (2) an expected result, and (3) an expected set of descriptors associated with an expected series of steps to generate the expected result, and

wherein the set of validation actions is constructed by:

(1) determining a set of specific use cases associated with a set of guidelines defining a set of operative boundaries of the AI model, and

(2) mapping the set of specific use cases to the set of validation actions;

using the trained ML model, execute each set of validation actions on the AI model to determine the presence of one or more certain patterns within the set of responses generated by the AI model by:

inputting, into the AI model, the test command set of each validation action to receive a first test output comprising (1) a set of test results and (2) a set of test descriptors associated with a second series of steps to generate the set of test results,

comparing, for each validation action, (1) the set of test results and (2) the set of test descriptors of the AI model with (1) the expected result and (2) the expected set of descriptors of a corresponding validation action, respectively, and

aggregating the comparisons of each validation action to generate a result indicating the presence of one or more certain patterns within the first test output generated by the AI model;

using the result, generate a set of corrective actions to remove a portion of the first test output generated by the AI model indicated by the one or more certain patterns;

presenting a representation including one or more of: a graphical user interface component or a set of text via a computing device, wherein the representation indicates at least one of: the result or the set of corrective actions;

responsive to a user input received via the computing device, automatically executing the set of corrective actions to remove the portion of the first test output generated by the AI model; and

transmit the test command set of one or more validation actions into one or more nodes of an input layer of the AI model to validate an absence of the one or more certain patterns within a second test output generated by the AI model.

2. The non-transitory, computer-readable storage medium of claim 1 ,

wherein the output is a first output, and

wherein the ML model is further trained to generate a second output including a confidence score associated with a likelihood of the presence of the one or more certain patterns within the set of responses generated by the AI model.

3. The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

identify one or more new patterns within at least one of: the set of test results or the set of test descriptors of the AI model, and

iteratively update the set of validation actions based on the one or more new patterns.

4. The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

receive an indicator of a type of application associated with the AI model, identify a relevant set of attributes associated with the type of the application defining one or more operation boundaries of the AI model, and

obtain the relevant set of attributes, via an Application Programming Interface (API).

5. The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

wherein the set of corrective actions include one or more of: adjusting parameters of the AI model or updating training data of the AI model to remove the portion of the response from the AI model indicated by the one or more certain patterns.

6. The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

evaluate the set of attributes by analyzing one or more of:

proximate locations of alphanumeric characters within the set of responses, frequency of alphanumeric characters within the set of responses, or associations between alphanumeric characters within the set of responses.

7. The non-transitory, computer-readable storage medium of claim 6 , wherein the instructions further cause the system to:

segment the alphanumeric characters of the set of responses into a set of tokens;

normalize the set of tokens by removing one or more of: suffixes or prefixes of words within the alphanumeric characters; and

using the normalized set of tokens to generate the one or more certain patterns.

8. A computing system comprising:

at least one processor; and

one or more non-transitory computer-readable media storing instructions, which when executed by the at least one processor, perform operations comprising:

for each certain pattern of a set of certain patterns, constructing a set of validation actions for a set of responses generated by an AI model,

wherein each validation action includes: (1) a test command set, (2) an expected result, and (3) an expected set of descriptors associated with an expected series of steps to generate the expected result,

wherein the set of validation actions is constructed by:

(1) determining a set of specific use cases associated with a set of guidelines defining a set of operative boundaries of the AI model, and

(2) mapping the set of specific use cases to the set of validation actions, and

wherein each certain pattern represents a disproportionate association of one or more attributes of a set of attributes within the set of responses;

using a trained ML model, executing one or more sets of validation actions on the AI model to determine a presence of one or more certain patterns within the set of responses generated by the AI model by:

inputting, into the AI model, the test command set of the one or more sets of validation actions to receive a first test output comprising (1) a set of test results and (2) a set of test descriptors associated with a second series of steps to generate the set of test results,

comparing, for the one or more sets of validation actions, (1) the set of test results and (2) the set of test descriptors of the AI model with (1) the expected result and (2) the expected set of descriptors of a corresponding validation action, respectively, and

aggregating the comparisons of the one or more sets of validation actions to generate a result indicating the presence of the one or more certain patterns within the first test output generated by the AI model; and

responsive to the result exceeding a predefined threshold:

using the result, generating a set of corrective actions to remove a portion of the first test output generated by the AI model indicated by the one or more certain patterns,

displaying, at a graphical user interface (GUI), a graphical layout including (1) a first graphical representation indicating the result and (2) a second graphical representation indicating the set of corrective actions,

responsive to receiving a user input, automatically executing the set of corrective actions to remove the portion of the first test output generated by the AI model, and

transmit the test command set of one or more validation actions into one or more nodes of an input layer of the AI model to validate an absence of the one or more certain patterns within a second test output generated by the AI model.

9. The system of claim 8 , wherein the operations further comprise:

comparing, for one or more sets of validation actions, the expected result of the one or more sets of validation actions to a corresponding set of test results received from the AI model; and

responsive to the expected result of the one or more sets of validation actions satisfying the corresponding set of test results received from the AI model, comparing the expected set of descriptors of the one or more sets of validation actions to a corresponding set of test descriptors of the corresponding set of test results.

10. The system of claim 8 , wherein the operations further comprise:

receiving a set of feedback from a user on the result indicating the presence of one or more certain patterns within the set of responses generated by the AI model; and

adjusting operational parameters of the trained ML model using the set of feedback.

11. The system of claim 8 , wherein the operations further comprise:

classifying the one or more certain patterns into categories using a set of predefined criteria; and

assigning a corresponding set of validation actions to each category of certain patterns.

12. The system of claim 8 , wherein modifying the AI model further comprises:

updating training data of the AI model to remove a portion of the first test output generated by the AI model indicated by the one or more certain patterns.

13. The system of claim 8 ,

wherein the graphical layout includes a third graphical representation indicating the constructed one or more sets set of validation actions.

14. The system of claim 13 , wherein the operations further comprise:

responsive to a received user input, automatically executing the generated set of corrective actions to remove the portion of the set of responses generated by the AI model.

15. A computer-implemented method for evaluating responses generated by one or more AI models, the method comprising:

for each certain pattern of a set of certain patterns, constructing a set of validation actions for a set of responses generated by a second AI model, wherein one or more validation actions include: (1) a test command set, (2) an expected result, and (3) an expected set of descriptors associated with an expected series of steps to generate the expected result,

wherein each certain pattern represents a disproportionate association of one or more attributes of a set of attributes within the set of responses;

using a trained first AI model, executing one or more sets of validation actions on the second AI model to determine a presence of one or more certain patterns within the set of responses generated by the second AI model by:

inputting, into the second AI model, the test command set of the one or more sets of validation actions to receive a first test output comprising (1) a set of test results and (2) a set of test descriptors associated with a second series of steps to generate the set of test results,

comparing, for the sets of one or more validation actions, (1) the set of test results and (2) the set of test descriptors of the second AI model with (1) the expected result and (2) the expected set of descriptors of a corresponding validation action, respectively, and

aggregating the comparisons of the sets of one or more validation actions to generate a result indicating the presence of one or more certain patterns within the first test output generated by the second AI model;

transmitting, via a computing device, a representation indicating the result;

responsive to an input, trigger execution of a set of actions to modify the first test output generated by the second AI model; and

transmit the test command set of the one or more sets of validation actions into one or more nodes of an input layer of an second AI model to validate an absence of the one or more certain patterns within a second test output generated by the second AI model.

16. The computer-implemented method of claim 15 , further comprising:

displaying, on a graphical user interface (GUI), a graphical layout including (1) a first graphical representation indicating the result and (2) a second graphical representation indicating the constructed set of validation actions.

17. The computer-implemented method of claim 16 , further comprising:

responsive to a received user input on a graphical user interface (GUI) the GUI, automatically executing the constructed set of validation actions on the second AI model.

18. The computer-implemented method of claim 15 , further comprising:

modifying the second AI model by adjusting one or more parameters of the second AI model,

wherein the modified second AI model is trained to, in response to the test command set, generate, using the adjusted one or more parameters, a second set of responses without the presence of the one or more certain patterns.

19. The computer-implemented method of claim 15 , wherein the expected series of steps is a first series of steps, further comprising:

evaluating the second AI model against the set of validation actions by applying one or more validation actions in the set of validation actions to the second AI model by, for each particular validation action of the one or more validation actions:

supplying the test command set of the particular validation action into the second AI model,

responsive to inputting the test command set, receiving, from the second AI model, a test result and a corresponding test set of descriptors associated with a second series of steps to generate the test result,

comparing the expected result of the particular validation action to the test result received from the second AI model, and

responsive to the expected result of the particular validation action satisfying the test result received from the second AI model, comparing the expected set of descriptors of the particular validation action to the corresponding test set of descriptors of the test result.

20. The computer-implemented method of claim 15 , further comprising:

modifying the second AI model by updating training data of the second AI model to remove a portion of the set of responses generated by the second AI model indicated by the one or more certain patterns.

Assignments (1)

Continuity (2)

References Cited (97)