Granted Patent B1

US 12,602,413 · App. 19/191,776 · Granted Apr 14, 2026

Inventors: Ofir Ezrielev (Be'er Sheva, IL); Hanna Yehuda (Acton, MA); Anne-Marie Mcreynolds (San Jose, CA)

Assignee: Dell Products L.P.

G06F16/33295G06F16/3328

Abstract

Claims (88)

1 . A method for providing computer-implemented services using generative artificial intelligence (AI) models, the method comprising:

obtaining a prompt for a generative AI model of the generative AI models;

obtaining, using the prompt and the generative AI model, a first output from the generative AI model;

performing, using at least the prompt, the first output, and the generative AI model, an output explainability process to obtain an explainability report for the first output, the explainability report indicating relationships between first information elements of the prompt and second information elements of the first output, and the relationships are based, at least in part, on a second output generated by the generative AI model using a modified prompt and the modified prompt being based on the prompt; and

providing the first output and the explainability report to a downstream consumer as part of providing computer-implemented services to the downstream consumer.

2 . The method of claim 1 , wherein performing the output explainability process comprises:

performing, using the prompt, a prompt modification process to obtain the modified prompt;

obtaining, using the modified prompt and the generative AI model, the second output from the generative AI model;

obtaining, using the second output, third information elements of the second output;

comparing the second information elements of the first output and the third information elements of the second output to obtain a difference; and

populating, based on the difference, the explainability report with at least a first relationship of the relationships.

3 . The method of claim 2 , wherein performing the prompt modification process comprises:

obtaining, using the prompt and the first output, the first information elements and the second information elements; and

performing at least one action selected from a list of actions consisting of:

removing a first information element of the first information elements to obtain the modified prompt; and

replacing the first information element with a replacement information element to obtain the modified prompt.

4 . The method of claim 3 , wherein performing the prompt modification process further comprises:

in an instance of the performing where the at least one action comprises replacing the first information element:

prior to replacing the first information element:

obtaining, based on the first information element and an information element replacement schema, the replacement information element.

5 . The method of claim 3 , wherein performing the prompt modification process further comprises:

in an instance of the performing where the at least one action comprises replacing the first information element:

prior to replacing the first information element:

displaying, via a graphical user interface (GUI), at least the first information elements to a user; and

obtaining, via an interaction with the GUI, feedback from the user, the feedback comprising the replacement information element.

6 . The method of claim 5 , wherein displaying the at least the first information elements to the user comprises:

obtaining a first graphical representation of the first information elements, the first graphical representation comprising:

a first set of nodes that correspond to the first information elements, and

first edges that indicate relationships between the first set of the nodes; and

populating the GUI with the first graphical representation.

7 . The method of claim 5 , wherein comparing the second information elements of the first output and the third information elements of the second output comprises:

obtaining a second graphical representation of the second information elements, the second graphical representation comprising:

a second set of nodes that correspond to the second information elements, and

second edges that represent relationships between the second set of the nodes;

obtaining a third graphical representation of the third information elements, the third graphical representation comprising:

a third set of nodes that correspond to the third information elements, and

third edges that represent relationships between the third set of the nodes; and

identifying, using the second graphical representation and the third graphical representation, one or more nodes of the second set of the nodes that is not present in the third graphical representation.

8 . The method of claim 7 , wherein the first output comprises a second information element and the second output does not comprise the second information element.

9 . The method of claim 8 , wherein the second output comprises a third information element and the first output does not comprise the third information element.

10 . The method of claim 7 , wherein the second graphical representation and the third graphical representation are displayed to the user via the GUI.

11 . The method of claim 1 , wherein performing the output explainability process comprises:

iteratively perturbating information content of the prompt to obtain perturbated prompts;

evaluating the perturbated prompts using the generative AI model to obtain perturbated outputs; and

using the perturbated prompts and the perturbated outputs to identify the relationships between the first information elements and the second information elements.

12 . The method of claim 1 , wherein the generative AI model is a large language model (LLM).

13 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for providing computer-implemented services using generative artificial intelligence (AI) models, the operations comprising:

obtaining a prompt for a generative AI model of the generative AI models;

obtaining, using the prompt and the generative AI model, a first output from the generative AI model;

performing, using at least the prompt, the first output, and the generative AI model, an output explainability process to obtain an explainability report for the first output, the explainability report indicating relationships between first information elements of the prompt and second information elements of the first output, and the relationships are based, at least in part, on a second output generated by the generative AI model using a modified prompt and the modified prompt being based on the prompt; and

providing the first output and the explainability report to a downstream consumer as part of providing computer-implemented services to the downstream consumer.

14 . The non-transitory machine-readable medium of claim 13 , wherein performing the output explainability process comprises:

performing, using the prompt, a prompt modification process to obtain the modified prompt;

obtaining, using the modified prompt and the generative AI model, the second output from the generative AI model;

obtaining, using the second output, third information elements of the second output;

comparing the second information elements of the first output and the third information elements of the second output to obtain a difference; and

populating, based on the difference, the explainability report with at least a first relationship of the relationships.

15 . The non-transitory machine-readable medium of claim 14 , wherein performing the prompt modification process comprises:

obtaining, using the prompt and the first output, the first information elements and the second information elements; and

performing at least one action selected from a list of actions consisting of:

removing a first information element of the first information elements to obtain the modified prompt; and

replacing the first information element with a replacement information element to obtain the modified prompt.

16 . The non-transitory machine-readable medium of claim 15 , wherein performing the prompt modification process further comprises:

in an instance of the performing where the at least one action comprises replacing the first information element:

prior to replacing the first information element:

obtaining, based on the first information element and an information element replacement schema, the replacement information element.

17 . A data processing system, comprising:

a processor; and

a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for providing computer-implemented services using generative artificial intelligence (AI) models, the operations comprising:

obtaining a prompt for a generative AI model of the generative AI models;

obtaining, using the prompt and the generative AI model, a first output from the generative AI model;

performing, using at least the prompt, the first output, and the generative AI model, an output explainability process to obtain an explainability report for the first output, the explainability report indicating relationships between first information elements of the prompt and second information elements of the first output, and the relationships are based, at least in part, on a second output generated by the generative AI model using a modified prompt and the modified prompt being based on the prompt; and

providing the first output and the explainability report to a downstream consumer as part of providing computer-implemented services to the downstream consumer.

18 . The data processing system of claim 17 , wherein performing the output explainability process comprises:

performing, using the prompt, a prompt modification process to obtain the modified prompt;

obtaining, using the modified prompt and the generative AI model, the second output from the generative AI model;

obtaining, using the second output, third information elements of the second output;

comparing the second information elements of the first output and the third information elements of the second output to obtain a difference; and

populating, based on the difference, the explainability report with at least a first relationship of the relationships.

19 . The data processing system of claim 18 , wherein performing the prompt modification process comprises:

obtaining, using the prompt and the first output, the first information elements and the second information elements; and

performing at least one action selected from a list of actions consisting of:

removing a first information element of the first information elements to obtain the modified prompt; and

replacing the first information element with a replacement information element to obtain the modified prompt.

20 . The data processing system of claim 19 , wherein performing the prompt modification process further comprises:

in an instance of the performing where the at least one action comprises replacing the first information element:

prior to replacing the first information element:

obtaining, based on the first information element and an information element replacement schema, the replacement information element.

Assignments (1)

References Cited (31)