Systems And Methods For Automating Analyses Of User Experience Tests With Generative Language Models
Techniques are described herein for using artificial intelligence (AI) and machine learning (ML) to automate, accelerate, and enhance various aspects of user experience testing. Embodiments incorporate generative language models into user experience testing applications to extract key findings for improving product designs and driving product optimizations. In some embodiments, programmatic processes conduct a dialogue with a generative language model by engineering a set of input prompts as a function of prompt fragments, user experience test results, and test contexts. The AI-generated findings may drive actions directed to optimizing product designs and improving user experiences.
1 . A method comprising:
generating, by a process that interfaces with a generative language model, as a function of a set of results for a user experience test and a set of prompt fragments, a set of input prompts for the generative language model as part of a dialogue with the generative language model; and
generating, by the process based on responses of the generative language model to the set of input prompts during the dialogue, a set of one or more findings associated with the user experience test;
wherein the set of one or more findings identify at least one design change to a product for optimizing user experiences.
2 . The method of claim 1 , wherein the set of prompt fragments include at least a first prompt fragment mapped to a first context associated with the user experience test and at least a second prompt fragment mapped to a second context associated with the user experience test; wherein the set of input prompts includes (a) at least a first input prompt generated as a function of the first context associated with the user experience test and at least a first result for the user experience test associated with the first context and (b) at least a second input prompt generated as a function of the second context associated with the user experience test and at least a second result for the user experience test associated with the second context.
3 . The method of claim 2 , wherein the second prompt is further generated based on at least one finding generated based on the first input prompt.
4 . The method of claim 1 , wherein the process iteratively selects context associated with the user experience test and generates a different set of input prompts for different selected contexts to generate different findings for the different selected contexts.
5 . The method of claim 4 , further comprising: dynamically updating a set of contexts for selection based on findings generated based on the dialogue for a particular context.
6 . The method of claim 4 , further comprising: generating an analysis document that includes a plurality of findings across different contexts associated with the user experience test.
7 . The method of claim 1 , wherein the set of input prompts includes a first input prompt and a second input prompt; wherein the second input prompt is generated based on an analysis of the first input prompt.
8 . The method of claim 1 , wherein the set of input prompts includes a steering message that directs the generative model to take a particular point-of-view during the dialogue.
9 . The method of claim 1 , wherein the set of input prompts includes a request to rephrase a previous output of the generative language model to generate the set of one or more findings.
10 . The method of claim 1 , further comprising: performing at least one of recommending an update to a user interface, applying the update to the user interface, or annotating a prototype of the user interface based on the at least one design changed identified by the one or more findings.
11 . One or more non-transitory computer readable media storing instructions which, when executed by one or more hardware processors, cause:
generating, by a process that interfaces with a generative language model, as a function of a set of results for a user experience test and a set of prompt fragments, a set of input prompts for the generative language model as part of a dialogue with the generative language model; and
generating, by the process based on responses of the generative language model to the set of input prompts during the dialogue, a set of one or more findings associated with the user experience test;
wherein the set of one or more findings identify at least one design change to a product for optimizing user experiences.
12 . The media of claim 11 , wherein the set of prompt fragments include at least a first prompt fragment mapped to a first context associated with the user experience test and at least a second prompt fragment mapped to a second context associated with the user experience test; wherein the set of input prompts includes (a) at least a first input prompt generated as a function of the first context associated with the user experience test and at least a first result for the user experience test associated with the first context and (b) at least a second input prompt generated as a function of the second context associated with the user experience test and at least a second result for the user experience test associated with the second context.
13 . The media of claim 12 , wherein the second prompt is further generated based on at least one finding generated based on the first input prompt.
14 . The media of claim 11 , wherein the process iteratively selects context associated with the user experience test and generates a different set of input prompts for different selected contexts to generate different findings for the different selected contexts.
15 . The media of claim 14 , wherein the instructions further cause: dynamically updating a set of contexts for selection based on findings generated based on the dialogue for a particular context.
16 . The media of claim 14 , wherein the instructions further cause: generating an analysis document that includes a plurality of findings across different contexts associated with the user experience test.
17 . The media of claim 11 , wherein the set of input prompts includes a first input prompt and a second input prompt; wherein the second input prompt is generated based on an analysis of the first input prompt.
18 . The media of claim 11 , wherein the set of input prompts includes a steering message that directs the generative model to take a particular point-of-view during the dialogue.
19 . The media of claim 11 , wherein the set of input prompts includes a request to rephrase a previous output of the generative language model to generate the set of one or more findings.
20 . A system comprising:
one or more hardware processors; and
one or more non-transitory computer readable media storing instructions which, when executed by one or more hardware processors, cause:
generating, by a process that interfaces with a generative language model, as a function of a set of results for a user experience test and a set of prompt fragments, a set of input prompts for the generative language model as part of a dialogue with the generative language model; and
generating, by the process based on responses of the generative language model to the set of input prompts during the dialogue, a set of one or more findings associated with the user experience test;
wherein the set of one or more findings identify at least one design change to a product for optimizing user experiences.