Systems and methods for automating comparative user experience test analyses
Techniques are described herein for using artificial intelligence (AI) and machine learning (ML) to automate, accelerate, and enhance various aspects of comparative user experience testing. Embodiments interface with generative language models to compare user experiences and summarize the results of the comparison. In some embodiments, automated systems and programmatic processes access a series of analysis contexts, where a context includes a collection of message content fragments. The systems and processes may use the message content fragments for a given context to construct a dialogue with a generative language model to compare separate user experiences based on the results of a set of user experience tests. The output of the generative language model at one stage of the analysis may be combined with content fragments for another context to craft a dialogue at another stage of the analysis and/or to perform additional analyses of the user experiences.
1. A method comprising:
generating, by a process based at least in part on a first set of message fragments and a plurality of user experience test result sets, a first set of automated prompts for a generative language model to generate an analysis for each user experience test result set of the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of a recurrent neural network or a transformer model to the first set of automated prompts;
generating, by the process based at least in part on a second set of message fragments and the analysis for each user experience test result set of the plurality of user experience test result sets, a second set of automated prompts for the generative language model to compare at least part of different analyses for the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of the recurrent neural network or the transformer model to the second set of automated prompts; and
presenting, within a user interface based on at least one response received from the generative language model to the second set of automated prompts, a comparison of the analysis for each user experience test result set of the plurality of user experience test result sets, wherein the comparison is used to implement at least one product design optimization to enhance user experiences with a product.
2. The method of claim 1 , further comprising: generating, using the generative language model, a summary of the analysis; wherein the summary is combined with the second set of message fragments to generate the second set of automated prompts for the generative language model.
3. The method of claim 1 , wherein the first set of automated prompts are constructed based at least in part on a combination of the first set of message fragments with each user experience test result set in the plurality of user experience test result sets.
4. The method of claim 1 , wherein the second set of automated prompts includes a first prompt directing the generative language model to create the comparison and at least a second prompt directing the generative language model refine the comparison generated by the generative language model responsive to the first prompt.
5. The method of claim 4 , wherein the second prompt directs the generative language model to refine the comparison to conform to a particular style format.
6. The method of claim 1 , further comprising: identifying at least one design change associated with the product predicted to improve user experiences with the product based on the comparison.
7. The method of claim 1 , wherein the comparison compares strengths and weaknesses of different versions of a design for the product.
8. The method of claim 1 , wherein the comparison compares strengths and weaknesses of a product design with competitive products.
9. The method of claim 1 , wherein implementing the product design optimization includes updating a webpage or application page predicted, by the generative language model, to optimize user experiences.
10. One or more non-transitory computer readable media storing instructions which, when executed by one or more hardware processors, cause:
generating, by a process based at least in part on a first set of message fragments and a plurality of user experience test result sets, a first set of automated prompts for a generative language model to generate an analysis for each user experience test result set of the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of a recurrent neural network or a transformer model to the first set of automated prompts;
generating, by the process based at least in part on a second set of message fragments and the analysis for each user experience test result set of the plurality of user experience test result sets, a second set of automated prompts for the generative language model to compare at least part of different analyses for the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of the recurrent neural network or the transformer model to the second set of automated prompts; and
presenting, within a user interface based on at least one response received from the generative language model to the second set of automated prompts, a comparison of the analysis for each user experience test result set of the plurality of user experience test result sets, wherein the comparison is used to implement at least one product design optimization to enhance user experiences with a product.
11. The media of claim 10 , wherein the instructions further cause: generating, using the generative language model, a summary of the analysis; wherein the summary is combined with the second set of message fragments to generate the second set of automated prompts for the generative language model.
12. The media of claim 10 , wherein the first set of automated prompts are constructed based at least in part on a combination of the first set of message fragments with each user experience test result set in the plurality of user experience test result sets.
13. The media of claim 10 , wherein the second set of automated prompts includes a first prompt directing the generative language model to create the comparison and at least a second prompt directing the generative language model refine the comparison generated by the generative language model responsive to the first prompt.
14. The media of claim 13 , wherein the second prompt directs the generative language model to refine the comparison to conform to a particular style format.
15. The media of claim 10 , wherein the instructions further cause: identifying at least one design change associated with the product predicted to improve user experiences with the product based on the comparison.
16. The media of claim 10 , wherein the comparison compares strengths and weaknesses of different versions of a design for the product.
17. The media of claim 10 , wherein the comparison compares strengths and weaknesses of a product design with competitive products.
18. A system comprising:
one or more hardware processors; and
one or more non-transitory computer readable media storing instructions which, when executed by one or more hardware processors, cause:
generating, by a process based at least in part on a first set of message fragments and a plurality of user experience test result sets, a first set of automated prompts for a generative language model to generate an analysis for each user experience test result set of the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of a recurrent neural network or a transformer model to the first set of automated prompts;
generating, by the process based at least in part on a second set of message fragments and the analysis for each user experience test result set of the plurality of user experience test result sets, a second set of automated prompts for the generative language model to compare at least part of different analyses for the plurality of user experience test result sets;
providing, by the process, the first set of automated prompts as input to the generative language model, wherein the generative language model applies at least one of the recurrent neural network or the transformer model to the second set of automated prompts; and
presenting, within a user interface based on at least one response received from the generative language model to the second set of automated prompts, a comparison of the analysis for each user experience test result set of the plurality of user experience test result sets, wherein the comparison is used to implement at least one product design optimization to enhance user experiences with a product.
19. The system of claim 18 , wherein the instructions further cause: generating, using the generative language model, a summary of the analysis; wherein the summary is combined with the second set of message fragments to generate the second set of automated prompts for the generative language model.
20. The media of claim 18 , wherein the first set of automated prompts are constructed based at least in part on a combination of the first set of message fragments with each user experience test result set in the plurality of user experience test result sets.