Techniques for automatically triaging and describing issues detected during use of a software application
Described herein are techniques that use machine learning to triage issues by classifying the issues into impact levels. Described herein are also techniques for generating a natural language description of issues that occur in sessions of a software application. The techniques collect data during sessions in which a user is interacting with the software application. The techniques process the data collected during the sessions using a language model to obtain natural language descriptions of issues that occur in the sessions.
1. A system for automatically generating natural language descriptions of issues that occur during interactions of users with a software application, the system comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to:
access data collected during at least one software application session in which at least one user was interacting with the software application and in which an issue occurred;
generate, using the data collected during the at least one software application session, at least one representation of the at least one software application session, wherein the at least one representation of the at least one software application session indicates a sequence of events that occurred in the at least one software application session; and
process, using a description generation module, the at least one representation of the at least one software application session using an automated series of input/output exchanges between the description generation module and a generative machine learning model to obtain the natural language description of the issue that occurred in the at least one software application session, the processing comprising:
generate an input to the generative machine learning model that includes at least a portion of the at least one representation of the at least one software application session;
provide the input to the generative machine learning model to obtain an output;
generate a subsequent input to the generative machine learning model by processing the output;
provide the subsequent input to the generative machine learning model to obtain a subsequent output; and
generate the natural language description of the issue using the subsequent output obtained from the generative machine learning model.
2. The system of claim 1 , wherein the generative machine learning model is a large language model.
3. The system of claim 1 , wherein the generative machine learning model comprises a transformer model.
4. The system of claim 1 , wherein:
the output indicates whether any issue occurred in the at least one software application session; and
generating the subsequent input to the generative machine learning model by processing the output comprises:
when the output indicates that the issue occurred in the at least one software application session:
generate, as the subsequent input, a request for the generative machine learning model to generate the natural language description of the issue.
5. The system of claim 4 , wherein generating the input comprises:
generate a query for whether any issue occurred in the at least one software application session.
6. The system of claim 1 , wherein the instructions further cause the processor to:
determine, using the data collected during the at least one software application session, whether the at least one software application session meets one or more rules; and
process the at least one representation of the at least one software application session using the generative machine learning model to obtain the natural language description of the issue that occurred in the at least one software application session when it is determined that the at least one software application session meets the one or more rules.
7. The system of claim 6 , wherein determining, using the data collected during the at least one software application session, whether the at least one software application session meets the one or more rules comprises:
determine whether repeated user activity occurred proximate occurrence of the issue in the at least one software application session.
8. The system of claim 6 , wherein determining, using the data collected during the at least one software application session, whether the at least one software application session meets the one or more rules comprises:
determine whether the at least one representation includes a threshold number of events in the at least one representation of the at least one software application session.
9. The system of claim 6 , wherein determining, using the data collected during the at least one software application session, whether the at least one software application session meets the one or more rules comprises:
determine whether a particular error occurred in the software application during the at least one software application session.
10. The system of claim 1 , wherein:
the at least one software application session comprises a plurality of software application sessions;
the at least one representation of the at least one software application session comprises a plurality of representations of the plurality of software application sessions; and
processing, using the description generation module, the at least one representation of the at least one software application session using the automated series of input/output exchanges between the description generation module and the generative machine learning model to obtain the natural language description of the issue that occurred in the at least one software application session comprises:
process the plurality of session representations using the generative machine learning model to determine whether the issue occurred in at least a threshold number of the plurality of software application sessions; and
when it is determined that the issue occurred in at least the threshold number of the plurality of software application sessions:
provide input to the generative machine learning model requesting the natural language description of the issue.
11. The system of claim 10 , wherein the threshold number of software application sessions is 3 software application sessions.
12. The system of claim 10 , wherein processing the plurality of session representations using the generative machine learning model to determine whether the issue occurred in at least a threshold number of the plurality of software application sessions comprises:
process a first one of the plurality of representations using the generative machine learning model to obtain a first output indicating that an issue occurred in a first one of the plurality of software application sessions;
process a second one of the plurality of representations using the generative machine learning model to obtain a second output indicating whether the issue that occurred in the first software application session occurred in a second one of the plurality of software application sessions; and
determine whether the issue occurred in the threshold number of the plurality of software application sessions based at least in part on the first output and the second output.
13. The system of claim 1 , wherein generating the input comprises:
identify a portion of the at least one representation of the at least one software application session; and
generate the input for the generative machine learning model that includes the identified portion of the at least one representation of the at least one software application session and excludes portions of the at least one representation of the at least one software application session that is outside of the identified portion.
14. The system of claim 13 , wherein identifying the portion of the at least one representation of the at least one software application session comprises:
identify a portion of the at least one representation of the at least one software application session corresponding to a reaction of the at least one user to occurrence of the issue.
15. The system of claim 1 , wherein generating, using the data collected during the at least one software application session, the at least one representation of the at least one software application session comprises:
generating, for the at least one software application session, at least one textual transcription of the sequence of events that occurred in the at least one software application session.
16. The system of claim 1 , wherein generating, using the data collected during the at least one software application session, the at least one representation of the at least one software application session comprises:
generating, for the at least one software application session, at least one image and/or video indicating the sequence of events that occurred in the at least one software application session.
17. The system of claim 1 , wherein the instructions further cause the processor to:
detect occurrence of the issue in the at least one software application session; and
determine a classification of the issue into one of a plurality of impact levels using the data collected during the at least one software application session;
wherein processing the at least one representation of the at least one software application session comprises performing the processing when the issue is classified into a first impact level of the plurality of impact levels.
18. The system of claim 17 , wherein determining the classification of the issue into one of the plurality of impact levels using the data collected during the at least one software application session comprises:
generating a set of feature values associated with the issue using the data collected during the at least one software application session; and
providing the set of feature values associated with the issue as input to a trained machine learning model to obtain output indicating the classification of the issue into the first impact level.
19. The system of claim 17 , wherein determining the classification of the issue into one of the plurality of impact levels using the data collected during the at least one software application session comprises:
determining values of one or more parameters indicating reaction of one or more users in one or more respective GUIs of the software application in response to occurrence of the issue; and
determining the classification of the issue into one of the plurality of impact levels using the values of the one or more parameters.
20. A method for automatically generating natural language descriptions of issues that occur during interactions of users with a software application, the method comprising:
using a processor to perform:
accessing data collected during at least one software application session in which at least one user was interacting with the software application and in which an issue occurred;
generating, using the data collected during the at least one software application session, at least one representation of the at least one software application session, wherein the at least one representation of the at least one software application session indicates a sequence of events that occurred in the at least one software application session; and
processing, using a description generation module, the at least one representation of the at least one software application session using an automated series of input/output exchanges between the description generation module and a generative machine learning model to obtain the natural language description of the issue that occurred in the at least one software application session, the processing comprising:
generating, an input to the generative machine learning model that includes at least a portion of the at least one representation of the at least one software application session;
providing the input to the generative machine learning model to obtain an output;
generating a subsequent input to the generative machine learning model by processing the output;
providing the subsequent input to the generative machine learning model to obtain a subsequent output; and
generating the natural language description of the issue using the subsequent output obtained from the generative machine learning model.
21. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method for automatically generating natural language descriptions of issues that occur during interactions of users with a software application, the method comprising:
accessing data collected during at least one software application session in which at least one user was interacting with the software application and in which an issue occurred;
generating, using the data collected during the at least one software application session, at least one representation of the at least one software application session, wherein the at least one representation of the at least one software application session indicates a sequence of events that occurred in the at least one software application session; and
processing, using a description generation module, the at least one representation of the at least one software application session using an automated series of input/output exchanges between the description generation module and a generative machine learning model to obtain the natural language description of the issue that occurred in the at least one software application session, the processing comprising:
generating, an input to the generative machine learning model that includes at least a portion of the at least one representation of the at least one software application session;
providing the input to the generative machine learning model to obtain an output;
generating a subsequent input to the generative machine learning model by processing the output;
providing the subsequent input to the generative machine learning model to obtain a subsequent output; and
generating the natural language description of the issue using the subsequent output obtained from the generative machine learning model.