Advanced analysis of online user experience studies
Advanced user experience study can be accomplished by enabling recording on a device such as a smart phone, providing a participant with one or more survey questions and a task. Typically, the participants are recorded and redirected to a starting URL. The recording can be terminated when the participant reaches a validation condition. Results, for example, videos, survey results, click flows, and heat maps may be filterable across any participant feature or any study validation criteria. Validation criteria includes time taken to complete a given action, ending up at a particular URL (or class of URLs), based upon question answers, or any combination thereof. These criteria can be classified as either a successful completion of the study, or a failed attempt. Additionally, the validation criteria can also include a decision by the participant to abandon the study.
1. A method for an advanced user experience study comprising:
enabling recording on a device;
providing a participant at least one survey question;
providing the participant a task as part of a study;
recording the participant;
redirecting the participant to a starting task;
receiving, from the participant, navigation inputs from the starting task to a final task;
comparing the final task to a set of validation conditions;
providing participant trait metadata and the set of validation conditions to a neural network for identifying traits that are associated with failure validation condition;
identifying a root cause for the failure validation condition responsive to the traits;
generating a clickstream analysis as an aggregation of the navigation inputs from a plurality of participants, wherein the aggregation includes a likelihood of any given path of a plurality of alternate paths presented as a branched graph;
providing the clickstream analysis to a machine learning model for identifying transition points of the alternate paths in the aggregated navigation inputs that are points of confusion which are predictive of a failed outcome; and
generating a clip of the recording including the point of confusion.
2. The method of claim 1 , wherein the device includes a desktop computer, a laptop computer, a mobile device, a smartwatch, or smart glasses.
3. The method of claim 1 , further comprising confirming that the device is capable of audio and video recording.
4. The method of claim 1 , further comprising disqualifying the participant based upon answers to one of the at least one survey question.
5. The method of claim 1 , wherein the task is a navigation task.
6. The method of claim 5 , wherein the task is to find a particular product or class of products.
7. The method of claim 5 , wherein the task is finding an end URL.
8. The method of claim 1 , wherein a success indicator includes at least one of arriving at a particular URL, arriving at a URL including a keyword, answering a success question or a timer.
9. The method of claim 1 , further comprising filtering answers to the at least one survey question and the recording by a validation condition.
10. The method of claim 1 , wherein an interrupting includes asking the participant a question or request an activity from the participant.
11. The method of claim 1 , wherein the machine learning model is a deep learning neural network which consumes the branched graph and a plurality of parameters of the study as features to perform predictive analysis of a failure indicator.
12. A system for an advanced user experience study comprising:
a device for enabling recording;
a study administrator for providing a participant at least one survey question, providing the participant a task, recording the participant, redirecting the participant to a starting task, receiving navigation inputs from the starting task to a final task, comparing the final task to a set of validation conditions, providing participant trait metadata and the set of validation conditions to a neural network for identifying traits that are associated with failure validation condition, identifying a root cause for the failure validation condition responsive to the traits, generating a clickstream analysis as an aggregation of the navigation inputs from a plurality of participants, wherein the aggregation includes a likelihood of any given path of a plurality of alternate paths presented as a branched graph, providing the clickstream analysis to a machine learning model for identifying transition points of the alternate paths in the aggregated navigation inputs that are points of confusion which are predictive of a failed outcome, and generating a clip of the recording including the point of confusion.
13. The system of claim 12 , wherein the device includes a desktop computer, a laptop computer, a mobile device, a smartwatch, or smart glasses.
14. The system of claim 12 , wherein the study administrator is further for confirming that the device is capable of audio and video recording.
15. The system of claim 12 , further comprising disqualifying the participant based upon answers to one of the at least one survey question.
16. The system of claim 12 , wherein the task is a navigation task.
17. The system of claim 16 , wherein the task is to find a particular product or class of products.
18. The system of claim 16 , wherein the task is finding an end URL.
19. The system of claim 12 , wherein a success indicator includes at least one of arriving at a particular URL, arriving at a URL including a keyword, answering a success question or a timer.
20. The system of claim 12 , further comprising an analyzer for filtering answers to the at least one survey question and the recording by a validation condition.
21. The system of claim 12 , wherein an interrupting includes asking the participant a question or request an activity from the participant.
22. The system of claim 12 , wherein the machine learning model is a deep learning neural network which consumes the branched graph and a plurality of parameters of the study as features to perform predictive analysis of a failure indicator.