Cognitive robotics analyzer
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a cognitive robotics analyzer are disclosed. In one aspect, a method includes the actions of receiving, for each user characteristic of a plurality of user characteristics, first data that identifies one or more first actions that perform a first process and second data that identifies one or more second actions that perform a second process that is labeled as similar to the first process. The actions further include training a predictive model. The actions further include receiving data that identifies actions performed by a user. The actions further include applying the predictive model to one or more of the actions. The actions further include classifying a process performed by the one or more actions as similar to a particular process. The actions further include associating the user with the particular user characteristic.
1. A computer-implemented method comprising:
receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions;
receiving first process data that identifies a first process performed by the one or more first actions and second process data that identifies a second process performed by the one or more second actions;
receiving process similarity data that indicates that the first process is similar to the second process;
training, using the first action data, the second action data, additional action data, the first process data, the second process data, additional process data, the process similarity data, and additional process similarity data, the predictive model that is configured to classify one or more actions as performing a process that is similar or not similar to the first process and the second process;
receiving data that identifies a group of actions that perform the first process and that include an unnecessary action for the group of actions of the process;
applying, to the group of actions, a predictive model that is configured to classify one or more actions as performing the process that is similar or not similar to the first process;
based on applying the predictive model to the one or more of the actions, classifying the process performed by the group of actions as similar to the first process;
identifying, from among the group of actions, a particular action;
applying the predictive model to the group of actions with the particular action removed;
based on applying the predictive model to the group of actions with the particular action removed, classifying the process performed by the group of actions with the particular action removed as similar to the first process; and
based on classifying the process performed by the group of actions as similar to the first process and based on classifying the process performed by the group of actions with the particular action removed as similar to the first process, determining that the particular action is the unnecessary action for the group of actions; and
based on determining that the particular action is the unnecessary action for the group of actions, providing guidance to a user that performed the unnecessary action.
2. The method of claim 1 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first screen capture data from a first device performing the one or more first actions;
receiving second screen capture data from a second device performing the one or more second actions;
generating the first action data by performing computer vision techniques on the first screen capture data; and
generating the second action data by performing computer vision techniques on the second screen capture data.
3. The method of claim 1 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first user input data from a first device performing the one or more first actions;
receiving second user input data from a second device performing the one or more second actions;
generating the first action data by analyzing the first user input data; and
generating the second action data by analyzing the second user input data.
4. The method of claim 1 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first network traffic data from a first device performing the one or more first actions;
receiving second network traffic data from a second device performing the one or more second actions;
generating the first action data by analyzing the first network traffic data; and
generating the second action data by analyzing the second network traffic data.
5. The method of claim 1 , wherein an action of the one or more first actions, the one or more second actions, or the group of actions comprises a key press, a mouse click, a screen touch, a foreground process change, a scene change, a network request, or a network receipt.
6. The method of claim 1 , comprising:
receiving data confirming that the process performed by the group of actions with the particular action removed is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions with the particular action removed is similar to the first process.
7. The method of claim 1 , comprising:
receiving data confirming that the process performed by the group of actions is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions is similar to the first process.
8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions;
receiving first process data that identifies a first process performed by the one or more first actions and second process data that identifies a second process performed by the one or more second actions;
receiving process similarity data that indicates that the first process is similar to the second process;
training, using the first action data, the second action data, additional action data, the first process data, the second process data, additional process data, the process similarity data, and additional process similarity data, the predictive model that is configured to classify one or more actions as performing a process that is similar or not similar to the first process and the second process;
receiving data that identifies a group of actions that perform the first process and that include an unnecessary action for the group of actions of the process;
applying, to the group of actions, a predictive model that is configured to classify one or more actions as performing the process that is similar or not similar to the first process;
based on applying the predictive model to the one or more of the actions, classifying the process performed by the group of actions as similar to the first process;
identifying, from among the group of actions, a particular action;
applying the predictive model to the group of actions with the particular action removed;
based on applying the predictive model to the group of actions with the particular action removed, classifying the process performed by the group of actions with the particular action removed as similar to the first process; and
based on classifying the process performed by the group of actions as similar to the first process and based on classifying the process performed by the group of actions with the particular action removed as similar to the first process, determining that the particular action is the unnecessary action for the group of actions; and
based on determining that the particular action is the unnecessary action for the group of actions, providing guidance to a user that performed the unnecessary action.
9. The system of claim 8 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first screen capture data from a first device performing the one or more first actions;
receiving second screen capture data from a second device performing the one or more second actions;
generating the first action data by performing computer vision techniques on the first screen capture data; and
generating the second action data by performing computer vision techniques on the second screen capture data.
10. The system of claim 8 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first user input data from a first device performing the one or more first actions;
receiving second user input data from a second device performing the one or more second actions;
generating the first action data by analyzing the first user input data; and
generating the second action data by analyzing the second user input data.
11. The system of claim 8 , wherein receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions comprises:
receiving first network traffic data from a first device performing the one or more first actions;
receiving second network traffic data from a second device performing the one or more second actions;
generating the first action data by analyzing the first network traffic data; and
generating the second action data by analyzing the second network traffic data.
12. The system of claim 8 , wherein an action of the one or more first actions, the one or more second actions, or the group of actions comprises a key press, a mouse click, a screen touch, a foreground process change, a scene change, a network request, or a network receipt.
13. The system of claim 8 , wherein the operations further comprise:
receiving data confirming that the process performed by the group of actions with the particular action removed is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions with the particular action removed is similar to the first process.
14. The system of claim 8 , wherein the operations further comprise:
receiving data confirming that the process performed by the group of actions is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions is similar to the first process.
15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving first action data that identifies one or more first actions and second action data that identifies one or more second actions;
receiving first process data that identifies a first process performed by the one or more first actions and second process data that identifies a second process performed by the one or more second actions;
receiving process similarity data that indicates that the first process is similar to the second process;
training, using the first action data, the second action data, additional action data, the first process data, the second process data, additional process data, the process similarity data, and additional process similarity data, the predictive model that is configured to classify one or more actions as performing a process that is similar or not similar to the first process and the second process;
receiving data that identifies a group of actions that perform the first process and that include an unnecessary action for the group of actions of the process;
applying, to the group of actions, a predictive model that is configured to classify one or more actions as performing the process that is similar or not similar to the first process;
based on applying the predictive model to the one or more of the actions, classifying the process performed by the group of actions as similar to the first process;
identifying, from among the group of actions, a particular action;
applying the predictive model to the group of actions with the particular action removed;
based on applying the predictive model to the group of actions with the particular action removed, classifying the process performed by the group of actions with the particular action removed as similar to the first process; and
based on classifying the process performed by the group of actions as similar to the first process and based on classifying the process performed by the group of actions with the particular action removed as similar to the first process, determining that the particular action is the unnecessary action for the group of actions; and
based on determining that the particular action is the unnecessary action for the group of actions, providing guidance to a user that performed the unnecessary action.
16. The medium of claim 15 , wherein the operations further comprise:
receiving data confirming that the process performed by the group of actions with the particular action removed is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions with the particular action removed is similar to the first process.
17. The medium of claim 15 , wherein the operations further comprise:
receiving data confirming that the process performed by the group of actions is similar to the first process; and
updating the predictive model based on the data confirming that the process performed by the group of actions is similar to the first process.