Systems and methods for analyzing computer input to provide next action

1. A method for analyzing computer actions on a computer desktop system, the method comprising using one or more computers:

using a data gathering process by data collection software executed by a processor on a first computer of the one or more computers, gathering a low-level user action information item describing input by a user to one of a plurality of software programs via a desktop system, the software programs separate from the data collection software, the low-level user action information item input to the desktop system and comprising an input type description, and screen window information;

based on the low-level user action information item and a current state of a user's interaction with a software program of the plurality of software programs, determining a best next low-level user action using reinforcement learning, wherein determining the best next low-level user action comprises choosing a state-action pair based on a total expected reward associated with the state-action pair, the total expected reward based on the amount of time a user spends before taking a next-action and normalized to the action taken by the user;

on the first computer, displaying the best next low-level user action; and

receiving a user input in response to the displaying of the best next low-level user action and modifying a table based on user input received in response to the displaying.

2. The method of claim 1 , wherein determining a next best low-level user action comprises accessing the table.

3. The method of claim 1 , wherein the modification increases an internal reward if the user executed the best next low-level user action and decreases the internal reward if the user did not execute the best next low-level user action.

4. The method of claim 1 , wherein the low level user action information item comprises event time, user identification, action description, a description of the screen window in which the action takes place, and the name of the program executing the window.

5. The method of claim 1 comprising determining a total expected reward associated with a set of state-action pairs based on a series of actual low-level user action information items, each associated with a process the first computer is engaging in with the user.

6. The method of claim 1 wherein the reward is based on whether or not the action alters a process associated with the state.

7. The method of claim 1 , wherein receiving a user input estimates the best next low-level user action based on an exponential decay of user actions.

8. The method of claim 1 wherein determining a best next low-level user action is based on an internal reward associated with the transition to a new state based on an action, where the reward is based on whether the action causes a deviation to a different process.

9. The method of claim 1 comprising performing an internal reward learning process which iterates over the actions, modifying the internal reward during the transition of the action to a new state.

10. The method of claim 1 , wherein determining a best next low-level user action using reinforcement learning is built offline based on historical data.

11. A system for analyzing computer actions on a computer desktop system, the system comprising a memory and one or more computers configured to:

execute a data gathering process by data collection software executed on a computer to gather a low-level user action information item describing input by a user to one of a plurality of software programs via a desktop system, the software programs separate from the data collection software, the low-level user action information item input to the desktop system and comprising an input type description, and screen window information;

based on the low-level user action information item and a current state of a user's interaction with a software program of the plurality of software programs, determine a best next low-level user action using reinforcement learning, wherein determining the best next low-level user action comprises choosing a state-action pair based on a total expected reward associated with the state-action pair, the total expected reward based on the amount of time a user spends before taking a next-action and normalized to the action taken by the user;

on the computer, display the best next low-level user action; and

receive a user input in response to the displaying of the best next low-level user action and modify a table based on user input received in response to the displaying.

12. The system of claim 11 , wherein determining a next best low-level user action comprises accessing the table.

13. The system of claim 11 , wherein the modification increases an internal reward if the user executed the best next low-level user action and decreases the internal reward if the user did not execute the best next low-level user action.

14. The system of claim 11 , wherein the low level user action information item comprises event time, user identification, action description, a description of the screen window in which the action takes place, and the name of the program executing the window.

15. The system of claim 11 wherein the one or more computers are configured to determine a total expected reward associated with a set of state-action pairs based on a series of actual low-level user action information items, each associated with a process the first computer is engaging in with the user.

16. The system of claim 11 wherein the reward is based on whether or not the action alters a process associated with the state.

17. A method for analyzing user-computer interaction on a computer desktop system, the method comprising:

gathering by data collection software a user action information item describing user input to one of a plurality of software programs via a desktop system, the software programs separate from the data collection software, the user action information item input to the desktop system and comprising an input type description, and screen window information;

determining a recommended next user action based on the user action information item and a current state of a user's interaction with a software program of the plurality of software programs using reinforcement learning, user action information items gathered from other users, and the user processes and time to complete information of the user action information items gathered from other users, each recommended next user action chosen based on a total expected reward associated with the next user action and normalized to the action taken by the user;

providing to a user the recommended next user action; and

receiving a user input in response to the providing of the recommended action and modifying a table based on user input received in response to the displaying.

18. The method of claim 17 , wherein determining a recommended low-level user action comprises accessing the table.

19. The method of claim 17 , wherein the modification increases an internal reward if the user executed the recommended user action and decreases the internal reward if the user did not execute the recommended user action.

20. The method of claim 17 , wherein the user action information item comprises event time, user identification, action description, a description of the screen window in which the action takes place, and the name of the program executing the window.

21. The method of claim 17 wherein the reward is based on whether or not the action alters the process associated with the state.

22. A method for analyzing computer actions on a computer desktop system, the method comprising using one or more computers:

using a data gathering process by data collection software executed by a processor on a first computer of the one or more computers, gathering a low-level user action information item describing input by a user to one of a plurality of software programs via a desktop system, the software programs separate from the data collection software, the low-level user action information item input to the desktop system and comprising an input type description, and screen window information;

based on the low-level user action information item and a current state of a user's interaction with a software program of the plurality of software programs, determining a best next low-level user action using reinforcement learning, wherein determining the best next low-level user action is based on an internal reward associated with the transition to a new state based on an action, where the reward is based on a user time to complete the action and whether or not the action alters a process associated with the state, and is normalized to the action taken by the user; and

on the first computer, displaying the best next low-level user action.