Browser based agent

1 . A method comprising:

receiving, via a first browser interface element, a prompt that includes an instruction;

determining, by a planning model, based on the instruction and browser data associated with a user, an execution sequence of a series of online-related actions, wherein the execution sequence designates at least a subset of the series of online-related actions to be performed in parallel to accelerate execution of the instruction;

performing, via an action model and in accordance with the execution sequence, the series of online-related actions by processing content from at least one online resource to determine and execute a corresponding sequence of low-level browser interactions; and

providing, via a second browser interface element, an indication that the instruction is completed.

2 . The method of claim 1 , further comprising retrieving, from a memory model and by the planning model, history information that stores historical user interactions and preferences, wherein the history information is used to determine the execution sequence.

3 . The method of claim 2 , further comprising:

providing an indication of the history information to the user; and

receiving an edit to the history information from the user, wherein the edited history information is used to determine the execution sequence.

4 . The method of claim 1 , further comprising, in response receiving the prompt:

parsing, by the planning model, content of a currently visited online resource to extract one or more parameters for the instruction;

providing the one or more parameters to the user for confirmation; and

receiving confirmation of a parameter of the one or more parameters,

wherein determining the execution sequence is further based on the parameter.

5 . The method of claim 1 , wherein the action model is configured to call a service that performs at least one of the series of online-related actions using a language-agnostic interface.

6 . The method of claim 1 , wherein the prompt is a first prompt, the method further comprising:

generating a second prompt to request additional information from the user in response to determining, via the planning model during the determination of the execution sequence, that additional information is necessary to perform at least one of the series of online-related actions;

receiving a response to the second prompt; and

updating the execution sequence based on the response.

7 . The method of claim 1 , wherein the prompt is a first prompt and the indication includes a second prompt or an alert message provided via an overlay window, a tab, or separate browser window.

8 . The method of claim 1 , wherein at least one of the series of online-related actions involves providing protected information to a form on the at least one online resource, and wherein performing the at least one of the series of online-related actions includes the action model invoking an autofill feature of a browser to provide the protected information in a manner that isolates the planning model and the action model from raw authentication data.

9 . The method of claim 1 , wherein the series of online-related actions is associated with at least one task associated with the instruction and the indication includes information related to completing the at least one task.

10 . The method of claim 1 , wherein at least one of the planning model or the action model is trained based on a recording of actions associated with a task executed by the user via a browser.

11 . The method of claim 1 , wherein the series of online-related actions includes a first action, the instruction is a first instruction, and at least one of the planning model or the action model is trained, via supervised learning, using a labeled data set that includes a recording of the user executing a second action related to a second instruction.

12 . The method of claim 1 , wherein the sequence of low-level browser interactions comprises at least one of a click, a type, or a scroll.

13 . The method of claim 1 , wherein performing the series of online-related actions is executed as a background process, allowing a user to continue to interact with a browser.

14 . The method of claim 1 , wherein the content comprises at least one of text, an image, or a user interface structure of the at least one online resource.

15 . The method of claim 1 , wherein the planning model designates the subset of the series of online-related actions to be performed in parallel based on determining an absence of a dependency between the actions in the subset.

16 . A non-transitory computer-readable medium storing executable instructions that when executed by an electronic processor, cause the electronic processor to:

receive, via a first browser interface element, a prompt that includes an instruction;

determine, by a planning model, based on the instruction and browser data associated with a user, execution sequence of a series of online-related actions, wherein the execution sequence designates at least a subset of the series of online-related actions to be performed in parallel to accelerate execution of the instruction;

perform, via an action model and in accordance with the execution sequence, the series of online-related actions by processing content from at least one online resource to determine and execute a corresponding sequence of low-level browser interactions; and

provide, via a second browser interface element, an indication that the instruction is completed.

17 . The non-transitory computer-readable medium of claim 16 , wherein the execution sequence further designates a first action and a second action of the series of online-related actions to be performed in a series based on a determined dependency between the first action and the second action.

18 . A system comprising:

a browser having a first interface element and a second interface element;

a planning model;

an action model; and

an electronic processor configured to:

receive, via the first interface element, a prompt that includes an instruction;

determine, by the planning model, based on the instruction and browser data associated with a user, an execution sequence of a series of online-related actions, wherein the execution sequence designates at least a subset of the series of online-related actions to be performed in parallel to accelerate execution of the instruction;

perform, via the action model and in accordance with the execution sequence, the series of online-related actions by processing content from at least one online resource to determine and execute a corresponding sequence of low-level browser interactions; and

provide, via the second interface element, an indication that the instruction is completed.

19 . The system of claim 18 , wherein the planning model is further configured to retrieve history information that stores historical user interactions and preferences and use the history information to determine the execution sequence.

20 . The system of claim 18 , wherein the at least one online resource includes a form, and wherein the series of online-related actions includes providing data to the form using an autofill feature of the browser and receiving a response from the form.