Workflow instruction interpretation for web task automation
View Patent ↗A method of executing a sequence of tasks includes receiving a natural language input indicative of the sequence of tasks. The natural language input may include a first task and a plurality of possible next tasks for the first task. The tasks may each be associated with a playback performance skeleton, indicative of a series of actions to be carried out on a web page. The series of action may have been generated, ahead of time, from a recorded performance of a similar task. The first task may be arranged to be performed. Then, based on a result of the performance of the first task, a successive task from among a plurality of possible next tasks associated with the result of performance of the first task may be selected. The successive task may then be arranged to be performed.
1. A computer-implemented method of arranging execution of natural language instructions, the method comprising:
receiving a natural language input indicative of the natural language instructions;
deriving, from the natural language input using a text analysis algorithm, information, the information including a plurality of action data, the plurality of action data including a key and a value corresponding to the key, the deriving including obtaining, from a database and based on the natural language input, the value;
interpreting the natural language input as a sequence of tasks including:
a first task defined using first action data among the plurality of action data; and
a conditional successive task:
defined using second action data among the plurality of action data; and
associated with a conditionality referencing a result returned from performance of the first task;
arranging performance of the first task at a headless browser;
determining that the result returned from the performance of the first task satisfies the conditionality, wherein the result includes, at least in part, an update message received from the headless browser, the update message indicating that an action has been performed and recorded within an object model; and
responsive to the determining, arranging performance of the conditional successive task at the headless browser.
2. The method of claim 1 , wherein the interpreting the natural language input comprises resolving ambiguous information in the natural language input.
3. The method of claim 2 , wherein the resolving comprises searching within a knowledge base to clarify the ambiguous information.
4. The method of claim 3 , wherein the knowledge base is stored on an external web hosting server.
5. The method of claim 3 , wherein the knowledge base is stored on a server hosting a playback engine.
6. The method of claim 5 , further comprising, based on the result returned from the performance of the first task, updating the knowledge base.
7. The method of claim 2 , wherein the resolving comprises using information retrieved from a search engine.
8. The method of claim 2 , wherein the resolving comprises generating a prompt on a user interface of an electronic device for providing additional information.
9. The method of claim 8 , wherein the prompt is a text entry field.
10. The method of claim 8 , wherein the prompt is a button.
11. The method of claim 8 , wherein the resolving further comprises receiving, via the user interface, additional information in response to the prompt.
12. A computer-implemented method of arranging execution of natural language instructions, the method comprising:
receiving a natural language input indicative of the natural language instructions;
deriving, from the natural language input, information, the information including:
a plurality of action data, the plurality of action data including a key and a value corresponding to the key, the deriving including using a text analysis algorithm to obtain, from a database and based on the natural language input, the value; and
a plurality of task data, the plurality of task data including task logic;
interpreting the plurality of task data as relating to a sequence of tasks including:
a first task defined according to first action data among the plurality of action data; and
a conditional successive task, wherein the conditional successive task is:
defined according to second action data among the plurality of action data; and
associated with the task logic, the task logic including a conditionality that references a future result to be returned from performance of the first task;
arranging performance of the first task at a headless browser, the arranging based on the first action data;
receiving, from the headless browser, an update message, the update message including a indication that an action has been performed and recorded within an object model, the update message forming at least part of an actual result returned from performance of the first task;
determining that the actual result returned from the performance of the first task satisfies the conditionality; and
responsive to the determining, arranging performance of the conditional successive task at the headless browser.