Autonomous generation and self-healing of web automation scripts

1 . A method comprising:

generating, with a large language model (LLM), an action list for performing an extraction task from at least one website, wherein generating the action list comprises for each of a plurality of actions in the action list:

capturing current webpage data comprising a screenshot of a current webpage and a set of interactable widgets on the current webpage,

obtaining, using an LLM processing the current webpage data, a next action attribute set for a next action to perform extraction task, the next action being in the plurality of actions,

adding the next action attribute set to the action list,

performing the next action according to the next action attribute set;

traversing the action list to filter out at least one superfluous action;

generating, after filtering, a set of machine executable instructions from the action list;

storing the set of machine executable instructions associated with a task identifier of the extraction task; and

executing the set of machine executable instructions to extract a data element from the at least one website.

2 . The method of claim 1 , further comprising:

receiving a request to perform the extraction task;

retrieving the set of machine executable instructions using the task identifier of the extraction task; and

responding to the request with the data element.

3 . The method of claim 2 , further comprising:

while traversing the action list, identifying a sequence of states when performing the plurality of actions,

wherein filtering out at least one superfluous actions comprises determining a subsequence of the sequence of states that are unchanged, wherein the at least one superfluous action corresponds to the subsequence.

4 . The method of claim 1 , further comprising:

executing a headless browser with the machine executable instructions to obtain the data element according to the extraction task.

5 . The method of claim 1 , further comprising:

periodically testing the set of machine executable instructions; and

regenerating the set of machine executable instructions when testing results in failure.

6 . The method of claim 1 , wherein periodically testing comprises:

transmitting, responsive to an expiration of a time limit, a test input to an orchestrator;

executing the set of machine executable instructions using the test input to generate a test output; and

determining whether failure of the test occurs according to the test output.

7 . The method of claim 6 , wherein executing the set of machine executable instructions comprises:

generating, with the LLM, a revised action list for performing the extraction task, wherein generating the revised action list comprises:

capturing the current webpage data of a current webpage,

prompting the LLM, with the current webpage data and a current next action attribute set of a current next action in the action list, to generate a revised next action attribute set for a revised next action to perform the extraction task,

comparing the revised next action attribute set to the current next action attribute set,

updating the current next action attribute set in the revised action list when the revised next action deviates from the current next action, and

performing the revised next action according to the next action attribute set.

8 . The method of claim 6 , further comprising:

executing, by the orchestrator, a headless browser with a set of machine executable instructions and the test input as input to obtain the test output from the test input.

9 . The method of claim 1 , wherein the set of machine executable instructions comprises at least one of a data file listing a series of steps and code defining the series of steps.

10 . A system comprising:

one or more computer processors;

a builder executing on the one or more computer processors and configured for:

generating, with a large language model (LLM), an action list for performing an extraction task from at least one website, wherein generating the action list comprises for each of a plurality of actions in the action list:

capturing current webpage data comprising a screenshot of a current webpage and a set of interactable widgets on the current webpage,

obtaining, using an LLM processing the current webpage data, a next action attribute set for a next action to perform extraction task, the next action in the plurality of actions,

adding the next action attribute set to the action list,

performing the next action according to the next action attribute set, and

generating a set of machine executable instructions from the action list;

an extraction data storage storing the set of machine executable instructions associated with a task identifier of the extraction task; and

an orchestrator executing on one or more computer processors and configured for:

receiving a request to perform the extraction task,

retrieving the set of machine executable instructions using the task identifier of the extraction task,

executing the set of machine executable instructions to extract a data element from the at least one website, and

responding to the request with the data element.

11 . The system of claim 10 , wherein the builder comprises an instruction generator configured for:

traversing the action list to filter out at least one superfluous action prior to generating the machine executable instructions from the action list.

12 . The system of claim 11 , wherein the builder comprises the instruction generator configured for:

while traversing the action list, identifying a sequence of states when performing the plurality of actions,

wherein filtering out at least one superfluous actions comprises determining a subsequence of the sequence of states that are unchanged, wherein the at least one superfluous action corresponds to the subsequence.

13 . The system of claim 10 , further comprising:

a headless browser executing the machine executable instructions to obtain the data element according to the extraction task.

14 . The system of claim 10 , further comprising:

a tester executing on the one or more computer processors for:

periodically testing the set of machine executable instructions, and

regenerating the set of machine executable instructions when testing results in failure.

15 . The system of claim 10 , wherein periodically testing comprises:

transmitting, responsive to an expiration of a time limit, a test input to an orchestrator;

executing the set of machine executable instructions using the test input to generate a test output; and

determining whether failure of the test occurs according to the test output.

16 . The system of claim 15 , wherein executing the set of machine executable instructions comprises:

generating, with the LLM, a revised action list for performing the extraction task, wherein generating the revised action list comprises:

capturing the current webpage data of a current webpage,

prompting the LLM, with the current webpage data and a current next action attribute set of a current next action in the action list, to generate a revised next action attribute set for a revised next action to perform the extraction task,

comparing the revised next action attribute set to the current next action attribute set,

updating the current next action attribute set in the revised action list when the revised next action deviates from the current next action,

performing the revised next action according to the next action attribute set.

17 . The system of claim 15 , wherein the orchestrator is further for:

executing a headless browser with a set of machine executable instructions and the test input as input to obtain the test output from the test input.

18 . A method comprising:

periodically testing a set of machine executable instructions to extract a data element from at least one webpage; and

regenerating the set of machine executable instructions when testing results in failure, wherein regenerating the set of machine executable instructions comprises:

generating, with a large language model (LLM), a revised action list for performing an extraction task, wherein generating the revised action list comprises:

capturing current webpage data of a current webpage,

prompting the LLM, with the current webpage data and a current next action attribute set of a current next action in an action list, to generate a revised next action attribute set for a revised next action to perform the extraction task,

comparing the revised next action attribute set to the current next action attribute set,

updating the current next action attribute set in the revised action list when the revised next action deviates from the current next action, and

performing the revised next action according to the revised next action attribute set.

19 . The method of claim 18 , wherein periodically testing comprises:

transmitting, responsive to an expiration of a time limit, a test input to an orchestrator;

executing the set of machine executable instructions using the test input to generate a test output; and

determining whether failure of the test occurs according to the test output.

20 . The method of claim 18 , further comprising:

executing a headless browser with a set of machine executable instructions and a test input as input to obtain a test output from the test input.