Web browsing robot system and method
A method for using a robot on the web is disclosed. The method may include assigning a goal to a robot. The robot may then direct a web browser to code corresponding to a URL. Using the code, the web browser may render a webpage comprising a plurality of rendered elements. The robot may identify each rendered element by using OCR or an OCR equivalent or by positioning a virtual mouse in a plurality of locations on the webpage and obtaining, from the code, element-identification information corresponding to each location. The robot may map each rendered elements with an element type selected from a closed set of element types stored within a knowledge base accessible by the robot. The robot may further select, from a set of possible actions, an action corresponding to each rendered element that is most likely to lead toward the goal and implement each such action.
1. A system comprising:
at least one processor; and
memory operably connected to the at least one processor, the memory storing a knowledge base and executable instructions programmed to cause the at least one processor to:
receive an original workflow referencing a plurality of webpages of a website, the original workflow including a plurality of original nodes in a specific order, including at least a start node and an end node, each node of the plurality of original nodes having a corresponding webpage of the plurality of webpages and a corresponding action to perform with respect to a portion of the corresponding webpage;
(a) determine, by a robot, that the original workflow is insufficient to reach the end node; and
in response to (a), autonomously add, by the robot, one or more new nodes to the original workflow to obtain an updated workflow, the one or more new nodes not included in the plurality of original nodes of the original workflow and each including a new action that is not included in the plurality of original nodes of the original workflow;
wherein the executable instructions, when executed by the at least one processor, cause the at least one processor to identify the one or more new nodes by:
directing a web browser to render, based on code corresponding to a URL, a webpage of the plurality of webpages, the webpage comprising a plurality of rendered elements;
identifying each rendered element of one or more rendered elements of the plurality of rendered elements;
selecting, from a set of possible actions, an action corresponding to each rendered element of the one or more rendered elements indicated by the original workflow;
implementing the action corresponding to each rendered element of the one or more rendered elements, the action comprising one of autonomously filling forms and autonomously interacting with at least one of a button, checkbox, and pick list;
(b) determining that the action is effective to advance from one webpage of the plurality of webpages to another webpage of the plurality of webpages; and
In response to (b) selecting the action as the new action of a new node of the one or more new nodes.
2. The system of claim 1 , wherein the executable instructions are further programmed to cause the at least one processor to select, from the set of possible actions, the action corresponding to each rendered element of the one or more rendered elements that will lead to the end node:
selecting the action from the set of possible actions using a learning cell.
3. The system of claim 2 , wherein the executable instructions are further programmed to cause the at least one processor to update the learning cell according to a result of the action.
4. The system of claim 3 , wherein the learning cell includes a machine learning model.
5. The system of claim 3 , wherein the executable instructions are further programmed to cause the at least one processor to select, from the set of possible actions, the action corresponding to each rendered element of the one or more rendered elements that will lead to the end node by:
selecting the action as corresponding to a first node in a node map corresponding to the URL and likely to advance to a webpage corresponding to a second node in the node map connected to the first node.
6. The system of claim 1 , wherein the executable instructions are further programmed to cause the at least one processor to implement the original workflow without intervention by a human user.
7. A system comprising:
at least one processor; and
memory operably connected to the at least one processor, the memory storing a executable instructions that, when executed by one or more processors, cause the one or more processors to:
receive, with a robot, an original workflow, containing a plurality of original nodes including at least a start node and an end node, each node of the plurality of original nodes referencing a portion of one or more webpages corresponding to each node and an action to perform with respect to the portion of the one or more webpages corresponding to each node;
(a) determine, by the robot, that the original workflow is insufficient to reach the end node; and
in response to (a), autonomously add, by the robot, one or more new nodes to the original workflow to obtain an updated workflow, the one or more new nodes not included in the plurality of original nodes of the original workflow and each including a new action that is not included in the plurality of original nodes of the original workflow;
wherein the executable instructions, when executed by one or more processors, further cause the one or more processors to add the one or more new nodes to the original workflow by:
selecting the one or more new nodes using a knowledge base;
implementing the new action of each node of the one or more new nodes;
(b) determining that updated workflow is sufficient to reach the end node of the original workflow; and
in response to (b), updating the knowledge base to include the updated workflow.
8. The system of claim 7 , wherein the executable instructions, when executed by one or more processors, further cause the one or more processors to:
(a) determine that the original workflow is insufficient to reach the end node by determining that a first node of the plurality of original nodes is not found in the one or more webpages; and
add, by the robot, the one or more new nodes to the original workflow to obtain the updated workflow by adding the one or more new nodes in place of the first node.
9. The system of claim 7 , wherein the knowledge base includes a node map, the original workflow being a portion of the node map; and
wherein updating the knowledge base to include the updated workflow includes adding the updated workflow to the node map.
10. The system of claim 7 , wherein updating the knowledge base further comprises using machine learning.
11. The system of claim 7 , wherein the new action of at least one of the one or more new nodes includes opening a connection to an external data source.
12. The system of claim 7 , wherein the new action of at least one node of the one or more new nodes includes autonomously interacting with an element of the one or more webpages, the element including at least one of a button, checkbox, and pick list.