Robotic process automation (RPA) debugging systems and methods
In some embodiments, a robotic process automation (RPA) robot is configured to identify a runtime target of an automation activity (e.g., a button to click, a form field to fill in, etc.) by searching a user interface for a UI element matching a set of characteristics of the target defined at design-time. When the target identification fails, some embodiments display an error message indicating which target characteristic could not be matched. Some embodiments further display for selection by the user a set of alternative target elements of the runtime interface.
1. A method of debugging a software robot configured to apply a robotic process automation (RPA) activity to a target element of a user interface (UI), the RPA activity mimicking an interaction of a human operator with the target element, the method comprising employing at least one hardware processor of a computer system to wherein
in response to receiving an RPA script indicating a plurality of characteristics of the target element, searching the UI for UI elements matching all of the plurality of characteristics of the target element; and
in response to searching the UI, when no element of the UI matches all of the plurality of characteristics of the target element:
displaying an error message to a user of the computer system, the error message selectively displaying at least one characteristic of the target element that is not matched in any UI element of the UI,
determining a candidate element of the UI according to a similarity between the candidate element and the target element,
presenting the candidate element to the user for selection as an alternative target for the RPA activity, and
in response to a user input selecting the candidate element as alternative target, applying the RPA activity to the candidate element, and
in response to a user input selecting the candidate element as alternative target, applying the RPA activity to the candidate element.
2. The method of claim 1 , wherein determining the candidate element comprises selecting the candidate element from UI elements matching a subset of the plurality of characteristics of the target element.
3. The method of claim 1 , wherein the plurality of characteristics comprises an ordered sequence of tags, each tag of the ordered sequence indicating a distinct node of a tree representation of the UI.
4. The method of claim 3 , wherein determining the candidate element comprises:
selecting a node of the tree representation of the UI according to whether a tag representing the selected node belongs to the ordered sequence of tags; and
selecting the candidate element from a subtree of the selected node.
5. The method of claim 3 , wherein:
the at least one characteristic comprises a selected tag of the ordered sequence of tags; and
the error message further displays an indicator of a position of the selected tag within the ordered sequence of tags.
6. The method of claim 1 , wherein presenting the candidate element to the user further comprises displaying a similarity measure quantifying the similarity between the candidate element and the target element.
7. The method of claim 1 , wherein presenting the candidate element for selection comprises presenting to the user a plurality of candidate in an order determined according to a similarity between each member of the plurality of candidate elements and the target element.
8. The method of claim 1 , wherein presenting the candidate element to the user further comprises grabbing an image of a region of the UI showing the candidate element and displaying the image to the user as part of the error message.
9. The method of claim 1 , wherein the target element comprises an item selected from a group consisting of a UI window, a menu, a button, a text area, and a form field.
10. A computer system comprising at least one hardware processor configured to debug a software robot configured to apply a robotic process automation (RPA) activity to a target element of a user interface (UI), the RPA activity mimicking an interaction of a human operator with the target element, wherein debugging the software robot comprises:
in response to receiving an RPA script indicating a plurality of characteristics of the target element, searching the UI for UI elements matching all of the plurality of characteristics of the target element; and
in response to searching the UI, when no element of the UI matches all of the plurality of characteristics of the target element:
displaying an error message to a user of the computer system, the error message selectively displaying at least one characteristic of the target element that is not matched in any UI element of the UI,
determining a candidate element of the UI according to a similarity between the candidate element and the target element,
presenting the candidate element to the user for selection as an alternative target for the RPA activity, and
in response to a user input selecting the candidate element as alternative target, applying the RPA activity to the candidate element.
11. The computer system of claim 10 , wherein determining the candidate element comprises selecting the candidate element from UI elements matching a subset of the plurality of characteristics of the target element.
12. The computer system of claim 10 , wherein the plurality of characteristics comprises an ordered sequence of tags, each tag of the ordered sequence indicating a distinct node of a tree representation of the UI.
13. The computer system of claim 12 , wherein determining the candidate element comprises:
selecting a node of the tree representation of the UI according to whether a tag representing the selected node belongs to the ordered sequence of tags; and
selecting the candidate element from a subtree of the selected node.
14. The computer system of claim 12 , wherein:
the at least one characteristic comprises a selected tag of the ordered sequence of tags, the selected tag not matched in any UI element of the UI; and
the error message further displays an indicator of a position of the selected tag within the ordered sequence of tags.
15. The computer system of claim 10 , wherein presenting the candidate element to the user further comprises displaying to the user a similarity measure quantifying the similarity between the candidate element and the target element.
16. The computer system of claim 10 , wherein presenting the candidate target for selection comprises presenting to the user a plurality of candidate elements including the candidate element, the plurality of candidate elements presented in an order determined according to a similarity between each member of the plurality of candidate elements and the target element.
17. The computer system of claim 10 , wherein presenting the candidate element to the user further comprises grabbing an image of a region of the UI showing the candidate element and displaying the image to the user as part of the error message.
18. The computer system of claim 10 , wherein the target element comprises an item selected from a group consisting of a UI window, a menu, a button, a text area, and a form field.
19. A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to debug a software robot configured to apply a robotic process automation (RPA) activity to a target element of a user interface (UI), the RPA activity mimicking an interaction of a human operator with the target element, wherein debugging the software robot comprises:
in response to receiving an RPA script indicating a plurality of characteristics of the target element, searching the UI for UI elements matching all of the plurality of characteristics of the target element; and
in response to searching the UI, when no element of the UI matches all of the plurality of characteristics of the target element:
displaying an error message to a user of the computer system, the error message selectively displaying at least one characteristic of the target element that is not matched in any UI element of the UI,
determining a candidate element of the UI according to a similarity between the candidate element and the target element,
presenting the candidate element to the user for selection as an alternative target for the RPA activity, and
in response to a user input selecting the candidate element as alternative target, applying the RPA activity to the candidate element.