Automatic update of user interface element identifiers for software artifact tests

1 . A computing system comprising:

at least one memory;

one or more hardware processor units coupled to the at least one memory; and

one or more computer readable storage media storing computer-executable instructions that, when executed, cause the computing system to perform operations comprising:

executing a first instance of a software test on a software artifact, the software test comprising one or more operations to manipulate a user interface element or to confirm content of a user interface element defined in code of a first version of the software artifact;

determining that a first operation of the one or more operations failed, being a failed operation;

determining an identifier assigned to a first user interface element associated with the failed operation;

analyzing code for a second version of the software artifact or analyzing a screenshot of a user interface associated with execution of the failed operation to determine a second user interface element corresponding to the first user interface element, wherein determining the second user interface element comprises at least one of:

(i) comparing a first numerical representation of text in an embedding space for text of the identifier assigned to the first user interface element and a second numerical representation of text in the embedding space for text of an identifier of a candidate second user interface element to determine whether the identifier assigned to the first user interface element and the identifier of the candidate second user interface element are semantically equivalent; and

(ii) providing the identifier assigned to the first user interface element and the identifier of the candidate second user interface element to a natural language inference model to obtain respective outputs and determining, based on the outputs, whether the identifier assigned to the first user interface element entails the identifier of the candidate second user interface element, the entailment indicating semantic equivalence between the identifiers; and

modifying the software test to specify the second user interface element in the first operation in place of the first user interface element.

2 . The computing system of claim 1 , wherein analyzing the code for the second version of the software artifact or the screenshot of the user interface associated with the execution of the failed operation comprises:

determining the code for the second version of the software artifact is not available; and

in response to determining the code for the second version of the software artifact is not available, analyzing the screenshot.

3 . The computing system of claim 1 , wherein analyzing the code for the second version of the software artifact or the screenshot of the user interface associated with the execution of the failed operation comprises analyzing the code for the second version of the software artifact.

4 . The computing system of claim 3 , wherein analyzing the code for the second version of the software artifact comprises:

scanning the code for the second version of the software artifact for identifiers of one or more user interface elements other than the first user interface element; and

determining that an identifier of the second user interface element is semantically equivalent to the identifier of the first user interface element.

5 . The computing system of claim 4 , wherein determining that the identifier of the second user interface element is semantically equivalent to the identifier of the first user interface element comprises:

comparing the first numerical representation of text and the second numerical representation of text; and

determining that a result of the comparing the first and the second numerical representations of text satisfies a threshold.

6 . The computing system of claim 4 , wherein the identifier of the second user interface element comprises a user interface class, a user interface identifier, an accessibility identifier, or text content assigned to the first user interface element.

7 . The computing system of claim 4 , the operations further comprising:

determining a first section of the code of the first version of the software artifact in which the identifier of the first user interface is located;

wherein determining an identifier in the second version of the code for the software artifact or the screenshot for the second user interface element corresponding to the first user interface element comprises scanning a corresponding section of code for the second version of the software artifact corresponding to the first section of the code.

8 . The computing system of claim 7 , wherein the first section of the code is defined as a level of HTML code, or a DOM generated from the HTML code.

9 . The computing system of claim 4 , wherein determining that the identifier of the second user interface element is semantically equivalent to the identifier of the first user interface element comprises:

providing the identifier of the first user interface element to the natural language inference model to provide a first result;

providing the identifier of the second user interface element to the natural language inference model to provide a second result;

comparing the first result with the second result; and

determining that the first result entails the second result.

10 . The computing system of claim 1 , wherein analyzing the code for the second version of the software artifact or the screenshot of the user interface associated with the execution of the failed operation comprises analyzing the screenshot, the screenshot being a second screenshot generated by execution of the code for the second version of the software artifact.

11 . The computing system of claim 10 , the operations further comprising:

comparing the second screenshot to a first screenshot generated by execution of the code for the first version of the software artifact to extract identifiers of one or more user interface elements other than the first user interface element; and

determining that an identifier of the second user interface element is semantically equivalent to the identifier of the first user interface element.

12 . The computing system of claim 11 , the operations further comprising:

extracting identifiers of user interface elements represented in the second screenshot;

comparing the first and second text numerical representations of text; and

determining that a result of the comparing the first and the second numerical representations of text-satisfies a threshold.

13 . The computing system of claim 11 , the operations further comprising:

determining a section of the first screenshot in which the identifier of the first user interface is located in the first version of the code for the software artifact;

wherein determining an identifier in the second version of the code for the software artifact or the screenshot for the second user interface element corresponding to the first user interface element comprises scanning a section of the second screenshot corresponding to the section of the first screenshot.

14 . The computing system of claim 11 , wherein determining that the identifier of the second user interface element is semantically equivalent to the identifier of the first user interface element comprises:

providing the identifier of the first user interface element to the natural language inference model to provide a first result;

providing the identifier of the second user interface element to the natural language inference model to provide a second result;

comparing the first result with the second result; and

determining that a result of the comparing the first result with the second result satisfies a threshold.

15 . The computing system of claim 14 , wherein determining that the result of the comparison satisfies the threshold comprises determining that the first result entails the second result.

16 . The computing system of claim 1 , the operations further comprising:

identifying one or more tests that comprise one or more operations to manipulate the first user interface element or to confirm content of the first user interface element; and

modifying at least one of the one or more tests to specify the second user interface element in the first operation in place of the first user interface element.

17 . The computing system of claim 1 , the operations further comprising:

in response to determining that the first operation of the one or more operations failed, determining whether the software artifact is an expected software artifact by comparing identifiers for one or more user interface elements proximate the user interface in the software artifact to a reference version of the expected software artifact.

18 . The computing system of claim 17 , wherein the comparing the identifiers use code of the software artifact and code of the reference version of the expected software artifact.

19 . A method, implemented in a computing system comprising at least one hardware processor and at least one memory coupled to the at least one hardware processor, the method comprising:

executing a first instance of a software test on a software artifact, the software test comprising one or more operations to manipulate a user interface element or to confirm content of a user interface element defined in code of a first version of the software artifact;

determining that a first operation of the one or more operations failed, being a failed operation;

determining an identifier assigned to a first user interface element associated with the failed operation;

analyzing code for a second version of the software artifact or analyzing a screenshot of a user interface associated with execution of the failed operation to determine a second user interface element corresponding to the first user interface element, wherein determining the second user interface element comprises at least one of:

(i) comparing a first numerical representation of text in an embedding space for text of the identifier assigned to the first user interface element and a second numerical representation of text in the embedding space for text of an identifier of a candidate second user interface element to determine whether the identifier assigned to the first user interface element and the identifier of the candidate second user interface element are semantically equivalent; and

(ii) providing the identifier assigned to the first user interface element and the identifier of the candidate second user interface element to a natural language inference model to obtain respective outputs and determining, based on the outputs, whether the identifier assigned to the first user interface element entails the identifier of the candidate second user interface element, the entailment indicating semantic equivalence between the identifiers; and

modifying the software test to specify the second user interface element in the first operation in place of the first user interface element.

20 . One or more computer-readable storage media comprising:

computer-executable instructions that, when executed by a computing system comprising at least one hardware processor and at least one memory coupled to the at least one hardware processor, cause the computing system to execute a first instance of a software test on a software artifact, the software test comprising one or more operations to manipulate a user interface element or to confirm content of a user interface element defined in code of a first version of the software artifact;

computer-executable instructions that, when executed by the computing system, cause the computing system to determine that a first operation of the one or more operations failed, being a failed operation;

computer-executable instructions that, when executed by the computing system, cause the computing system to determine an identifier assigned to a first user interface element associated with the failed operation;

computer-executable instructions that, when executed by the computing system, cause the computing system to analyze code for a second version of the software artifact or to analyze a screenshot of a user interface associated with execution of the failed operation to determine a second user interface element corresponding to the first user interface element, wherein determining the second user interface element comprises at least one of:

(i) comparing a first numerical representation of text in an embedding space for text of the identifier assigned to the first user interface element and a second numerical representation of text in the embedding space for text of an identifier of a candidate second user interface element to determine whether the identifier assigned to the first user interface element and the identifier of the candidate second user interface element are semantically equivalent; and

(ii) providing the identifier assigned to the first user interface element and the identifier of the candidate second user interface element to a natural language inference model to obtain respective outputs and determining, based on the outputs, whether the identifier assigned to the first user interface element entails the identifier of the candidate second user interface element, the entailment indicating semantic equivalence between the identifiers; and

computer-executable instructions that, when executed by the computing system, cause the computing system to modify the software test to specify the second user interface element in the first operation in place of the first user interface element.