Human interactions with artificial intelligence (AI)-based testing tool
Embodiments of the disclosure provide systems and methods for performing functional testing on an application using artificial intelligence. According to one embodiment, a method for performing functional testing on an Application Under Test (AUT) can comprise maintaining, by a processor of an edge computing device, a model comprising a script defining one or more automatic tests for the AUT and trained data identifying objects of a user interface of the AUT. One or more tests can be executed on the AUT by the processor of the edge computing device based on the script and the trained data identifying the objects of the user interface of the AUT. Results of executing the one or more tests on the AUT can be provided by the processor of the edge computing device.
1. A method for performing functional testing on an Application Under Test (AUT), the method comprising:
maintaining, by a processor of an edge computing device, an Artificial Intelligence (AI)-based model comprising a script defining one or more automatic tests for the AUT and trained data identifying objects of a user interface of the AUT based on a graphical appearance of each identified object;
executing, by the processor of the edge computing device, the one or more automatic tests on the AUT based on the script and the trained data identifying the objects of the user interface of the AUT based on the graphical appearance of each identified object, wherein executing the one or more automatic tests on the AUT comprises performing a two-level caching process on the user interface of the AUT, the two-level caching process based on a hash of an image of the user interface of the AUT, a hash of a description of the image, and a hash of one or more areas of interest in the image, and wherein the two-level caching process comprises receiving the image of the user interface of the AUT, hashing the received image using a combination hash, the combination hash comprising a hash of the image using a permissive hashing function producing results in which matches between hashed data are found when the data is less than an exact match and a hash of the description of the image using a strict hashing function producing results in which matches between hashed data are found when the data is an exact match, hashing each of the one or more areas of interest in the received image using the strict hashing function, and clustering the received image with one or more clusters in a cache based on the hash of each of the one or more areas of interest in the image using the second hashing function; and
providing, by the processor of the edge computing device, results of executing the one or more automatic tests on the AUT.
2. The method of claim 1 , further comprising updating, by the processor of the edge computing device, the model based on the results of executing the one or more automatic tests on the AUT.
3. The method of claim 1 , wherein executing the one or more automatic tests on the AUT comprises:
determining, by the processor of the edge computing device, whether the user interface of the AUT is currently changing; and
in response to determining the user interface of the AUT is currently changing, waiting, by the processor of the edge computing device, before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the user interface of the AUT is not currently changing, proceeding, by the processor of the edge computing device, with execution of the one or more automatic tests on the AUT.
4. The method of claim 1 , wherein executing the one or more automatic tests on the AUT comprises:
determining, by the processor of the edge computing device, whether an object identified by the model is present in the user interface of the AUT based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the object identified by the model is present in the user interface of the AUT, waiting, by the processor of the edge computing device, before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the object identified by the model is not present in the user interface of the AUT, proceeding, by the processor of the edge computing device, with execution of the one or more automatic tests on the AUT.
5. The method of claim 1 , wherein executing the one or more automatic tests on the AUT comprises:
determining, by the processor of the edge computing device, whether text of the user interface of the AUT indicates waiting based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the text of the user interface of the AUT indicates waiting, waiting, by the processor of the edge computing device, before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the text of the user interface of the AUT does not indicate waiting, proceeding, by the processor of the edge computing device, with execution of the one or more automatic tests on the AUT.
6. The method of claim 1 , wherein maintaining the model comprises training the trained data using a development mock-up of the user interface of the AUT.
7. A computing device comprising:
a processor; and
a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to perform functional testing on an Application Under Test (AUT) by:
maintaining an Artificial Intelligence (AI)-based model comprising a script defining one or more automatic tests for the AUT and trained data identifying objects of a user interface of the AUT based on a graphical appearance of each identified object;
executing the one or more automatic tests on the AUT based on the script and the trained data identifying the objects of the user interface of the AUT based on the graphical appearance of each identified object, wherein executing the one or more automatic tests on the AUT comprises performing a two-level caching process on the user interface of the AUT, the two-level caching process based on a hash of an image of the user interface of the AUT, a hash of a description of the image, and a hash of one or more areas of interest in the image, and wherein the two-level caching process comprises receiving the image of the user interface of the AUT, hashing the received image using a combination hash, the combination hash comprising a hash of the image using a permissive hashing function producing results in which matches between hashed data are found when the data is less than an exact match and a hash of the description of the image using a strict hashing function producing results in which matches between hashed data are found when the data is an exact match, hashing each of the one or more areas of interest in the received image using the strict hashing function, and clustering the received image with one or more clusters in a cache based on the hash of each of the one or more areas of interest in the image using the second hashing function; and
providing results of executing the one or more automatic tests on the AUT.
8. The computing device of claim 7 , wherein the instructions further cause the processor to update the model based on the results of executing the one or more automatic tests on the AUT.
9. The computing device of claim 7 , wherein executing the one or more automatic tests on the AUT comprises:
determining whether the user interface of the AUT is currently changing; and
in response to determining the user interface of the AUT is currently changing, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the user interface of the AUT is not currently changing, proceeding with execution of the one or more automatic tests on the AUT.
10. The computing device of claim 7 , wherein executing the one or more automatic tests on the AUT comprises:
determining, whether an object identified by the model is present in the user interface of the AUT based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the object identified by the model is present in the user interface of the AUT, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the object identified by the model is not present in the user interface of the AUT, proceeding with execution of the one or more automatic tests on the AUT.
11. The computing device of claim 7 , wherein executing the one or more automatic tests on the AUT comprises:
determining whether text of the user interface of the AUT indicates waiting based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the text of the user interface of the AUT indicates waiting, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the text of the user interface of the AUT does not indicate waiting, proceeding with execution of the one or more automatic tests on the AUT.
12. The computing device of claim 7 , wherein maintaining the model comprises training the trained data using a development mock-up of the user interface of the AUT.
13. A non-transitory, computer-readable medium comprising a set of instructions stored therein which, when executed by a processor, causes the processor to perform functional testing on an Application Under Test (AUT) by:
maintaining an Artificial Intelligence (AI)-based model comprising a script defining one or more automatic tests for the AUT and trained data identifying objects of a user interface of the AUT based on a graphical appearance of each identified object;
executing the one or more automatic tests on the AUT based on the script and the trained data identifying the objects of the user interface of the AUT based on a graphical appearance of each identified object, wherein executing the one or more automatic tests on the AUT comprises performing a two-level caching process on the user interface of the AUT, the two-level caching process based on a hash of an image of the user interface of the AUT, a hash of a description of the image, and a hash of one or more areas of interest in the image, and wherein the two-level caching process comprises receiving the image of the user interface of the AUT, hashing the received image using a combination hash, the combination hash comprising a hash of the image using a permissive hashing function producing results in which matches between hashed data are found when the data is less than an exact match and a hash of the description of the image using a strict hashing function producing results in which matches between hashed data are found when the data is an exact match, hashing each of the one or more areas of interest in the received image using the strict hashing function, and clustering the received image with one or more clusters in a cache based on the hash of each of the one or more areas of interest in the image using the second hashing function; and
providing results of executing the one or more automatic tests on the AUT.
14. The non-transitory, computer-readable medium of claim 13 , wherein the instructions further cause the processor to update the model based on the results of executing the one or more automatic tests on the AUT.
15. The non-transitory, computer-readable medium of claim 13 , wherein executing the one or more automatic tests on the AUT comprises:
determining whether the user interface of the AUT is currently changing; and
in response to determining the user interface of the AUT is currently changing, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the user interface of the AUT is not currently changing, proceeding with execution of the one or more automatic tests on the AUT.
16. The non-transitory, computer-readable medium of claim 13 , wherein executing the one or more automatic tests on the AUT comprises:
determining, whether an object identified by the model is present in the user interface of the AUT based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the object identified by the model is present in the user interface of the AUT, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the object identified by the model is not present in the user interface of the AUT, proceeding with execution of the one or more automatic tests on the AUT.
17. The non-transitory, computer-readable medium of claim 13 , wherein executing the one or more automatic tests on the AUT comprises:
determining whether text of the user interface of the AUT indicates waiting based on the trained data identifying objects of the user interface of the AUT; and
in response to determining the text of the user interface of the AUT indicates waiting, waiting before proceeding with execution of the one or more automatic tests on the AUT; and
in response to determining the text of the user interface of the AUT does not indicate waiting, proceeding with execution of the one or more automatic tests on the AUT.