Using hierarchical models and artificial intelligence (AI) to improve recognition of graphical objects

1 . A system comprising:

a microprocessor; and

a computer readable medium, coupled with the microprocessor and comprising microprocessor readable and executable instructions that, when executed by the microprocessor, cause the microprocessor to:

identify, using an Artificial Intelligence (AI) algorithm, a first graphical object in a graphical user interface, wherein the graphical user interface comprises a second graphical object that was not properly identified by the AI algorithm;

retrieve source code from a hierarchical model of the graphical user interface;

determine, based on the source code, if the first graphical object are a second graphical are the same first type;

in response to the first graphical object and the second graphical object being the same first type, compare an attribute of the first graphical object to an attribute of the second graphical object;

in response to the attribute of the first graphical object being the same as the attribute of the second graphical object, identify the second graphical object as being the same graphical object type as the first graphical object; and

in response to the identifying the second graphical object being the same graphical object type as the first graphical object, automatically generate and run a test script to exercise the second graphical object in the graphical user interface.

2 . The system of claim 1 , wherein the microprocessor readable and executable instructions further cause the microprocessor to:

identify, using the AI algorithm, a third graphical object in the graphical user interface, wherein the graphical user interface comprises a fourth graphical object that was not properly identified by the AI algorithm;

determine if the third graphical object and the fourth graphical object are being a same second type;

in response to the determining that the third graphical object and the fourth graphical object are the same second type, compare an attribute of the third graphical object to an attribute of the fourth graphical object; and

in response to the attribute of the third graphical object being the same as the attribute of the fourth graphical object, identifying the fourth graphical object as being the same graphical object type as the third graphical object.

3 . The system of claim 1 , wherein the attribute of the first graphical object comprises at least one of: a font size, a font type, a color, a shade, and a background.

4 . The system of claim 1 , wherein the hierarchical model is a Document Object Model (DOM).

5 . The system of claim 1 , wherein the first graphical object is one of: a text object, a text field object, a button object, menu object, a menu item object, a hyperlink, a slider object, a scroll bar object, a table object, and a tab object.

6 . The system of claim 1 ,

wherein the test script is associated with the graphical object type of the first graphical object.

7 . The system of claim 6 , wherein the test script to exercise the second graphical object automatically runs in response to the identifying the second graphical object being the same graphical object type as the first graphical object.

8 . The system of claim 1 , wherein an image of the second graphical object is used to retrain the AI algorithm.

9 . A method comprising:

identifying, by a microprocessor, using an Artificial Intelligence (AI) algorithm, a first graphical object in a graphical user interface, wherein the graphical user interface comprises a second graphical object that was not properly identified by the AI algorithm;

retrieving, by the microprocessor, source code from a hierarchical model of the graphical user interface;

determining, by the microprocessor, based on the source code, if the first graphical object and the second graphical object are a same first type;

in response to the first graphical object and the second graphical object being the same first type, comparing, by the microprocessor, an attribute of the first graphical object to an attribute of the second graphical object;

in response to the attribute of the first graphical object being the same as the attribute of the second graphical object, identifying, by the microprocessor, the second graphical object as being the same graphical object type as the first graphical object; and

in response to the identifying the second graphical object as being the same graphical object type as the first graphical object, automatically generating and running a test script to exercise the second graphical object in the graphical user interface.

10 . The method of claim 9 , further comprising:

identifying, using the AI algorithm, a third graphical object in the graphical user interface, wherein the graphical user interface comprises a fourth graphical object that was not properly identified by the AI algorithm;

determining if the third graphical object and the fourth graphical object are a same second type;

in response to the determining that the third graphical object and the fourth graphical object are the same second type, comparing an attribute of the third graphical object to an attribute of the fourth graphical object; and

in response to the attribute of the third graphical object being the same as the attribute of the fourth graphical object, identifying the fourth graphical object as being same graphical object type as the third graphical object.

11 . The method of claim 9 , wherein the attribute of the first graphical object comprises at least one of: a font size, a font type, a color, a shade, and a background.

12 . The method of claim 9 , wherein the hierarchical model is a Document Object Model (DOM).

13 . The method of claim 9 , wherein the first graphical object is one of: a text object, a text field object, a button object, menu object, a menu item object, a hyperlink, a slider object, a scroll bar object, a table object, and a tab object.

14 . The method of claim 9 ,

wherein the test script is associated with the graphical object type of the first graphical object.

15 . The method of claim 9 , wherein the second graphical object is used to retrain the AI algorithm.

16 . A non-transitory computer readable medium having stored thereon instructions that cause a microprocessor to:

identify, using an Artificial Intelligence (AI) algorithm, a first graphical object in a graphical user interface, wherein the graphical user interface comprises a second graphical object that was not properly identified by the AI algorithm;

retrieve source code from a hierarchical model of the graphical user interface;

determine, based on the source code, if the first graphical object and the second graphical object are a same first type;

in response to the first graphical object and the second graphical object being the same first type, compare an attribute of the first graphical object to an attribute of the second graphical object;

in response to the attribute of the first graphical object being the same as the attribute of the second graphical object, identify the second graphical object as being the same graphical object type as the first graphical object; and

in response to the identifying the second graphical object as being the same graphical object type as the first graphical object, automatically generate and run a test script to exercise the second graphical object in the graphical user interface.

17 . The non-transitory computer readable medium of claim 16 , wherein the instructions further cause the microprocessor to:

identify, using the AI algorithm, a third graphical object in the graphical user interface, wherein the graphical user interface comprises a fourth graphical object that was not properly identified by the AI algorithm;

determine if the third graphical object and the fourth graphical object are a same second object type;

in response to the determining that the third graphical object and the fourth graphical object are the same second object type, compare an attribute of the third graphical object to an attribute of the fourth graphical object; and

in response to the attribute of the third graphical object being the same type as the attribute of the fourth graphical object, identify the fourth graphical object as being the same graphical object type as the third graphical object.

18 . The non-transitory computer readable medium of claim 16 , wherein the attribute of the first graphical object comprises at least one of: a font size, a font type, a color, a shade, and a background.

19 . The non-transitory computer readable medium of claim 16 ,

wherein the test script is associated with the graphical object type of the first graphical object.