IP Library Granted Patent US 10,866,883
Granted Patent B2
US 10,866,883 · App. 16/204,158 · Granted Dec 15, 2020

Detection of graphical user interface element of application undergoing functional testing

Inventors: Avishay Ben Shabtai (Holon, IL); Dror Saaroni (Yahud, IL); Vika Milgrom (Yehud, IL); Anton Kaminsky (Yehud, IL); Yossi Rachelson (Petach Tikwa, IL)
Assignee: MICRO FOCUS LLC
G06F11/3664G06F9/451G06F11/261G06F11/3688G06F11/3692
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,866,883
App. No.
16/204,158
Granted
Dec 15, 2020
Kind
B2
Abstract

A script directing functional testing of an application specifies values for properties of a specific graphical user interface (GUI) and specifies an image of the specific GUI element. A computing device detects whether the application has displayed the specific GUI element by evaluating the values for the properties of the specific GUI element against GUI elements that the application has displayed. If unsuccessful, the computing device detects whether the application has displayed the specific GUI element by evaluating the image of the specific GUI element against a screen image including the GUI elements that the application has displayed.

Claims (40)

1. A method comprising:

receiving, by a computing device functionally testing an application, values for a plurality of properties and an image of a specific graphical user interface (GUI) element, from a script directing functional testing of the application;

detecting, by the computing device, whether the application has displayed the specific GUI element by evaluating the received values for the properties of the specific GUI element against a plurality of GUI elements that the application has displayed;

in response to unsuccessfully detecting that the application has displayed the specific GUI element by evaluating the received values for the properties of the specific GUI element against the GUI elements that the application has displayed, detecting, by the computing device, whether the application has displayed the specific GUI element by:

attempting to locate the received image of the specific GUI element within the screen image including the GUI elements that the application has displayed;

upon successfully locating the received image of the specific GUI element within the screen image, identifying a displayed GUI element at a location within the screen image at which the received image of the specific GUI element has been located;

in response to unsuccessfully successfully locating the received image of the specific GUI element within the screen image, failing and prematurely terminating, by the computing device, the functional testing of the application; and

in response to successfully detecting that the application has displayed the specific GUI element, continuing the functional testing of the application, by the computing device.

2. The method of claim 1 , wherein responsive to successfully detecting that the application has displayed the specific GUI element by evaluating the received values for the properties of the specific GUI element against the GUI elements that the application has displayed, the functional testing of the application is continued without having to use the image of the specific GUI element to detect that the application has displayed the specific GUI element.

3. The method of claim 1 , wherein detecting whether the application has displayed the specific GUI element by evaluating the received image of the specific GUI element against the screen image including the GUI elements that the application has displayed further comprises:

upon identifying the displayed GUI element, evaluating the received values of a sub-plurality of the properties of the specific GUI element against the identified displayed GUI element.

4. The method of claim 3 , wherein evaluating the received values of the sub-plurality of the properties of the specific GUI element against the identified displayed GUI element comprises:

determining whether the identified displayed GUI element has a same value for each property of the sub-plurality of the properties as the specific GUI element does.

5. The method of claim 4 , wherein evaluating the received values of the sub-plurality of the properties of the specific GUI element against the identified displayed GUI element further comprises:

in response to determining that the identified displayed GUI element does not have the same value for each property of the sub-plurality of the properties as the specific GUI element does, concluding that detecting whether the application has displayed the specific GUI element by evaluating the received image of the specific GUI element against the screen image including the GUI elements that the application has displayed has been unsuccessful.

6. The method of claim 4 , wherein evaluating the received values of the sub-plurality of the properties of the specific GUI element against the identified displayed GUI element further comprises:

in response to determining that the identified displayed GUI element has the same value for each property of the sub-plurality of the properties as the specific GUI element does, concluding that detecting whether the application has displayed the specific GUI element by evaluating the received image of the specific GUI element against the screen image including the GUI elements that the application has displayed has been successful.

7. The method of claim 4 , wherein evaluating the received values of the sub-plurality of the properties of the specific GUI element against the identified displayed GUI element further comprises:

in response to determining that the identified displayed GUI element has the same value for each property of the sub-plurality of the properties as the specific GUI element does, modifying the script to specify that the specific GUI element is to be detected during subsequent functional testing by the sub-plurality of the properties instead of by the plurality of the properties.

8. The method of claim 1 , wherein detecting whether the application has displayed the specific GUI element by evaluating the received image of the specific GUI element against the screen image including the GUI elements that the application has displayed further comprises:

upon unsuccessfully locating the received image of the specific GUI element within the screen image, concluding that detecting whether the application has displayed the specific GUI element by evaluating the received image of the specific GUI element against the screen image including the GUI elements that the application has displayed has been unsuccessful.

9. The method of claim 1 , wherein the sub-plurality of the properties are user specified by GUI element type.

10. The method of claim 1 , wherein detecting whether the application has displayed the specific GUI element by evaluating the received values of the properties of the specific GUI element against the GUI elements that the application has displayed comprises:

determining whether any of the GUI elements that the application has displayed has a same value for each property of the properties as the specific GUI element does;

in response to determining that any of the GUI elements has the same value for each property of the properties as the specific GUI element does, concluding that detecting whether the application has displayed the specific GUI element by evaluating the received values of the properties of the specific GUI element against the GUI elements that the application has displayed has been successful; and

in response to determining that none of the GUI elements has the same value for each property of the properties as the specific GUI element does, concluding that detecting whether the application has displayed the specific GUI element by evaluating the received values of the properties of the specific GUI element against the GUI elements that the application has displayed has been unsuccessful.

11. A computing system comprising:

a processor; and

a non-transitory computer-readable storage medium storing program code executable by the processor to functionally test an application as directed by a script, including:

detecting whether the application has displayed a specific graphical user interface (GUI) element primarily by values for a plurality of properties of the specific GUI element as specified by the script and secondarily by an image of the specific GUI element as specified by the script;

in response to unsuccessfully detecting that the application has displayed the specific GUI element, failing and prematurely terminating functional testing of the application; and

in response to successfully detecting that the application has displayed the specific GUI element, continuing the functional testing of the application,

wherein the processor detects that the application has displayed the specific GUI element by the image of the specific GUI element by:

once the image of the specific GUI element has been successfully identified within a screen image of all GUI elements that the application has displayed;

verifying that a displayed GUI element at a location at which the image of the specific GUI element has been located within the screen image matches the values for a sub-plurality of the properties of the specific GUI element.

12. The computing system of claim 11 , wherein the processor detects whether the application has displayed the specific GUI element by the image of the specific GUI element just responsive to failing to detect that the application has displayed the specific GUI element by the values for the properties of the specific GUI element.

13. The computing system of claim 11 , wherein the program code is further executable by the processor to modify the script to specify that the specific GUI element is to be detected during subsequent functional testing by the sub-plurality of the properties instead of by the plurality of properties.

14. The computing system of claim 11 , wherein the processor and the non-transitory computer-readable data storage medium are part of a client computing device that further includes:

network hardware to communicate over a network with a server computing device at which the application undergoing the functional testing is being executed.

15. The computing system of claim 11 , wherein the processor further executes the application undergoing the functional testing is being executed.

Assignments (6)
RELEASE OF SECURITY INTEREST REEL/FRAME 052294/0522 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC; MICRO FOCUS SOFTWARE INC. (F/K/A NOVELL, INC.); NETIQ CORPORATION
Reel/Frame 062624/0449 →
RELEASE OF SECURITY INTEREST REEL/FRAME 052295/0041 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC; MICRO FOCUS SOFTWARE INC. (F/K/A NOVELL, INC.); NETIQ CORPORATION
Reel/Frame 062625/0754 →
SECURITY AGREEMENT Recorded Apr 2, 2020
From: MICRO FOCUS LLC; BORLAND SOFTWARE CORPORATION; MICRO FOCUS SOFTWARE INC.; NETIQ CORPORATION; MICRO FOCUS (US), INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 052294/0522 →
SECURITY AGREEMENT Recorded Apr 2, 2020
From: MICRO FOCUS LLC; BORLAND SOFTWARE CORPORATION; MICRO FOCUS SOFTWARE INC.; NETIQ CORPORATION; MICRO FOCUS (US), INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 052295/0041 →
CHANGE OF NAME Recorded Aug 8, 2019
From: ENTIT SOFTWARE LLC
To: MICRO FOCUS LLC
Reel/Frame 050004/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 29, 2018
From: BEN SHABTAI, AVISHAY; SAARONI, DROR; MILGROM, VIKA; KAMINSKY, ANTON; RACHELSON, YOSSI
To: ENTIT SOFTWARE LLC
Reel/Frame 047624/0416 →