IP Library Granted Patent US 10,169,853
Granted Patent B2
US 10,169,853 · App. 15/384,817 · Granted Jan 1, 2019

Score weights for user interface (UI) elements

Inventors: Olga Kogan (Yehud, IL); Amit Levin (Yehud, IL); Ilan Shufer (Tel Aviv, IL)
Assignee: ENTIT SOFTWARE LLC
G06T7/0002G06K9/00G06K9/18G06K9/4647G06K9/4652G06K9/6218G06N99/005G06T2200/24G06T2207/20072G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,169,853
App. No.
15/384,817
Filed
Dec 20, 2016
Granted
Jan 1, 2019
Kind
B2
Art Unit
2666
USPC
382/100
Abstract

An example device in accordance with an aspect of the present disclosure includes an identification engine, a clustering engine, and a weighting engine. The identification engine is to identify User Interface (UI) elements of an application under test based on an image processing analysis of screens of the application under test. The clustering engine is to cluster the UI elements into at least one cluster based on at least one feature of the UI elements. The weighting engine is to assign score weights to the UI elements of a given screen according to which of the at least one cluster that a given one of the UI elements is clustered in.

Claims (31)

1. A system comprising:

an identification engine to identify, based on an image processing analysis of screens of an application under test, User Interface (UI) elements of the application under test;

a clustering engine to cluster the UI elements into at least one cluster based on at least one feature of the UI elements;

a weighting engine to assign score weights to the UI elements according to which of the at least one cluster that a given one of the UI elements is clustered in; and

a testing coverage engine to automatically identify, based on the score weights of which of the UI elements are covered by the test, a testing coverage score of the application under test.

2. The system of claim 1 , wherein the clustering engine is to cluster the UI elements based on the at least one feature including a histogram of colors of the UI elements or a histogram of intensity of the UI elements.

3. The system of claim 1 , wherein the clustering engine is to perform Optical Character Recognition (OCR) on the UI elements, and group the UI elements into a first group of those UI elements including text, and a second group of those UI elements not including text, and apply clustering to the first group distinct from the second group.

4. The system of claim 1 , wherein the clustering engine is to cluster the UI elements based on a machine learning unsupervised clustering technique.

5. The system of claim 1 , wherein the weighting engine is to assign a score weight to a given UI element of a given cluster based on a degree of user interaction with other UI elements of that given cluster to which the given UI element is clustered in.

6. The system of claim 5 , wherein the user interaction is based on a mouse click on the UI element.

7. The system of claim 5 , wherein the user interaction is based on user eye gaze duration on the UI element.

8. The system of claim 1 , wherein the weighting engine is to assign a score weight to a given UI element based on a number of times with which that given UI element has been interacted, independent from the given UI element's cluster.

9. The system of claim 1 , wherein the weighting engine is to assign score weights according to a customizable score assigned to a given cluster.

10. The system of claim 1 , wherein the weighting engine is to assign a score weight to a given UI element according to a customizable score assigned to that particular UI element.

11. The system of claim 1 , wherein the weighting engine is to assign score weights for a given screen, based on UI elements contained in the given screen, and wherein the testing coverage engine is to identify the testing coverage score for the given screen according to which UI elements of the given screen are covered by the test.

12. A method, comprising:

identifying, by a testing coverage device, user interface (UI) elements of an application under test based on an analysis of screens of the application under test;

clustering, by the testing coverage device, the UI elements of the application under test into at least one cluster based on at least one feature of the UI elements;

assigning, by the testing coverage device, score weights to the UI elements according to which of the at least one cluster that a given one of the UI elements is clustered in; and

identifying, by the testing coverage device, a testing coverage score of the application under test based on the score weights of which UI elements are covered by the test.

13. The method of claim 12 , wherein the at least one feature of the UI elements includes histograms automatically determined by the testing coverage device for the UI elements.

14. A non-transitory machine-readable storage medium encoded with instructions executable by a computing system that, when executed, cause the computing system to:

identify user interface (UI) elements of an application under test based on an image processing analysis of screens of the application under test;

cluster the UI elements into at least one cluster based on at least one feature of the UI elements;

assign score weights to the UI elements of a given screen according to which of the at least one cluster that a given one of the UI elements is clustered in, wherein the score weights are assigned on a per-screen basis.

15. The storage medium of claim 14 , further comprising instructions that cause the computing system to automatically identify a testing coverage score of the application under test based on the score weights of which UI elements are covered by the test.

16. The storage medium of claim 14 , further comprising instructions that cause the computing system to perform Optical Character Recognition (OCR) on the UI elements, and group the UI elements into a first group of those UI elements including text, and a second group of those UI elements not including text, and apply clustering to the first group distinct from the second group.

17. The storage medium of claim 14 , further comprising instructions that cause the computing system to cluster the UI elements based on a machine learning unsupervised clustering technique.

18. The storage medium of claim 14 , further comprising instructions that cause the computing system to assign a score weight to a given UI element of a given cluster based on a degree of user interaction with other UI elements of that given cluster to which the given UI element is clustered in.

19. The storage medium of claim 14 , further comprising instructions that cause the computing system to assign score weights according to a customizable score assigned to a given cluster.

20. The storage medium of claim 14 , further comprising instructions that cause the computing system to assign a score weight to a given UI element according to a customizable score assigned to that particular UI element.

Assignments (7)
RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC); BORLAND SOFTWARE CORPORATION; MICRO FOCUS (US), INC.; SERENA SOFTWARE, INC; ATTACHMATE CORPORATION; MICRO FOCUS SOFTWARE INC. (F/K/A NOVELL, INC.); NETIQ CORPORATION
Reel/Frame 062746/0399 →
RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0577 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC)
Reel/Frame 063560/0001 →
CHANGE OF NAME Recorded Aug 8, 2019
From: ENTIT SOFTWARE LLC
To: MICRO FOCUS LLC
Reel/Frame 050004/0001 →
SECURITY INTEREST Recorded Oct 11, 2017
From: ATTACHMATE CORPORATION; BORLAND SOFTWARE CORPORATION; NETIQ CORPORATION; MICRO FOCUS (US), INC.; MICRO FOCUS SOFTWARE, INC.; ENTIT SOFTWARE LLC; ARCSIGHT, LLC; SERENA SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 044183/0718 →
SECURITY INTEREST Recorded Oct 11, 2017
From: ENTIT SOFTWARE LLC; ARCSIGHT, LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 044183/0577 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2017
From: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
To: ENTIT SOFTWARE LLC
Reel/Frame 042746/0130 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2016
From: KOGAN, OLGA; LEVIN, AMIT; SHUFER, ILAN
To: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Reel/Frame 040689/0641 →
Continuity (1)
Related Publication 20180174288A1 · Jun 21, 2018