IP Library › Granted Patent US 11,651,245
Granted Patent B2
US 11,651,245 · App. 16/218,327 · Granted May 16, 2023

User interface (UI) design compliance determination

Inventor: Daniel Defiebre (Mannheim, DE)
Assignee: SAP SE
G06N5/04G06F16/9027G06F17/18G06N7/005G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,245
App. No.
16/218,327
Filed
Dec 12, 2018
Granted
May 16, 2023
Kind
B2
Art Unit
2123
USPC
706/45
Abstract

Embodiments relate to learning and determining user interface (UI) compliance with design guidelines. Those guidelines may enforce operability with requirements of an external UI platform. Embodiments receive as an input, a UI comprising a plurality of elements. An image of the UI is rendered, and a plurality of elements of the UI are identified from the image according to a recursive procedure. The plurality of elements are stored in a first logic tree exhibiting a first hierarchical structure having the plurality of elements as leaf nodes. The first logic tree is compared to a second logic tree exhibiting a second hierarchical structure, that is labeled with semantic metadata and stored within a knowledge base of a database. Input for the comparison may comprise the tree and/or additional meta information. Based upon the comparing, an output is generated indicating a similarity between the first logic tree and the second logic tree.

Claims (68)

1. A computer-implemented method comprising:

receiving as an input, a user interface (UI) comprising a plurality of elements;

rendering an image of the UI;

identifying the plurality of elements from the image according to a recursive procedure;

storing the plurality of elements in a first logic tree exhibiting a first hierarchical structure having the plurality of elements as leaf nodes, and logic terms as intermediate nodes;

using a Bayesian network, an in-memory database engine of an in-memory database comparing the first logic tree to a second logic tree that,

exhibits a second hierarchical structure,

is labeled with semantic metadata, and

is stored within a knowledge base of the in-memory database;

based upon the comparing, generating an output indicating a similarity between the first logic tree and the second logic tree;

generating a recommendation regarding a cluster of elements present in the knowledge base, the recommendation comprising,

a probability,

a user interface platform component, and

a location of a GIT address for reuse of code.

2. The method as in claim 1 wherein the recursive procedure comprises:

generating from the image, a first two-dimensional array reflecting pixel information; and

generating from the first two-dimensional array, a second two-dimensional array reflecting information density.

3. The method as in claim 2 wherein the recursive procedure further comprises cropping the image based upon the second two-dimensional array.

4. The method as in claim 1 further comprising constructing the knowledge base utilizing a machine learning technique.

5. The method as in claim 4 wherein the machine learning technique comprises case-based reasoning (CBR).

6. The method as in claim 5 wherein the comparing is first performed between the semantic metadata and a solution description, and is then only performed if necessary between the semantic metadata and a problem description.

7. A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising:

receiving as an input, a user interface (UI) comprising a plurality of elements;

rendering an image of the UI;

identifying the plurality of elements from the image according to a recursive procedure;

storing the plurality of elements in a first logic tree exhibiting a first hierarchical structure having the plurality of elements as leaf nodes, and a plurality of logic terms as intermediate nodes;

utilizing a Bayesian network, an in-memory database engine of an in-memory database comparing the first logic tree to a second logic tree that,

exhibits a second hierarchical structure,

is labeled with semantic metadata, and

is stored within a knowledge base of the in-memory database;

based upon the comparing, generating an output indicating a similarity between the first logic tree and the second logic tree; and

generating a recommendation regarding a cluster of elements present in the knowledge base, the recommendation comprising,

a probability,

a user interface platform component, and

a location of a GIT address for reuse of code.

8. The non-transitory computer readable storage medium as in claim 7 wherein the method further comprises:

generating from the image, a first two-dimensional array reflecting pixel information; and

generating from the first two-dimensional array, a second two-dimensional array reflecting information density.

9. The non-transitory computer readable storage medium as in claim 8 wherein the recursive procedure further comprises cropping the image based upon the second two-dimensional array.

10. The non-transitory computer readable storage medium as in claim 7 wherein the method further comprises:

constructing the knowledge base utilizing a machine learning technique.

11. The non-transitory computer readable storage medium as in claim 10 wherein:

the machine learning technique comprises case-based reasoning; and

the comparing is performed between the semantic metadata and a solution description.

12. A computer system comprising:

one or more processors;

a software program, executable on said computer system, the software program configured to cause an in-memory database engine of an in-memory database to:

receive as an input, a user interface (UI) comprising a plurality of elements;

render an image of the UI;

identify the plurality of elements from the image according to a recursive procedure;

store the plurality of elements in a first logic tree exhibiting a first hierarchical structure having the plurality of elements as leaf nodes, and a plurality of logic terms as intermediate nodes;

use a Bayesian network to compare the first logic tree to a second logic tree that,

exhibits a second hierarchical structure,

is labeled with semantic metadata, and

is stored within a knowledge base of the in-memory database;

based upon the comparing, generate an output indicating a similarity between the first logic tree and the second logic tree; and

generate a recommendation regarding a cluster of elements present in the knowledge base, the recommendation comprising,

a probability,

a user interface platform component, and

a location of a GIT address for reuse of code.

13. The computer system as in claim 12 wherein the software program is further configured to cause the in-memory database engine to construct the knowledge base utilizing a machine learning technique.

14. The computer system as in claim 13 wherein:

the machine learning technique comprises case-based reasoning; and

the software program is configured to cause the in-memory database to compare the semantic metadata and a solution description.

15. The computer system as in claim 12 wherein the recursive procedure comprises:

generating from the image, a first two-dimensional array reflecting pixel information;

generating from the first two-dimensional array, a second two-dimensional array reflecting information density; and

cropping the image based upon the second two-dimensional array.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2018
From: DEFIEBRE, DANIEL
To: SAP SE
Reel/Frame 047759/0845 →
Continuity (1)
Related Publication 20200193306A1 · Jun 18, 2020
Cited By (2)
US 12,430,103 US 12,632,223