IP Library Granted Patent US 10,210,001
Granted Patent B2
US 10,210,001 · App. 14/914,590 · Granted Feb 19, 2019

Automatic execution of objects in a user interface

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,210,001
App. No.
14/914,590
Granted
Feb 19, 2019
Kind
B2
Abstract

Embodiments are directed to traversing and mapping a user interface. In one scenario, a computer system performs a method for traversing and mapping a user interface that includes the following: rendering a user interface (UI) based on data received from an information source, constructing an element collection with UI elements, the element collection describing relationships that exist between the UI elements, traversing the element collection to identify various node element types, identifying, within a pre-defined action set, an action associated with at least one of the identified node element types, executing the identified action against the identified node element types, and logging at least one network request resulting from the executed action.

Claims (52)

1. A method, implemented at a computer system that includes at least one processor, for traversing and mapping a user interface, the method comprising:

rendering a user interface (UI) based on data received from an information source;

navigating, with a navigation module within the computer system, through the UI by providing a plurality of navigation inputs from the navigation module to the UI;

when changes occur to the UI based upon an input from the plurality of navigation inputs, assign a unique view ID to the UI view;

creating an element collection with one or more UI elements, the element collection describing relationships that exist between the one or more UI elements within the user interface;

creating a unique view ID for each created element collection, wherein a unique view ID is created for each screen of an application or each page of a web site;

automatically traversing, with the computer system, the element collection to identify one or more node element types;

automatically identifying with the computer, within a pre-defined action set, an action associated with at least one of the identified one or more node element types, wherein the action comprises a response by the one or more UI elements based upon an input to the one or more UI elements;

automatically executing, with the computer, the identified action against the identified one or more node element types;

identifying at least one network request being sent in response to automatically executing the identified action, wherein the network request is sent by a firing tag; and

logging the at least one network request resulting from the executed action.

2. The method of claim 1 , wherein the information source comprises at least one of a software application or a web page.

3. The method of claim 1 , wherein the unique view IDs from the application screens or web pages are combined into a comprehensive element tree for multiple application screens or multiple pages of the website.

4. The method of claim 1 , wherein the user interface for the information source is rendered in an emulator or in a physical device.

5. The method of claim 1 , wherein an element tree is constructed for each UI view, and wherein a plurality of constructed element trees are linked together.

6. The method of claim 5 , wherein each UI view is assigned a unique ID, and wherein the unique ID comprises the name of the tree.

7. The method of claim 1 , wherein the element collection is constructed with multiple different types of UI elements.

8. A method, implemented at a computer system that includes at least one processor, for traversing and mapping a user interface, the method comprising:

rendering a user interface (UI) based on data received from an information source;

navigating through the information source by providing a plurality of navigation inputs to the information source;

mapping one or more UI views that are generated in response to the plurality of navigation inputs and assigning unique view IDs to each UI view within the one or more UI views, wherein a unique view ID is created for each screen of an application or each page of a web site;

constructing an element collection with one or more UI elements based on the mapped one or more UI views, the element collection describing relationships that exist between the one or more UI elements;

automatically traversing, with the computer system, the element collection to identify one or more node element types;

automatically identifying with the computer system, within a pre-defined action set, an action associated with at least one of the identified one or more node element types, wherein the action comprises a response by the one or more UI elements based upon an input to the one or more UI elements;

automatically executing, with the computer, the identified action against the identified one or more node element types;

identifying at least one network request being sent in response to automatically executing the identified action, wherein the network request is sent by a firing tag; and

logging the at least one network request resulting from the executed action.

9. The method of claim 8 , wherein block UI elements for the user interface are identified by accessing screen instructions for the UI.

10. The method of claim 8 , wherein the navigation inputs comprise at least one of screen taps, screen swipes, text entry, mouse clicks, cursor movements or touch gestures.

11. The method of claim 8 , further comprising:

determining that a specified UI view is substantially identical to a second, different UI view; and

assigning the same view ID to the specified UI view and the second, different UI view.

12. The method of claim 8 , wherein constructing an element collection with one or more UI elements further comprises analyzing source code for the information source.

13. The method of claim 8 , further comprising:

identifying each action that fires a tag; and

for each action that fires a tag, saving an indication of which action fires which tag.

14. A computer system for traversing and mapping a user interface, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following:

rendering a user interface (UI) based on data received from an information source;

navigating, with a navigation module within the computer system, through the UI by providing a plurality of navigation inputs from the navigation module to the UI;

when changes occur to the UI based upon an input from the plurality of navigation inputs, assign a unique view ID to the UI view;

creating an element collection with one or more UI elements, the element collection describing relationships that exist between the one or more UI elements within the user interface;

creating a unique view ID for each created element collection, wherein a unique view ID is created for each screen of an application or each page of a web site;

automatically traversing, with the computer system, the element collection to identify one or more node element types;

automatically identifying with the computer, within a pre-defined action set, an action associated with at least one of the identified one or more node element types, wherein the action comprises a response by the one or more UI elements based upon an input to the one or more UI elements;

automatically executing, with the computer, the identified action against the identified one or more node element types;

identifying at least one network request being sent in response to automatically executing the identified action, wherein the network request is sent by a firing tag; and

logging the at least one network request resulting from the executed action.

15. The system of claim 14 , wherein an element tree is constructed for each UI view, and wherein a plurality of constructed element trees are linked together.

16. The system of claim 15 , wherein each UI view is assigned a unique ID, and wherein the unique ID comprises the name of the tree.

17. The method of claim 14 , wherein the element collection is constructed with multiple different types of UI elements.

Assignments (2)
SECURITY INTEREST Recorded Jun 12, 2024
From: OBSERVEPOINT, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 067703/0520 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2016
From: SEOLAS, ROBERT K.; PESTANA, JOHN; BROADBENT, TYLER; LARSON, GREGORY
To: OBSERVEPOINT, INC.
Reel/Frame 037833/0409 →