IP Library Granted Patent US 12,619,521
Granted Patent B2
US 12,619,521 · App. 18/263,808 · Granted May 5, 2026

Automated testing of walkthroughs

Inventors: Stas Shwarzman (Netanya, IL); Yossi Zarfati (Tel Aviv-Jaffa, IL); Shlomi Sasson (Givat Zeev, IL)
Assignee: WALKME LTD.
G06F11/3688G06F11/3684G06F11/3692G06F11/3696
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,619,521
App. No.
18/263,808
Granted
May 5, 2026
Kind
B2
Abstract

A product, system and method comprising: obtaining a test of a walkthrough of an application, wherein the application comprises a Graphical User Interface (GUI), wherein the walkthrough is defined by a sequence of steps and associated advancement conditions, wherein the test comprises a sequence of automated user actions to be provided to the GUI that is configured to invoke the sequence of steps of the walkthrough; executing, at least partially, the test on the walkthrough, wherein said executing comprises providing the sequence of automated user actions in a manner that is configured to cause the sequence of steps to be invoked; generating a trace display of the test execution, wherein the trace display comprises a visual display of events including both steps of the walkthrough and automated user actions of the test; and outputting the trace display.

Claims (64)

1 . A method comprising:

obtaining a walkthrough of an application, the walkthrough comprising an executable tutorial of the application, wherein the application comprises a Graphical User Interface (GUI), wherein the walkthrough is defined by a sequence of steps and an associated sequence of advancement conditions, wherein for each advancement condition of the sequence of advancement conditions, in response to the advancement condition being met, a next step in the sequence of steps is configured to be invoked, wherein a step of the sequence of steps comprises presenting a descriptive element that is associated to a functionality of a GUI element of the GUI of the application, wherein at least one advancement condition of the advancement conditions is configured to be met by actions of a user that is interacting with the GUI;

obtaining a test of the walkthrough of the application, wherein the test comprises a sequence of automated user actions to be provided to the GUI of the application, wherein the sequence of automated user actions is configured to invoke the sequence of steps of the walkthrough, wherein the sequence of automated user actions is configured to mimic interactions of the user with the GUI; and

executing, at least partially, the test over the walkthrough, wherein said executing comprises providing the sequence of automated user actions in a manner that is configured to cause the sequence of steps to be invoked, at least in part, whereby providing a test execution, wherein the test execution comprises at least automated user actions of the test intertwined with walkthrough steps, wherein the test execution comprises at least an execution of a first step of the walkthrough, followed by an execution of an automated user action of the test, followed by an execution of a second step of the walkthrough.

2 . The method of claim 1 ,

wherein the second step of the walkthrough follows the first step of the walkthrough in the sequence of steps, wherein the first step is associated with a first advancement condition;

wherein the test comprises one or more first automated user actions and one or more second automated user actions, wherein the one or more first automated user actions comprise at least the automated user action, wherein the one or more second automated user actions follow the first automated user actions in the sequence, wherein the first automated user actions are configured to cause the first advancement condition of the first step to be met; and

wherein said executing comprises:

implementing the one or more first automated user actions of the test;

monitoring the application to identify that the second step is invoked; and

in response to identifying that the second step is invoked,

implementing the one or more second automated user actions of the test.

3 . The method of claim 2 , wherein the second step is associated with a second advancement condition, wherein the walkthrough comprises a third step, wherein the third step follows the second step in the sequence, wherein said executing comprises:

in response to identifying that the third step is not invoked after said implementing the one or more second automated user actions of the test, indicating that the walkthrough failed to properly operate.

4 . The method of claim 1 further comprising:

generating a trace display of the test execution, wherein the trace display comprises a visual display of events, wherein the events comprise both the walkthrough steps and the automated user actions of the test, whereby providing an interleaved display between a visual representation of the walkthrough and a visual representation of the test; and

outputting the trace display,

wherein said executing the test comprises identifying a failed step of the sequence of steps, wherein the test did not cause a subsequent step of the walkthrough to be invoked, wherein the trace display comprises a visual indication of the failed step.

5 . The method of claim 1 further comprising:

generating a trace display of the test execution, wherein the trace display comprises a visual display of events, wherein the events comprise both the walkthrough steps and the automated user actions of the test, whereby providing an interleaved display between a visual representation of the walkthrough and a visual representation of the test; and

outputting the trace display, wherein said executing the test comprises determining a failed sub-sequence of the sequence of steps, wherein the failed sub-sequence comprises a plurality of steps starting with an initial step that is estimated to have not been properly operated by the test, wherein the failed sub-sequence comprises the initial step and a failed step, wherein the failed sub-sequence ends with the failed step, wherein the trace display comprises a visual indication of the failed sub-sequence.

6 . The method of claim 1 , wherein the walkthrough is provided by a provider different than a provider of the application, wherein the walkthrough is configured to utilize an acquisition process, wherein the acquisition process is configured to identify an element in the GUI based on parameters associated with the element.

7 . The method of claim 1 further comprises: injecting the test and the walkthrough into a page of the application.

8 . The method of claim 1 further comprises:

displaying a walkthrough display, wherein the walkthrough display comprises a visual display of the sequence of steps, wherein the walkthrough display is different from the trace display; and

displaying a test display, wherein the test display comprises a visual display of the sequence of automated user actions, wherein the test display is different from the trace display.

9 . The method of claim 1 , wherein the advancement conditions are configured to be met by at least one of:

a timeout;

a user interaction;

the automated user action; and

an external event.

10 . A system comprising a processor and coupled memory, the processor being adapted to:

obtain a walkthrough of an application, the walkthrough comprising an executable tutorial of the application, wherein the application comprises a Graphical User Interface (GUI), wherein the walkthrough is defined by a sequence of steps and an associated sequence of advancement conditions, wherein for each advancement condition of the sequence ent conditions, in response to the advancement condition being met, a next step in the sequence of steps is configured to be invoked, wherein a step of the sequence of steps comprises presenting a descriptive element that is associated to a functionality of a GUI element of the GUI of the application, wherein at least one advancement condition of the advancement conditions is configured to be met by actions of a user that is interacting with the GUI;

obtain a test of the walkthrough of the application, wherein the test comprises a sequence of automated user actions to be provided to the GUI of the application, wherein the sequence of automated user actions is configured to invoke the sequence of steps of the walkthrough, wherein the sequence of automated user actions is configured to mimic interactions of the user with the GUI; and

execute, at least partially, the test over the walkthrough, wherein said executing comprises providing the sequence of automated user actions in a manner that is configured to cause the sequence of steps to be invoked, at least in part, whereby providing a test execution, wherein the test execution comprises at least automated user actions of the test intertwined with walkthrough steps, wherein the test execution comprises at least execution of a first step of the walkthrough, followed by an execution of an automated user action of the test, followed by an execution of a second step of the walkthrough.

11 . The system of claim 10 ,

wherein the second step of the walkthrough follows the first step of the walkthrough in the sequence of steps, wherein the first step is associated with a first advancement condition;

wherein the test comprises one or more first automated user actions and one or more second automated user actions, wherein the one or more first automated user actions comprises at least the automated user action, wherein the one or more second automated user actions follow the first automated user actions in the sequence, wherein the first automated user actions are configured to cause the first advancement condition of the first step to be met; and

wherein said executing comprises:

implementing the one or more first automated user actions of the test;

monitoring the application to identify that the second step is invoked; and

in response to identifying that the second step is invoked,

implementing the one or more second automated user actions of the test.

12 . The system of claim 11 , wherein the second step is associated with a second advancement condition, wherein the walkthrough comprises a third step, wherein the third step follows the second step in the sequence, wherein said executing comprises:

in response to identifying that the third step is not invoked after said implementing the one or more second automated user actions of the test, indicating that the walkthrough failed to properly operate.

13 . The system of claim 10 further comprising:

generating a trace display of the text execution, wherein the trace display comprises a visual display of events, wherein the events comprise both the walkthrough steps and the automated user actions of the test, whereby providing an interleaved display between a visual representation of the walkthrough and a visual representation of the test; and

outputting the trace display, wherein said executing the test comprises identifying a failed step of the sequence of steps, wherein the test did not cause a subsequent step of the walkthrough to be invoked, wherein the trace display comprises a visual indication of the failed step.

14 . The system of claim 10 further comprising:

generating a trace display of the test execution, wherein the trace display comprises a visual display of events, wherein the events comprise both the walkthrough steps and the automated user actions of the test, whereby providing an interleaved display between a visual representation of the walkthrough and a visual representation of the test; and

outputting the trace display, wherein said executing the test comprises determining a failed sub-sequence of the sequence of steps, wherein the failed sub-sequence comprises a plurality of steps starting with an initial step that is estimated to have not been properly operated by the test, wherein the failed sub-sequence comprises the initial step and a failed step, wherein the failed sub-sequence ends with the failed step, wherein the trace display comprises a visual indication of the failed sub-sequence.

15 . The system of claim 10 , wherein the walkthrough is provided by a provider different than a provider of the application, wherein the walkthrough is configured to utilize an acquisition process, wherein the acquisition process is configured to identify an element in the GUI based on parameters associated with the element.

16 . The system of claim 10 , wherein the processor is adapted to inject the test and the walkthrough into a page of the application.

17 . The system of claim 10 , wherein the processor is adapted to display a test display, wherein the test display comprises a visual display of the sequence of automated user actions, wherein the test display is different from the trace display.

18 . The system of claim 10 , wherein the advancement conditions are configured to be met by at least one of:

a timeout;

a user interaction;

the automated user action; and

an external event.

19 . A non-transitory computer readable medium retaining program instructions, which program instructions when read by a processor, cause the processor to:

obtain a walkthrough of an application, the walkthrough comprising an executable tutorial of the application, wherein the application comprises a Graphical User Interface (GUI), wherein the walkthrough is defined by a sequence of steps and an associated sequence of advancement conditions, wherein for each advancement condition of the sequence of advancement conditions, in response to the advancement condition being met, a next step in sequence of steps is configured to be invoked, wherein a step of the sequence of steps comprises presenting a descriptive element that is associated to a functionality of a GUI element of the GUI of the application, wherein at least one advancement condition of the advancement conditions is configured to be met by actions of a user that is interacting with the GUI;

obtain a test of the walkthrough of the application, wherein the test comprises a sequence of automated user actions to be provided to the GUI of the application, wherein the sequence of automated user actions is configured to invoke the sequence of steps of the walkthrough, wherein the sequence of automated user actions is configured to mimic interactions of the user with the GUI; and

execute, at least partially, the test over the walkthrough, wherein said executing comprises providing the sequence of automated user actions in a manner that is configured to cause the sequence of steps to be invoked, at least in part, whereby providing a test execution, wherein the test execution comprises at least automated user actions of the test intertwined with walkthrough steps, wherein the test execution comprises at least an execution of a first step of the walkthrough, followed by an execution of an automated user action of the test, followed by an execution of a second step of the walkthrough.

20 . The computer readable medium of claim 19 , wherein the walkthrough is provided by a provider different than a provider of the application, wherein the walkthrough is configured to utilize an acquisition process, wherein the acquisition process is configured to identify an element in the GUI based on parameters associated with the element.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 1, 2023
From: SHWARZMAN, STAS; ZARFATI, YOSSI; SASSON, SHLOMI
To: WALKME LTD.
Reel/Frame 064454/0014 →
Continuity (1)
Related Publication 20240118997A1 · Apr 11, 2024
References Cited (24)
US 9703684B1 · Sanborn de Asis · 2017 [cited by examiner]
US 9922008B2 · Cohen et al. · 2018 [cited by applicant]
US 11119906B1 · Edouard · 2021 [cited by examiner]
US 11334471B1 · Stocker · 2022 [cited by applicant]
US 20080141221A1 · Benesovska · 2008 [cited by examiner]
US 20090150872A1 · Russell · 2009 [cited by examiner]
US 20090288070A1 · Cohen · 2009 [cited by examiner]
US 20120023484A1 · Demant · 2012 [cited by examiner]
US 20140325479A1 · Barak · 2014 [cited by examiner]
US 20160147645A1 · Kandpal · 2016 [cited by applicant]
US 20160314066A1 · Mirza · 2016 [cited by examiner]
US 20170109263A1 · Anand · 2017 [cited by examiner]
US 20170277625A1 · Shtuchkin et al. · 2017 [cited by applicant]
US 20200012587A1 · Girata, Jr. et al. · 2020 [cited by applicant]
CN 106339319A · 2017 [cited by applicant]
WO 2012073197A1 · 2012 [cited by applicant]
WO 2020222219A1 · 2020 [cited by applicant]
C. Klammer, R. Ramler and H. Stummer, “Harnessing Automated Test Case Generators for GUI Testing in Industry,” 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Limassol, Cyprus, 2… [cited by examiner]
Ritam Jyoti Sarmah et al., “Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications,” Jul. 19, 2020, arXiv:2007.09809v1 [cs.HC] (Year: 2020). [cited by examiner]
Anonymous, Quality Assurance Testing—WalkMe Support, WalkMe [online], 2018 [retrieved Apr. 15, 2020], Retrieved from Internet: <URL: https://web.archive.org/web/20200927033253/https://support.walkme.com/knowledge-base/q… [cited by applicant]
International Search Report from International Application No. PCT/IL2022/050092 dated Apr. 7, 2022, 3 pgs. [cited by applicant]
Clayton Lewis, et al., Testing a Walkthrough Methodology for Theory-Based Design of WalkUp- and-Use Interfaces, In:CHI 1990 Proceedings, Seattle Washington, Apr. 1-5, 1990, pp. 235-242, ACM, New York. [cited by applicant]
Xun Yuan, Generating Event Sequence-Based Test Cases Using GUI Runtime State Feedback, IEEE Transactions on Software Engineering, Jan. 2010, pp. 81-95, vol. 36, No. 1. [cited by applicant]
List of Patents or Patent Applications Treated as Related dated Aug. 1, 2023, 7 pages. [cited by applicant]