IP Library Granted Patent US 12,346,708
Granted Patent B1
US 12,346,708 · App. 18/103,241 · Granted Jul 1, 2025

Concurrent visualization of session data and session playback

Inventors: Umang Agarwal (Berkeley, CA); Akila Balasubramanian (Campbell, CA); Calvin Chan (Sunnyvale, CA); Khawar Deen (Sunnyvale, CA); Nikhil Kasthurirangan (Santa Clara, CA); Matthew William Pound (Seattle, CA); Justin Smith (San Francisco, CA); Taavo-Taur Tammur (Tartu, EE); Rashmi Kalyani Vasudevan (San Francisco, CA); Pragati Vyas (San Jose, CA); Sally Wahba (Apex, NC); John Bennett Wundes (Santa Rosa, CA)
Assignee: Splunk LLC
G06F9/451G06F9/542G06F11/34G06F11/3414G06F11/3438
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,346,708
App. No.
18/103,241
Granted
Jul 1, 2025
Kind
B1
Abstract

Described herein are techniques for concurrently visualizing real user's session data with a playback of a user's experience during a session. The disclosed techniques can correlate session data collected in response to interactions with an application during a user's session with session recreation data generated by the application and recorded during the user's session. The correlations between the session data and the recreation data can be used to render and control a recreation of the user's experience during the session concurrently with a visualization of the session data.

Claims (42)

1. A computer-implemented method, comprising:

receiving spans and recreation data generated in response to users using respective client devices to interact with an application installed on a server system, wherein:

the recreation data is generated on the respective client devices;

each span represents an operation performed by the application on the server system in response to a user interaction with the application;

the spans are generated on the server system by the application from data generated on the server system by the application; and

hierarchies of the spans each represent a collection of operations performed by the application in response to a corresponding user interaction, and relationships between child spans and parent spans in each hierarchy represent relationships between corresponding child operations in the collection of operations that are initiated by corresponding parent operations in the collection of operations;

rendering, in a first graphical user interface (GUI), a visualization of first spans of the spans that were generated in response to a user interacting with the application using a client device during a session,

wherein the visualization displays hierarchies of the first spans together;

identifying, in response to receiving a request from the first GUI for a playback of the session, first recreation data generated on the client device in response to events generated by the application during the session; and

rendering a recreation of the application during the session in the first GUI using the first recreation data.

2. The computer-implemented method of claim 1 , wherein the first spans are associated with a first set of timestamps, the first recreation data is associated with a second set of timestamps that overlap with the first set of timestamps, and the method further comprises:

receiving a selection of a first span associated with a first timestamp from the first set of timestamps; and

advancing, using the second set of timestamps, the rendering of the recreation of the application to be after a second timestamp from the second set of timestamps that occurred before the first timestamp.

3. The computer-implemented method of claim 1 , wherein the first recreation data is associated with a first set of timestamps, the first spans are associated with a second set of timestamps that overlap with the first set of timestamps, and the method further comprises:

updating the rendering of the recreation of the application to be at a first timestamp from the first set of timestamps; and

rendering, using the second set of timestamps, a subset of the first spans that were generated before the first timestamp in the visualization of the first spans.

4. The computer-implemented method of claim 3 , wherein the first recreation data is associated with a page redirection ID and the subset of the first spans were generated with the page redirection ID.

5. The computer-implemented method of claim 1 , wherein the visualization is a waterfall visualization, and the method further comprises:

filtering the first spans rendered in the waterfall visualization according to a first span type selected from the group consisting of a document load span type, a frontend error span type, a network error span type, a data transfer span type, a resource span type, a user interaction span type, and a custom event span type.

6. The computer-implemented method of claim 1 , wherein the session is associated with a session identifier and the first recreation data is identified using the session identifier.

7. The computer-implemented method of claim 1 , wherein a user interface of the application was rendered in a second GUI during the session using an object model, the first recreation data includes changes to the object model that occurred during the session, and rendering a recreation of the application comprises rendering the user interface in the first GUI using the object model with an option to render the changes to the object model in chronological order.

8. The computer-implemented method of claim 7 , wherein the first recreation data includes changes to the object model made in response to interactions with the application via the user interface during the session.

9. The computer-implemented method of claim 7 , further comprising recording the object model and the changes to the object model as the first recreation data using a service that listens for the events generated by the application during the session.

10. The computer-implemented method of claim 9 , further comprising configuring the service to redact or omit one or more types of data in the object model and the changes to the object model while recording recreation data.

11. The computer-implemented method of claim 10 , wherein the one or more types of data include text, images, and personally identifiable information (PII).

12. The computer-implemented method of claim 1 , further comprising rendering a timeline of the session in the first GUI, wherein the timeline comprises one or more event indicators at points along the timeline that correspond to times during the session when a subset of events occurred.

13. The computer-implemented method of claim 12 , wherein the subset of events are identified from the first recreation data and represent user interactions.

14. The computer-implemented method of claim 12 , wherein the subset of events are identified from the first spans and represent user interactions or wherein the subset of events are identified from the first spans and represent errors detected while performing an operation by the application in response to the user interacting with the application.

15. The computer-implemented method of claim 12 , wherein the timeline of the session is rendered as an interactive scrubber in the first GUI, and the method further comprising:

receiving a selection of a first event type;

identifying, in response to receiving the selection, the subset of events that correspond to the first event type; and

populating the interactive scrubber with the one or more event indicators that correspond to the subset of events identified in response to receiving the selection.

16. The computer-implemented method of claim 12 , further comprising rendering, in response to receiving a selection of an event indicator, the recreation of the application at the time during the session corresponding to the event indicator.

17. The computer-implemented method of claim 1 , wherein the spans and the recreation data are received from different data stores.

18. The computer-implemented method of claim 1 , wherein

the application is a website, a web application, or a web service installed on the server system and accessed by the user with a user agent executing on the client device;

each child span includes an identifier of the parent span from which the child span was generated; and

the recreation data is generated by the user agent executing on the client device.

19. A computing device, comprising:

a processor; and

a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations corresponding to the computer-implemented method of claim 1 .

20. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations cause the processor to perform operations corresponding to the computer-implemented method of claim 1 .

Assignments (3)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2025
From: AGARWAL, UMANG; BALASUBRAMANIAN, AKILA; CHAN, CALVIN; DEEN, KHAWAR; KASTHURIRANGAN, NIKHIL; POUND, MATTHEW WILLIAM; SMITH, JUSTIN; TAMMUR, TAAVO-TAUR; VASUDEVAN, RASHMI KALYANI; VYAS, PRAGATI; WAHBA, SALLY; WUNDES, JOHN BENNETT
To: SPLUNK INC.
Reel/Frame 071415/0576 →
References Cited (25)
US 7937344B2 · Baum et al. · 2011 [cited by applicant]
US 8112425B2 · Baum et al. · 2012 [cited by applicant]
US 8751529B2 · Zhang et al. · 2014 [cited by applicant]
US 8788525B2 · Neels et al. · 2014 [cited by applicant]
US 9215240B2 · Merza et al. · 2015 [cited by applicant]
US 9286413B1 · Coates et al. · 2016 [cited by applicant]
US 10127258B2 · Lamas et al. · 2018 [cited by applicant]
US 10430212B1 · Bekmambetov · 2019 [cited by examiner]
US 11144441B1 · Colwell · 2021 [cited by examiner]
US 20170090688A1 · Anderson · 2017 [cited by examiner]
US 20170286184A1 · Semenov · 2017 [cited by examiner]
US 20180113577A1 · Burns · 2018 [cited by examiner]
US 20180253373A1 · Mathur · 2018 [cited by examiner]
US 20190098106A1 · Mungel et al. · 2019 [cited by applicant]
US 20200396304A1 · Webber · 2020 [cited by examiner]
Splunk Enterprise 8.0.0 Overview, available online, retrieved May 20, 2020 from docs.splunk.com. [cited by applicant]
Splunk Cloud 8.0.2004 User Manual, available online, retrieved May 20, 2020 from docs.splunk.com. [cited by applicant]
Splunk Quick Reference Guide, updated 2019, available online at https://www.splunk.com/pdfs/solution-guides/splunk-quick-reference-guide.pdf, retrieved May 20, 2020. [cited by applicant]
Carraso, David, “Exploring Splunk,” published by CITO Research, New York, NY, Apr. 2012. [cited by applicant]
Bitincka, Ledion et al., “Optimizing Data Analysis with a Semi-structured Time Series Database,” self-published, first presented at “Workshop on Managing Systems via Log Analysis and Machine Learning Techniques (SLAML)”… [cited by applicant]
Sobolik, Thomas et al. “Use Datadog Session Replay to view real-time user journeys” pulled from the Internet on Jan. 30, 2023 “https://www.datadoghq.com/blog/session-replay-datadog/”, Datadog, published Jul. 28, 2021. [cited by applicant]
“Session Replay” pulled from the Intrnet on Jan. 30, 2023 “https://www.dynatrace.com/platform/session-replay/” Dynatrace LLC. [cited by applicant]
“Session replay solutions for websites and web apps” pulled from the Intrnet on Jan. 30, 2023 “https://logrocket.com/for/session-replay/” LogRocket. [cited by applicant]
“Session Replay: Understand the user experience behind every data point” pulled from the Intrnet on Jan. 30, 2023 “https://www.fullstory.com/platform/session-replay/” fullstory. [cited by applicant]
“Record and replay the web” pulled from the Intrnet on Jan. 30, 2023 “https://www.rrweb.io/” rrweb. [cited by applicant]
Cited By (2)
US 12,554,374 US 12,675,301