Granted Patent B2

US 12,608,397 · App. 15/710,776 · Granted Apr 21, 2026

Inventors: Michael Glueck (Toronto, CA); Azam Khan (Toronto, CA); Jian Zhao (Toronto, CA)

Assignee: AUTODESK, INC.

G06F16/26G06F16/24573G06F40/169G06Q10/103

Abstract

Claims (68)

1 . A computer-implemented method for playback of sense-making operations, the method comprising:

receiving first session data that includes a set of timeline steps, wherein each timeline step included in the set of timeline steps corresponds to a user action performed to annotate data in a data set;

receiving a playback command to display an animation of a first sequence of timeline steps included in the set of timeline steps, wherein the first sequence of timeline steps comprises one or more user actions to directly move a node or an edge of an annotation graph; and

generating the animation by, for each timeline step included in the first sequence of timeline steps:

automatically generating a first graph that includes a plurality of nodes based on a saved layout that was used to generate the first graph when the timeline step occurred, wherein the plurality of nodes represent a plurality of user actions for annotating data items,

rendering the first graph for display in a first interface that resides within a first portion of a display area, and

updating one or more additional interfaces to display one or more references and one or more annotations that were displayed in the one or more additional interfaces when the timeline step occurred, wherein the one or more additional interfaces reside within one or more additional portions of the display area.

2 . The computer-implemented method of claim 1 , wherein updating the one or more additional interfaces comprises:

retrieving a reference from the data set that is associated with the timeline step; and

displaying, after rendering the first graph, the reference simultaneously with the first graph.

3 . The computer-implemented method of claim 1 , wherein updating the one or more additional interfaces comprises:

retrieving a first annotation from an annotation set associated with the data set, wherein the first annotation is associated with the timeline step; and

displaying, after rendering the first graph, the first annotation simultaneously with the first graph.

4 . The computer-implemented method of claim 3 , wherein the first annotation is included in the first graph.

5 . The computer-implemented method of claim 3 , further comprising:

retrieving a reference from the data set that is associated with the timeline step;

displaying, after rendering the first graph, the reference simultaneously with the first graph;

determining a portion of the reference is associated with the first annotation; and

highlighting the portion of the reference when displayed.

6 . The computer-implemented method of claim 1 , further comprising:

displaying, after rendering the first graph, the set of timeline steps from the first session data simultaneously with the first graph.

7 . The computer-implemented method of claim 1 , further comprising:

receiving a first user action performed to annotate data in the data set; and

storing the first user action as an additional timeline step in the set of timeline steps in the first session data.

8 . The computer-implemented method of claim 1 , further comprising:

receiving, after rendering the first graph, a first user action performed to annotate data in the data set; and

storing the first user action as a timeline step in a second set of timeline steps that is included in second session data.

9 . The computer-implemented method of claim 1 , further comprising rendering a timeline for display, wherein the timeline includes the set of timeline steps.

10 . The computer-implemented method of claim 1 , further comprising, for each timeline step included in the first sequence of timeline steps, updating a second interface to display a structure of the data set that was displayed in the second interface when the timeline step occurred, wherein the second interface resides within a second portion of the display area.

11 . The computer-implemented method of claim 10 , wherein the first interface, the second interface, and the one or more additional interfaces are included in a graphical user interface.

12 . The computer-implemented method of claim 1 , wherein the one or more user actions directly move the node and the edge in the annotation graph.

13 . The computer-implemented method of claim 1 , wherein the annotation graph comprises a plurality of nodes and a plurality of edges, and each of the plurality of edges connects between two different node types in the annotation graph.

14 . One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to playback sense-making operations by performing the steps of:

receiving first session data that includes a set of timeline steps, wherein each timeline step included in the set of timeline steps corresponds to a user action performed to annotate data in a data set;

receiving a playback command to display an animation of a first sequence of timeline steps included in the set of timeline steps, wherein the first sequence of timeline steps comprises one or more user actions to directly move a node or an edge of an annotation graph; and

generating the animation by, for each timeline step included in the first sequence of timeline steps:

automatically generating a first graph that includes a plurality of nodes based on a saved layout that was used to generate the first graph when the timeline step occurred, wherein the plurality of nodes represent a plurality of user actions for annotating data items,

rendering the first graph for display in a first interface that resides within a first portion of a display area, and

updating one or more additional interfaces to display one or more references and one or more annotations that were displayed in the one or more additional interfaces when the timeline step occurred, wherein the one or more additional interfaces reside within one or more additional portions of the display area.

15 . The one or more non-transitory computer-readable storage media of claim 14 , wherein receiving the playback command to display the first sequence of timeline steps comprises receiving an indication from a user-selectable button in a playback interface.

16 . The one or more non-transitory computer-readable storage media of claim 14 , wherein receiving the playback command to display the first sequence of timeline steps comprises receiving an indication of a movement of a marker along a timeline slider, wherein the timeline slider corresponds to the first sequence of timeline steps.

17 . The one or more non-transitory computer-readable storage media of claim 14 , wherein receiving the playback command to display the first sequence of timeline steps comprises receiving an indication of a selection of a graphical representation of a plurality of timeline steps included in the first sequence of timeline steps.

18 . The one or more non-transitory computer-readable storage media of claim 14 , wherein updating the one or more additional interfaces comprises:

retrieving a reference from the data set that is associated with the timeline step; and

displaying, after rendering the first graph, the reference simultaneously with the first graph.

19 . The one or more non-transitory computer-readable storage media of claim 14 , wherein updating the one or more additional interfaces comprises:

retrieving a first annotation from an annotation set associated with the data set, wherein the first annotation is associated with the timeline step; and

displaying, after rendering the first graph, the first annotation simultaneously with the first graph.

20 . The one or more non-transitory computer-readable storage media of claim 19 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to further perform the steps of:

retrieving a reference from the data set that is associated with the timeline step;

displaying, after rendering the first graph, the reference simultaneously with the first graph;

determining a portion of the reference is associated with the first annotation; and

highlighting the portion of the reference when displayed.

21 . The one or more non-transitory computer-readable storage media of claim 14 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to further perform the steps of:

receiving, after rendering the first graph, a first user action performed to annotate data in the data set; and

storing the first user action as a timeline step in a second set of timeline steps that is included in second session data.

22 . A system comprising:

one or more memories storing instructions; and

one or more processor that are coupled to the one or more memories and, when executing the instructions, are configured to:

receive first session data that includes a set of timeline steps, wherein each timeline step included in the set of timeline steps corresponds to a user action performed to annotate data in a data set,

receive a playback command to display an animation of a first sequence of timeline steps included in the set of timeline steps, wherein the first sequence of timeline steps comprises one or more user actions to directly move a node or an edge of an annotation graph, and

generate the animation by, for each timeline step included in the first sequence of timeline steps:

automatically generate a first graph that includes a plurality of nodes based on a saved layout that was used to generate the first graph when the timeline step occurred, wherein the plurality of nodes represent a plurality of user actions for annotating data items;

render the first graph for display in a first interface that resides within a first portion of a display area; and

update one or more additional interfaces to display one or more references and one or more annotations that were displayed in the one or more additional interfaces when the timeline step occurred, wherein the one or more additional interfaces reside within one or more additional portions of the display area.

23 . The system of claim 22 , wherein updating the one or more additional interfaces comprises:

retrieving a reference from the data set that is associated with the timeline step; and

displaying, after rendering the first graph, the reference simultaneously with the first graph.

Assignments (1)

Continuity (4)

References Cited (188)