Granted Patent B2

US 12,725,328 · App. 18/533,674 · Granted Sep 1, 2026

Inventors: Jeevana Priya Inala (Hillsboro, OR); Chenglong Wang (Bellevue, WA); Priyan Vaithilingam (Boston, MA)

Assignee: Microsoft Technology Licensing, LLC

G06T11/26G06F3/0484G06F8/31G06F8/35G06F8/38G06F9/451G06F40/40G06N3/0475

Abstract

Claims (53)

1 . A method comprising:

receiving, by a user through a user interface (UI), data indicating a dataset of structured data;

displaying, by the UI, the data on a chart;

receiving, by the UI, a request from the user to alter a representation of the data on the chart including an initial chart-design edit of the chart;

dynamically synthesizing a widget operable to directly alter the representation of the data on the chart based on user interaction with the widget, the widget including a visualization specification and a callback function, the visualization specification including a script that defines how the widget is displayed on the UI, the callback function including code that is executed responsive to the user interacting with the widget, and the code translating user inputs provided through user interaction with the widget to visualization properties of the chart, the dynamically synthesizing including:

issuing a prompt to a large language model (LLM), the prompt including a first code template for the visualization specification, a second code template for the callback function, and a current chart visualization specification; and

receiving, from the LLM, the visualization specification and the callback function as filled in versions of the first and second code templates for the visualization specification and the callback function, respectively;

presenting, by the UI, the widget on the UI alongside the chart with the initial chart-design edit; and

altering, based on user interaction with the widget and by calling the callback function with input populated based on the user interaction, the chart, resulting in an altered chart.

2 . The method of claim 1 , wherein the request is provided in natural language through a widget input control.

3 . The method of claim 1 , the code further providing an ordered list of transform objects executed using the widget, the method further comprising:

concurrently performing the initial chart-design edit to the chart along with presentation of the widget on the UI and executing the transform objects in chronological order across widgets before rendering to generate the altered chart.

4 . The method of claim 3 , further comprising providing, by the UI, an enable/disable control for each transform contributed by a widget and wherein the transform objects are included based on a state of the enable/disable control.

5 . The method of claim 1 , further comprising:

pre-processing the dataset resulting in a data summary; and

wherein prompting the LLM further includes providing the first and second code templates along with the data summary to the LLM.

6 . The method of claim 5 , wherein prompting the LLM further includes providing few shot examples along with the prompt.

7 . A system comprising:

processing circuitry;

a display;

a user interface (UI), provided on the display by the processing circuitry, the UI configured to:

receive, by a user, data indicating a dataset of structured data;

display a representation of the data on a chart;

receive a request from the user to alter a representation of the data on the chart including an initial chart-design edit of the chart; and

issue a command to dynamically synthesize a widget operable to directly alter the representation of the data on the chart based on user interaction with the widget;

a widget engine configured to:

receive the command;

generate a prompt to generate the widget based on the command and the dataset, the prompt including a first code template for a visualization specification, a second code template for a callback function and a current chart visualization specification;

issue the prompt to a large language model (LLM); and

receive the widget from the LLM, the widget including the visualization specification and the callback function as filled in versions of the first and second code templates for the visualization specification and the callback function, respectively, the visualization specification including a script that defines how the widget is displayed on the UI, the callback function including code that is executed responsive to the user interacting with the widget, and the code translating user inputs provided through user interaction with the widget to visualization properties of the chart;

the UI further configured to present the widget alongside the chart with the initial chart-design edit; and

a chart engine configured to alter, based on user interaction with the widget on the UI and by calling the callback function with input populated based on the user interaction, the chart, resulting in an altered chart.

8 . The system of claim 7 , wherein the request is provided in natural language through a widget input control of the UI.

9 . The system of claim 7 , wherein the chart engine concurrently performs the initial chart-design edit to the chart along with presentation of the widget on the UI.

10 . The system of claim 7 , further comprising:

a data summarizer configured to pre-process the dataset resulting in a data summary; and

wherein prompting the LLM further includes providing the first and second code templates along with the data summary to the LLM.

11 . The system of claim 10 , wherein prompting the LLM further includes providing few shot examples along with the prompt.

12 . A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for dynamically synthesized widgets, the operations comprising:

receiving, by a user through a user interface (UI), data indicating a dataset of structured data;

displaying, by the UI, a representation of the data on a chart;

receiving, by the UI, a request from the user to alter a representation of the chart including an initial chart-design edit of the chart;

issuing, by the UI, a command to dynamically synthesize a widget operable to alter the representation of the data on the chart based on user interaction with the widget;

receiving, by a widget engine, the command;

generating, by the widget engine, a prompt to generate the widget based on the command and the dataset, the prompt including a first code template for a visualization specification, a second code template for a callback function, and a current chart visualization specification;

issuing, by the widget engine, the prompt to a large language model (LLM);

receiving the widget from the LLM, the widget including the visualization specification and the callback function as filled in versions of the first and second code templates for the visualization specification and the callback function, respectively, the visualization specification including a script that defines how the widget is displayed on the UI, the callback function including code that is executed responsive to the user interacting with the widget, and the code translating (i) the visualization specification altered through user interaction with the widget and (ii) the chart to an altered chart;

presenting, by the UI, the widget alongside the chart with the initial chart-design edit;

altering, by a chart engine and based on user interaction with the widget on the UI and by calling the callback function with input populated based on the user interaction and the chart with the initial chart-design edit, the chart, resulting in the altered chart.

13 . The non-transitory machine-readable medium of claim 12 , further comprising:

receiving, from a data summarizer configured to pre-process the dataset, a data summary; and

wherein prompting the LLM further includes providing the code templates along with the data summary to the LLM.

14 . The non-transitory machine-readable medium of claim 13 , wherein prompting the LLM further includes providing few shot examples along with the prompt.

Assignments (1)

Continuity (1)

References Cited (72)