IP Library › Granted Patent US 11,797,614
Granted Patent B2
US 11,797,614 · App. 17/887,387 · Granted Oct 24, 2023

Incremental updates to natural language expressions in a data visualization user interface

Inventors: Jeffrey Ericson (Menlo Park, CA); Vidya Raghavan Setlur (Portola Valley, CA); Alex Djalali (Athens, GA); Sanaz Golbabaei (Palo Alto, CA); Theodore Kornish (Palo Alto, CA); Xin Zhao (Mountain View, CA); An Shi (Vancouver, CA); Christoph Sachse (Stanford, CA); Eliana Leite Goldner (Vancouver, CA); Suyang Duan (Vancouver, CA)
Assignee: Tableau Software, LLC
G06F16/904G06F16/243G06F16/248G06F16/26G06F16/287G06F16/9038G06F16/90332G06F40/211G06F40/253G06F40/279G06F40/30G06N5/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,797,614
App. No.
17/887,387
Granted
Oct 24, 2023
Kind
B2
Abstract

A computing device displays, in a graphical user interface, a data visualization and a plurality of phrases that collectively define the data visualization. The device receives user input to specify a natural language command related to the data visualization. The device determines that the natural language command includes a metacommand to: (1) add a data field to the data visualization, or (2) remove a data field from the data visualization, or (3) replace a data field in the data visualization with another data field. The device determines one or more proposed actions in accordance with the determined metacommand and ranks the proposed actions. The device receives user selection of a first proposed action. In response to the user selection, the device generates and displays an updated data visualization and one or more updated phrases that collectively define the updated data visualization.

Claims (80)

1. A method of using natural language for updating visual analysis of datasets, comprising:

at a computing device having a display, one or more processors, and memory storing one or more programs configured for execution by the one or more processors:

displaying, in a graphical user interface, a data visualization and a plurality of phrases that collectively define the data visualization, each phrase of the plurality of phrases including a respective one or more first field names of data fields of a dataset corresponding to the data visualization;

receiving user input to specify a natural language command related to the data visualization;

in response to receiving the user input:

determining that the natural language command includes a metacommand to: (1) add a data field to the data visualization, or (2) remove a data field from the data visualization, or (3) replace a data field in the data visualization with another data field;

determining one or more proposed actions in accordance with the determined metacommand;

generating, for each of the one or more proposed actions, one or more respective phrases corresponding to the respective proposed action;

ranking the one or more proposed actions based on a saliency or weight of a missing term that is inferred from the natural language command according to one or more inferencing rules based on syntactic constraints imposed by a context-free grammar and semantic constraints imposed by attributes of the data fields of the dataset; and

displaying the ranked one or more proposed actions in the graphical user interface;

receiving user selection of a first proposed action of the ranked one or more proposed actions; and

in response to the user selection of the first proposed action:

generating and displaying (1) an updated data visualization and (2) one or more updated phrases in accordance with the first proposed action.

2. The method of claim 1 , wherein:

the one or more updated phrases include one or more second field names of data fields in the dataset; and

the first field names and the second field names include a common field name.

3. The method of claim 1 , wherein the one or more first field names are displayed in a visually distinctive manner from other words included in the plurality of phrases.

4. The method of claim 1 , wherein the graphical user interface includes a natural language input region for receiving the user input.

5. The method of claim 4 , wherein the ranked one or more proposed actions are displayed in a dropdown menu adjacent to the natural language input region.

6. The method of claim 1 , wherein the natural language command includes a metacommand to remove a data field from the data visualization; and

the one or more proposed actions include a proposed action to remove a first phrase, of the plurality of phrases, containing a field name of the data field.

7. The method of claim 6 , further comprising displaying the proposed action to remove the first phrase in a strikethrough format.

8. The method of claim 1 , wherein the plurality of phrases includes a first phrase having a first analytical concept; and

the one or more proposed actions include a second analytical concept distinct from the first analytical concept.

9. The method of claim 8 , wherein each of the first and second analytical concepts is selected from the group consisting of: a field/measure, an aggregation type, a level of detail of an aggregation, a filter, a sort, and a calculation over a time range.

10. The method of claim 1 , wherein:

the natural language command includes a metacommand to remove a filter expression containing a specified data field; and

the method further comprises:

identifying a first phrase, from the plurality of phrases, that is a filter expression containing the specified data field;

wherein the one or more proposed actions include a proposed action to remove the first phrase, while retaining one or more of the plurality of phrases, containing the specified data field, that are not filter expressions.

11. The method of claim 1 , wherein the natural language command includes a command to replace a first aggregation operator for a first data field with a second aggregation operator, distinct from the first aggregation operator; and

displaying the one or more proposed actions includes displaying a proposed action that includes a first text string and a second text string, wherein:

the first text string includes the first data field and the first aggregation operator;

the second text string includes the first data field and the second aggregation operator; and

the first text string is visually distinguished from the second text string.

12. The method of claim 1 , wherein the natural language command further specifies replacement of one field name with a distinct field name; and

displaying one or more proposed actions includes displaying a proposed action that includes the one field name and the distinct field name.

13. The method of claim 1 , wherein displaying the one or more proposed actions includes:

displaying a second proposed action that includes an add command; and

displaying a third proposed action that includes a replace command.

14. The method of claim 1 , wherein:

the data fields of the dataset are retrieved from a database; and

generating the updated data visualization includes requerying the database using a set of one or more queries based on the updated phrases, thereby retrieving an updated dataset.

15. The method of claim 1 , wherein:

the data visualization has a first data visualization type; and

the updated data visualization has a second data visualization type distinct from the first data visualization type.

16. The method of claim 1 , wherein:

the data visualization has a first data visualization type; and

displaying the updated data visualization includes displaying a plurality of second data visualizations, each having the first data visualization type.

17. The method of claim 1 , wherein:

the plurality of phrases is displayed as a plurality of distinct entities; and

each entity of the plurality of entities is separated from respective adjacent entities by a comma and/or a conjunction.

18. The method of claim 1 , further comprising ranking the one or more proposed actions based on a frequency of usage of a field name based on previous user history.

19. A computing device, comprising:

a display;

one or more processors; and

memory coupled to the one or more processors, the memory storing one or more programs configured for execution by the one or more processors, the one or more programs including instructions for:

displaying, in a graphical user interface, a data visualization and a plurality of phrases that collectively define the data visualization, each phrase of the plurality of phrases including a respective one or more first field names of data fields of a dataset corresponding to the data visualization;

receiving user input to specify a natural language command related to the data visualization;

in response to receiving the user input:

determining that the natural language command includes a metacommand to: (1) add a data field to the data visualization, or (2) remove a data field from the data visualization, or (3) replace a data field in the data visualization with another data field;

determining one or more proposed actions in accordance with the determined metacommand;

generating, for each of the one or more proposed actions, one or more respective phrases corresponding to the respective proposed action;

ranking the one or more proposed actions based on a saliency or weight of a missing term that is inferred from the natural language command according to one or more inferencing rules based on syntactic constraints imposed by a context-free grammar and semantic constraints imposed by attributes of the data fields of the dataset; and

displaying the ranked one or more proposed actions in the graphical user interface;

receiving user selection of a first proposed action of the ranked one or more proposed actions; and

in response to the user selection of the first proposed action:

generating and displaying (1) an updated data visualization and (2) one or more updated phrases in accordance with the first proposed action.

20. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device having a display, cause the computing device to perform operations comprising:

displaying, in a graphical user interface, a data visualization and a plurality of phrases that collectively define the data visualization, each phrase of the plurality of phrases including a respective one or more first field names of data fields of a dataset corresponding to the data visualization;

receiving user input to specify a natural language command related to the data visualization;

in response to receiving the user input:

determining that the natural language command includes a metacommand to: (1) add a data field to the data visualization, or (2) remove a data field from the data visualization, or (3) replace a data field in the data visualization with another data field;

determining one or more proposed actions in accordance with the determined metacommand;

generating, for each of the one or more proposed actions, one or more respective phrases corresponding to the respective proposed action;

ranking the one or more proposed actions based on a saliency or weight of a missing term that is inferred from the natural language command according to one or more inferencing rules based on syntactic constraints imposed by a context-free grammar and semantic constraints imposed by attributes of the data fields of the dataset; and

displaying the ranked one or more proposed actions in the graphical user interface;

receiving user selection of a first proposed action of the ranked one or more proposed actions; and

in response to the user selection of the first proposed action:

generating and displaying (1) an updated data visualization and (2) one or more updated phrases in accordance with the first proposed action.

Continuity (3)
Continuation 16601437 · Oct 14, 2019
Provisional Application 62897187 · Sep 6, 2019
Related Publication 20220382815A1 · Dec 1, 2022