IP Library › Granted Patent US 12,462,107
Granted Patent B2
US 12,462,107 · App. 18/673,275 · Granted Nov 4, 2025

Analyzing underspecified natural language utterances in a data visualization user interface

Inventors: Vidya Raghavan Setlur (Portola Valley, CA); Alex Djalali (Athens, GA)
Assignee: Tableau Software, LLC
G06F40/30G06F3/0481G06F3/0484G06F16/24556G06F16/248G06F16/90332G06F16/9038G06F16/904G06F40/205G06F40/211G06F40/284
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,462,107
App. No.
18/673,275
Filed
May 23, 2024
Granted
Nov 4, 2025
Kind
B2
Art Unit
2655
USPC
704/9
Abstract

A computing device parses a user-specified natural language command to form a first expression. The computing device determines that the first expression is ambiguous or underspecified. The computing device, in accordance with the determination, infers first information using one or more inferencing rules, where at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source. The computing device forms a second expression based on the first expression using and the first information. The computing device retrieves one or more data sets from the data source using according to the second expression. The computing device generates and displays a data visualization of the retrieved one or more data sets.

Claims (67)

1 . A method for generating data visualizations from natural language expressions, comprising:

at a computing device having a display, one or more processors, and memory storing one or more programs configured for execution by the one or more processors:

parsing a user-specified natural language command to form a first expression;

determining that the first expression is ambiguous or underspecified;

in accordance with the determination:

inferring first information using one or more inferencing rules, wherein at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source and the first information is information that is (i) not specified in the user-specified natural language command and (ii) obtained from a semantic model that is derived from the data source, the semantic model including metadata about the attribute of data fields and/or data values in the data source and including metadata of one or more synonyms of the attribute;

forming a second expression based on the first expression and the first information;

retrieving one or more data sets from the data source according to the second expression; and

generating and displaying a data visualization of the retrieved one or more data sets.

2 . The method of claim 1 , further comprising:

concurrently displaying, with the data visualization, an indication of the first information.

3 . The method of claim 1 , wherein the first information is a data field or a data value, of the data source, that is not specified in the user-specified natural language command.

4 . The method of claim 1 , wherein:

the first information includes an open variable of the data source; and

inferring the first information includes:

assigning a non-logical constant to the open variable; and

inferring an analytical concept for the non-logical constant.

5 . The method of claim 4 , wherein the analytical concept is selected from the group consisting of: field, value, aggregation, group, filter, limit, and sort.

6 . The method of claim 1 , wherein:

the first information includes an underspecified concept;

inferring the first information includes:

identifying a data field associated with the underspecified concept; and

inferring a range of predefined values associated with the data field; and

generating and displaying the data visualization includes generating and displaying the data visualization based on the range of predefined values.

7 . The method of claim 6 , wherein the range of predefined values includes one or more of an average value, a standard deviation, and a maximum value associated with the data field.

8 . The method of claim 1 , further comprising receiving the user-specified natural language command via a user interface control in a data visualization interface.

9 . The method of claim 8 , further comprising:

after inferring the first information, displaying the first information as one or more options in the user interface control, each of the one or more options representing an interpretation of the first information.

10 . The method of claim 9 , wherein displaying the first information as one or more options in the user interface control includes displaying the one or more options in a dropdown menu of the user interface control.

11 . The method of claim 1 , wherein:

the user-specified natural language command includes a first temporal concept;

inferring the first information includes:

identifying a first temporal hierarchy associated with the first temporal concept;

inferring a second temporal hierarchy associated with the data source; and

retrieving, from the data source, a plurality of data fields having the second temporal hierarchy; and

generating and displaying the data visualization uses the plurality of data fields having the second temporal hierarchy.

12 . The method of claim 11 , wherein the plurality of data fields having the second temporal hierarchy has a level of detail that is more granular than the level of detail of data fields in the data source having the first temporal hierarchy.

13 . The method of claim 1 , wherein the user-specified natural language command includes a data visualization type; and

generating and displaying the data visualization of the retrieved one or more data sets includes displaying the data visualization having the data visualization type.

14 . The method of claim 13 , wherein the data visualization type is selected from the group consisting of: a bar chart, a Gantt chart, a line chart, a map, a pie chart, a scatter plot, and a tree map.

15 . The method of claim 1 , wherein at least one of the inferencing rules is based on a constraint imposed by an attribute of data fields and/or data values in the data source.

16 . The method of claim 1 , wherein determining that the first expression is ambiguous or underspecified includes determining that the first expression omits sufficient information for generating a data visualization from the data source.

17 . A computing device for generating data visualizations from natural language expressions, comprising:

one or more processors;

memory coupled to the one or more processors;

a display; and

one or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs including instructions for:

parsing a user-specified natural language command to form a first expression;

determining that the first expression is ambiguous or underspecified;

in accordance with the determination:

inferring first information using one or more inferencing rules, wherein at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source and the first information is information that is (i) not specified in the user-specified natural language command and (ii) obtained from a semantic model that is derived from the data source, the semantic model including metadata about the attribute of data fields and/or data values in the data source and including metadata of one or more synonyms of the attribute;

forming a second expression based on the first expression and the first information;

retrieving one or more data sets from the data source according to the second expression; and

generating and displaying a data visualization of the retrieved one or more data sets.

18 . A non-transitory computer readable storage medium storing one or more programs configured for execution by one or more processors of a computing device that includes a display, the one or more programs comprising instructions for:

parsing a user-specified natural language command to form a first expression;

determining that the first expression is ambiguous or underspecified;

in accordance with the determination:

inferring first information using one or more inferencing rules, wherein at least one of the inferencing rules is based on an attribute of data fields and/or data values in a data source and the first information is information that is (i) not specified in the user-specified natural language command and (ii) obtained from a semantic model that is derived from the data source, the semantic model including metadata about the attribute of data fields and/or data values in the data source and including metadata of one or more synonyms of the attribute;

forming a second expression based on the first expression and the first information;

retrieving one or more data sets from the data source according to the second expression; and

generating and displaying a data visualization of the retrieved one or more data sets.

19 . The non-transitory computer readable storage medium of claim 18 , wherein:

the first information includes an open variable of the data source; and

the instructions for inferring the first information include instructions for:

assigning a non-logical constant to the open variable; and

inferring an analytical concept for the non-logical constant.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2026
From: TABLEAU SOFTWARE, LLC
To: SALESFORCE, INC.
Reel/Frame 076161/0201 →
CHANGE OF NAME Recorded Feb 9, 2026
From: TABLEAU SOFTWARE, INC.
To: TABLEAU SOFTWARE, LLC
Reel/Frame 075516/0504 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2025
From: SETLUR, VIDYA RAGHAVAN; DJALALI, ALEX
To: TABLEAU SOFTWARE, INC.
Reel/Frame 072244/0718 →
Continuity (4)
Continuation 17667474 · Feb 8, 2022
Continuation 16234470 · Dec 27, 2018
Provisional Application 62742857 · Oct 8, 2018
Related Publication 20240311571A1 · Sep 19, 2024
References Cited (43)
US 7302383B2 · Valles · 2007 [cited by examiner]
US 7603267B2 · Wang · 2009 [cited by examiner]
US 9299173B2 · Rope · 2016 [cited by examiner]
US 9501585B1 · Gautam · 2016 [cited by examiner]
US 9818211B1 · Gibb · 2017 [cited by examiner]
US 9858292B1 · Setlur · 2018 [cited by examiner]
US 10331720B2 · Neels · 2019 [cited by examiner]
US 10418032B1 · Mohajer · 2019 [cited by examiner]
US 10515121B1 · Setlur · 2019 [cited by examiner]
US 10546001B1 · Nguyen · 2020 [cited by examiner]
US 10546003B2 · Gupta · 2020 [cited by examiner]
US 10564622B1 · Dean · 2020 [cited by examiner]
US 10817527B1 · Setlur · 2020 [cited by examiner]
US 10956655B2 · Choe · 2021 [cited by examiner]
US 11080336B2 · Van Dusen · 2021 [cited by examiner]
US 11114189B2 · Prosky · 2021 [cited by examiner]
US 11720240B1 · Setlur · 2023 [cited by examiner]
US 20070129936A1 · Wang · 2007 [cited by examiner]
US 20090313576A1 · Neumann · 2009 [cited by examiner]
US 20110119047A1 · Ylonen · 2011 [cited by examiner]
US 20120323948A1 · Li · 2012 [cited by examiner]
US 20130249917A1 · Fanning · 2013 [cited by examiner]
US 20160092090A1 · Stojanovic · 2016 [cited by examiner]
US 20170091277A1 · Zoch · 2017 [cited by examiner]
US 20180032576A1 · Romero · 2018 [cited by examiner]
US 20180039614A1 · Govindarajulu · 2018 [cited by examiner]
US 20180129941A1 · Gustafson · 2018 [cited by examiner]
US 20180137424A1 · Gabaldon Royval · 2018 [cited by examiner]
US 20180158245A1 · Govindan · 2018 [cited by examiner]
US 20180203924A1 · Agrawal · 2018 [cited by examiner]
US 20180210883A1 · Ang · 2018 [cited by examiner]
US 20180329987A1 · Tata · 2018 [cited by examiner]
US 20190121801A1 · Jethwa · 2019 [cited by examiner]
US 20190138648A1 · Gupta · 2019 [cited by examiner]
US 20190197605A1 · Sadler · 2019 [cited by examiner]
US 20190236144A1 · Hou · 2019 [cited by examiner]
US 20190384815A1 · Patel · 2019 [cited by examiner]
US 20200065385A1 · Dreher · 2020 [cited by examiner]
US 20200089700A1 · Ericson · 2020 [cited by examiner]
US 20200089760A1 · Ericson · 2020 [cited by examiner]
US 20200110803A1 · Djalali · 2020 [cited by examiner]
US 20200134103A1 · Mankovskii · 2020 [cited by examiner]
US 20230134235A1 · Setlur · 2023 [cited by examiner]
Cited By (1)
US 1,142,410