IP Library Granted Patent US 12,499,141
Granted Patent B2
US 12,499,141 · App. 17/472,250 · Granted Dec 16, 2025

Ontology-based data visualization

Inventors: Yara Rizk (Cambridge, MA); Zilu Tang (Cambridge, MA); Yasaman Khazaeni (Needham, MA); Vatche Isahagian (Belmont, MA)
Assignee: International Business Machines Corporation
G06F16/3344G06F16/243G06F16/248G06F16/338G06F16/90332G06F40/169G06F40/284G06F40/30G06T11/206
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,499,141
App. No.
17/472,250
Granted
Dec 16, 2025
Kind
B2
Abstract

A data visualization method, system, and computer program product that includes identifying an intent from a natural language query in a conversation with a conversational system, utilizing verbiage from the natural language query and the intent to determine one or more data columns for visualization of results of the natural language query, and displaying a visualization of the determined one or more data columns.

Claims (55)

1 . A computer-implemented method, comprising:

identifying an intent from terminology of a natural language query input by a user in a conversation within a chatbot interface of a conversational system;

receiving a natural language request from the user in the conversation within the conversational system for a data visualization;

utilizing verbiage from the natural language query, the identified intent, and entities identified from the natural language request to determine one or more data columns for the data visualization;

extracting, from the determined one or more data columns, one or more features indicative of a data-type of the determined one or more data columns;

annotating, based on an ontology, the extracted one or more features into a plot-label;

generating, based on the determined one or more data columns, multiple plots, wherein each plot of the generated multiple plots is of a different plot-type;

calculating, based on an amount of information associated with each plot of the generated multiple plots, a respective interestingness score for each plot of the generated multiple plots;

selecting a first plot of the generated multiple plots based on the respective interestingness score of the first plot being highest among interestingness scores of the generated multiple plots, and a plot-type of the first plot corresponding to the annotated plot-label; and

controlling display, in the conversation with the conversational system, of an ontology-based data visualization in a form of a graph that corresponds to the selected first plot, wherein the ontology-based data visualization leverages context of the conversation.

2 . The computer-implemented method of claim 1 , wherein the respective interestingness score of the first plot, that is highest, is less than a predefined threshold value.

3 . The computer-implemented method of claim 1 , further comprising:

requesting a feedback return for a quality of the ontology-based data visualization; and

learning from the feedback return with a trained supervised learning model by creating a threshold for an acceptable value for the highest interestingness score of the first plot based upon the trained supervised learning model trained at least in-part with user feedback.

4 . The computer-implemented method of claim 1 , further comprising:

requesting a feedback return for a quality of the ontology-based data visualization; and

learning from the feedback return by creating a threshold for an acceptable value for the highest interestingness score of the first plot and dynamically adjusting the ontology to change the annotated plot-label.

5 . The computer-implemented method of claim 1 , wherein the computer-implemented method is embodied in a cloud-computing environment.

6 . A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:

identifying an intent from terminology of a natural language query input by a user in a conversation within a chatbot interface of a conversational system;

receiving a natural language request from the user in the conversation within the conversational system for a data visualization;

utilizing verbiage from the natural language query, the identified intent, and entities identified from the natural language request to determine one or more data columns for the data visualization;

extracting, from the determined one or more data columns, one or more features indicative of a data-type of the determined one or more data columns;

annotating, based on an ontology, the extracted one or more features into a plot-label;

generating, based on the determined one or more data columns, multiple plots, wherein each plot of the generated multiple plots is of a different plot-type;

calculating, based on an amount of information associated with each plot of the generated multiple plots, a respective interestingness score for each plot of the generated multiple plots;

selecting a first plot of the generated multiple plots based on the respective interestingness score of the first plot being highest among interestingness scores of the generated multiple plots, and a plot-type of the first plot corresponding to the annotated plot-label; and

controlling display, in the conversation with the conversational system, of an ontology-based data visualization in a form of a graph that corresponds to the selected first plot, wherein the ontology-based data visualization leverages context of the conversation.

7 . The computer program product of claim 6 , wherein the respective interestingness score of the first plot, that is highest, is less than a predefined threshold value.

8 . The computer program product of claim 6 , wherein the program instructions further cause the computer to perform:

requesting a feedback return for a quality of the ontology-based data visualization; and

learning from the feedback return with a trained supervised learning model by creating a threshold for an acceptable value for the highest interestingness score of the first plot based upon the trained supervised learning model trained at least in-part with user feedback.

9 . The computer program product of claim 6 , wherein the program instructions further cause the computer to perform:

requesting a feedback return for a quality of the ontology-based data visualization; and

learning from the feedback return by creating a threshold for an acceptable value for the highest interestingness score of the first plot and dynamically adjusting the ontology to change the annotated plot-label.

10 . A system, comprising:

a processor; and

a memory, the memory storing instructions to cause the processor to:

identify an intent from terminology of a natural language query input by a user in a conversation within a chatbot interface of a conversational system;

receive a natural language request from the user in the conversation within the conversational system for a data visualization;

utilize verbiage from the natural language query, the identified intent, and entities identified from the natural language request to determine one or more data columns for the data visualization;

extract, from the determined one or more data columns, one or more features indicative of a data-type of the determined one or more data columns;

annotate, based on an ontology, the extracted one or more features into a plot-label;

generate, based on the determined one or more data columns, multiple plots, wherein each plot of the generated multiple plots is of a different plot-type;

calculate, based on an amount of information associated with each plot of the generated multiple plots, a respective interestingness score for each plot of the generated multiple plots;

select a first plot of the generated multiple plots based on the respective interestingness score of the first plot being highest among interestingness scores of the generated multiple plots, and a plot-type of the first plot corresponding to the annotated plot-label; and

control display, in the conversation with the conversational system, of an ontology-based data visualization in a form of a graph that corresponds to the selected first plot, wherein the ontology-based data visualization leverages context of the conversation.

11 . The system of claim 10 , wherein the respective interestingness score of the first plot, that is highest, is less than a predefined threshold value.

12 . The system of claim 10 , further cause the processor to:

request a feedback return for quality of the ontology-based data visualization; and

learn from the feedback return by creating a threshold for an acceptable value for the highest interestingness score of the first plot, and dynamically adjusting the ontology to change the annotated plot-label.

13 . The system of claim 10 , further cause the processor to:

request a feedback return with a trained supervised learning model for a quality of the ontology-based data visualization; and

learn from the feedback return by creating a threshold for an acceptable value for the highest interestingness score of the first plot based upon the trained supervised learning model trained at least in-part with user feedback.

14 . The system of claim 10 , wherein the system is embodied in a cloud-computing environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2021
From: RIZK, YARA; TANG, ZILU; KHAZAENI, YASAMAN; ISAHAGIAN, VATCHE
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 057543/0820 →
Continuity (1)
Related Publication 20230081598A1 · Mar 16, 2023
References Cited (28)
US 8812298B1 · Wolfram · 2014 [cited by examiner]
US 9720984B2 · McClung · 2017 [cited by applicant]
US 10169454B2 · Ait-Mokhtar · 2019 [cited by applicant]
US 10452668B2 · Heimendinger et al. · 2019 [cited by applicant]
US 10572473B2 · Allen et al. · 2020 [cited by applicant]
US 10762299B1 · Parastatidis · 2020 [cited by examiner]
US 10846318B1 · McNabney et al. · 2020 [cited by applicant]
US 10943072B1 · Jaganmohan · 2021 [cited by examiner]
US 11030255B1 · Tory et al. · 2021 [cited by applicant]
US 11263405B2 · Gnanasambandam · 2022 [cited by examiner]
US 20040230572A1 · Omoigui · 2004 [cited by applicant]
US 20120150548A1 · Rajagopalan · 2012 [cited by examiner]
US 20120191716A1 · Omoigui · 2012 [cited by applicant]
US 20130096946A1 · Shah · 2013 [cited by applicant]
US 20140280256A1 · Wolfram · 2014 [cited by examiner]
US 20150006518A1 · Baumgartner · 2015 [cited by examiner]
US 20160179945A1 · Lastra Diaz · 2016 [cited by applicant]
US 20180137179A1 · Kawanabe · 2018 [cited by examiner]
US 20190340281A1 · Maiti · 2019 [cited by examiner]
US 20190347118A1 · Mukherjee · 2019 [cited by examiner]
US 20200019608A1 · Gentile · 2020 [cited by examiner]
US 20200090189A1 · Tutuk · 2020 [cited by examiner]
US 20200117740A1 · Schmidt · 2020 [cited by examiner]
US 20210326519A1 · Lin · 2021 [cited by examiner]
US 20220156823A1 · Tremblay · 2022 [cited by examiner]
WO WO2020042530A1 · 2020 [cited by applicant]
Adadi et al. Aug. 5, 2018, “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)” (Year: 2018). [cited by examiner]
Mel, et al. “The NIST Definition of Cloud Computing”. Recommendations of the National Institute of Standards and Technology. Nov. 16, 2015. [cited by applicant]