IP Library Granted Patent US 11,195,050
Granted Patent B2
US 11,195,050 · App. 16/731,778 · Granted Dec 7, 2021

Machine learning to generate and evaluate visualizations

Inventors: Saurabh Abhyankar (McLean, VA); Glenn Boysko (Vienna, VA); Scott Rigney (Vienna, VA); Timothy Lang (Vienna, VA)
Assignee: MicroStrategy Incorporated
G06K9/6253G06K9/6218G06K9/6257G06K9/6269G06N5/04G06N20/00G06T11/206G06T11/60G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,195,050
App. No.
16/731,778
Granted
Dec 7, 2021
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer-readable storage media, for machine learning to generate and evaluate visualizations. In some implementations, a system determines properties of a dataset. The system generates visualization specifications that each define a different visualization for the dataset, wherein the visualization specifications specify different subsets of the dataset being illustrated with different visualization formats. The system evaluates the visualization specifications using a machine learning model trained based on user feedback for visualizations for multiple datasets. The system selects a subset of the visualization specifications based on output of the machine learning model. The system provides, for display, visualization data for the subset of visualization specifications that were selected based on the output of the machine learning model.

Claims (53)

1. A method performed by one or more computers, the method comprising:

determining, by the one or more computers, properties of a dataset;

generating, by the one or more computers, visualization specifications that each define a different visualization for the dataset, wherein the visualization specifications specify different subsets of the dataset being illustrated with different visualization formats;

evaluating, by the one or more computers, the visualization specifications using a machine learning model trained based on user feedback for visualizations for multiple datasets;

selecting, by the one or more computers, a subset of the visualization specifications based on output of the machine learning model; and

providing, by the one or more computers, visualization data for display, the visualization data corresponding to the selected subset of the visualization specifications selected based on the output of the machine learning model.

2. The method of claim 1 , wherein determining properties of the dataset comprises determining structural properties of the dataset including at least one of a number of columns, a number of attributes, a number of metrics, a value for an aggregation for a data range, a span of values for a data range, a data type for a data range, or a statistical measure for a data range.

3. The method of claim 1 , wherein determining properties of the dataset comprises determining semantic properties of the dataset.

4. The method of claim 1 , wherein the different visualization formats comprise different visualization types including two or more from the group consisting of a graph, a line chart, a bar chart, a pie chart, a scatterplot, a heat map, a geographical map, a word-size map, a bubble chart, a hierarchy chart, a waterfall chart, a radar chart, a statistical chart, and an area chart.

5. The method of claim 1 , wherein the trained machine learning model has been trained to output scores indicating a level of usefulness of visualizations, where the scores are conditioned on properties of the datasets the visualizations are derived from, wherein the machine learning model is configured to generate the scores based on input indicating visualization properties and dataset properties, wherein the trained machine learning model has been trained based on user feedback data indicating actions of multiple users after being presented visualizations representing portions of different datasets.

6. The method of claim 5 , wherein the user feedback data comprises survey responses indicating ratings that users applied to visualizations.

7. The method of claim 5 , wherein the user feedback data comprises usage data comprising measures of user behavior, other than rating of visualizations, in response to the presentation of the visualizations.

8. The method of claim 7 , wherein the usage data comprises at least one of:

a number of times a visualization was viewed, interacted with, shared, or included in a document by one or more users; or

an amount of time that a visualization was viewed or interacted with by one or more users.

9. The method of claim 1 , wherein evaluating the visualization specifications comprises:

for each of the visualization specifications:

providing, to the trained machine learning model, input that indicates (i) properties of a visualization and (ii) properties of the dataset; and

receiving, as output of the trained machine learning model, a score for the visualization;

wherein selecting a subset of the visualization specifications comprises:

ranking the visualization specifications based on the scores provided as output of the machine learning model; and

selecting a highest-ranking subset of the visualization specifications.

10. The method of claim 1 , comprising receiving data indicating interaction with a control on a user interface; and

wherein providing the visualization data is performed in response to receiving the data indicating the interaction with the control on the user interface.

11. The method of claim 10 , further comprising determining a portion of the data set that has been selected using the user interface;

wherein selecting the subset of the visualization specifications comprises filtering the visualization specifications to exclude one or more visualization specifications that do not represent the determined portion of the data set.

12. The method of claim 11 , wherein determining that a portion of the data set has been selected comprises determining that a column of the data set has been selected; and

wherein selecting the subset of the visualization specifications comprises selecting only visualization specifications that are generated from or provide information regarding the selected column.

13. The method of claim 1 , wherein the machine learning model comprises an artificial neural network, a classifier, a support vector machine, a decision tree, a regression model, a clustering model, a Gaussian process model, a genetic algorithm, or a reinforcement learning model.

14. The method of claim 1 , further comprising identifying a user of a computing device;

wherein selecting the subset of the visualization specifications is personalized based on the identity of the user of the computing device; and

wherein providing the visualization data comprises providing the visualization data to the computing device for display by the computing device.

15. The method of claim 14 , further comprising identifying one or more topics based on a role of the user, a user profile for the user, or a usage history for the user;

wherein selecting the subset of the visualization specifications is personalized by adjusting scores for the visualization specifications to indicate increased relevance for visualization specifications associated with the one or more topics.

16. The method of claim 14 , further comprising determining, based on a usage history for the user, one or more visualization formats with which the user previously interacted;

wherein selecting the subset of the visualization specifications is personalized by adjusting scores to indicate increased relevance for visualization specifications specifying visualization formats that have at least a minimum level of similarity to the one or more visualization formats with which the user previously interacted.

17. The method of claim 10 , wherein evaluating the visualization specifications using the machine learning model is performed before receiving the data indicating the interaction with the control.

18. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

determining, by the one or more computers, properties of a dataset;

generating, by the one or more computers, visualization specifications that each define a different visualization for the dataset, wherein the visualization specifications specify different subsets of the dataset being illustrated with different visualization formats;

evaluating, by the one or more computers, the visualization specifications using a machine learning model trained based on user feedback for visualizations for multiple datasets;

selecting, by the one or more computers, a subset of the visualization specifications based on output of the machine learning model; and

providing, by the one or more computers, visualization data for display, the visualization data corresponding to the selected subset of the visualization specifications selected based on the output of the machine learning model.

19. The system of claim 18 , wherein the trained machine learning model has been trained to output scores for visualizations, wherein the trained machine learning model is configured to generate a score for a visualization with respect to a dataset based on input indicating (i) properties of the visualization and (ii) properties of the dataset.

20. The system of claim 18 , wherein the trained machine learning model has been trained based on examples of actions that users performed after being presented one or more visualizations, wherein at least some of the examples respectively indicate actions performed after presentation of visualizations for different data sets.

21. One or more non-transitory computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

determining, by the one or more computers, properties of a dataset;

generating, by the one or more computers, visualization specifications that each define a different visualization for the dataset, wherein the visualization specifications specify different subsets of the dataset being illustrated with different visualization formats;

evaluating, by the one or more computers, the visualization specifications using a machine learning model trained based on user feedback for visualizations for multiple datasets;

selecting, by the one or more computers, a subset of the visualization specifications based on output of the machine learning model; and

providing, by the one or more computers, visualization data for display, the visualization data corresponding to the selected subset of the visualization specifications selected based on the output of the machine learning model.

Assignments (5)
CHANGE OF NAME Recorded Sep 19, 2025
From: MICROSTRATEGY INCORPORATED
To: STRATEGY INC
Reel/Frame 072974/0529 →
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT AT REEL/FRAME: 056647/0687, REEL/FRAME: 057435/0023, REEL/FRAME: 059256/0247, REEL/FRAME: 062794/0255 AND REEL/FRAME: 066663/0713 Recorded Sep 26, 2024
From: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS SUCCESSOR IN INTEREST TO U.S. BANK NATIONAL ASSOCIATION, IN ITS CAPACITY AS COLLATERAL AGENT FOR THE SECURED PARTIES
To: MICROSTRATEGY INCORPORATED; MICROSTRATEGY SERVICES CORPORATION
Reel/Frame 069065/0539 →
SUPPLEMENTARY PATENT SECURITY AGREEMENT Recorded Feb 18, 2022
From: MICROSTRATEGY INCORPORATED; MICROSTRATEGY SERVICES CORPORATION
To: U.S. BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 059256/0247 →
SECURITY INTEREST Recorded Jun 22, 2021
From: MICROSTRATEGY INCORPORATED
To: U.S. BANK NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 056647/0687 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2020
From: ABHYANKAR, SAURABH; BOYSKO, GLENN; RIGNEY, SCOTT; LANG, TIMOTHY
To: MICROSTRATEGY INCORPORATED
Reel/Frame 052020/0024 →
Continuity (2)
Provisional Application 62801320 · Feb 5, 2019
Related Publication 20200250472A1 · Aug 6, 2020
Cited By (1)
US 12,455,900