IP Library Granted Patent US 9,881,210
Granted Patent B2
US 9,881,210 · App. 14/840,186 · Granted Jan 30, 2018

Generating a computer executable chart visualization by annotating a static image

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,881,210
App. No.
14/840,186
Granted
Jan 30, 2018
Kind
B2
Abstract

A computer executable chart visualization may be generated from a static image of the chart visualization. The static image, a set of pre-defined chart elements, and a portion of the static image that corresponds to a pre-defined chart element may each be identified. A mapping between the portion of the static image and the pre-defined chart elements may be determined, and a visual or logical property of each portion of the static image may be determined. A declarative description of the chart visualization may be generated by setting the visual properties for each of the respective pre-defined chart elements.

Claims (52)

1. A method of generating a computer executable chart visualization from a static image of said chart visualization, the method comprising the steps of:

identifying said static image of said chart visualization;

identifying a set of pre-defined chart elements;

identifying one or more portions of said static image as corresponding to one or more respective pre-defined chart elements, wherein the identifying one or more portions comprises:

using image recognition to locate one or more visual borders of said one or more pre-defined chart elements within the static image; and

receiving, from a user, input regarding accuracy and completeness of the one or more visual borders of said one or more pre-defined chart elements within the static image, wherein input regarding completeness includes one or more annotations regarding missing chart elements;

determining a mapping between said one or more portions of the static image and said one or more respective pre-defined chart elements;

determining visual properties of each of said one or more portions of said static image; and

generating a declarative description of the chart visualization by setting the visual properties for each of said one or more respective pre-defined chart elements.

2. The method of claim 1 , further comprising rendering the declarative description to create an interactive visualization corresponding in appearance to the static image, wherein the interactive visualization includes a dynamic visualization of an editable chart generated from the declarative description and rendered with graphic data.

3. The method of claim 1 , wherein said using image recognition to locate one or more visual borders of pre-defined chart elements within the static image comprises:

transforming the static image into abstract shapes using known image recognition techniques; and

comparing an assembly of the abstract shapes against a number of known pre-defined chart elements and selecting a best matching chart element.

4. The method of claim 1 , wherein said pre-defined chart elements include axes, elements, gridlines, and text.

5. The method of claim 1 , wherein the declarative description is directly translated into computer executable code.

6. The method of claim 1 , wherein visual properties include color, line color, line width, font size, and label format.

7. The method of claim 1 , wherein input regarding accuracy and completeness of the one or more visual borders of said one or more pre-defined chart elements within the static image is received from a cognitive machine learning system.

8. The method of claim 1 , wherein the static image includes portions corresponding to dynamic elements.

9. A system for generating a computer executable chart visualization from a static image of said chart visualization, the system comprising:

a computer readable medium with program instructions stored thereon; and

one or more processors configured to execute the program instructions to perform the method comprising:

identifying said static image of said chart visualization;

identifying a set of pre-defined chart elements;

identifying one or more portions of said static image as corresponding to one or more respective pre-defined chart elements, wherein the identifying one or more portions comprises:

using image recognition to locate one or more visual borders of said one or more pre-defined chart elements within the static image; and

receiving, from a user, input regarding accuracy and completeness of the one or more visual borders of said one or more pre-defined chart elements within the static image, wherein input regarding completeness includes one or more annotations regarding missing chart elements;

determining a mapping between said one or more portions of the static image and said one or more respective pre-defined chart elements;

determining visual properties of each of said one or more portions of said static image; and

generating a declarative description of the chart visualization by setting the visual properties for each of said one or more respective pre-defined chart elements.

10. The system of claim 9 , wherein the method further comprises rendering the declarative description to create an interactive visualization corresponding in appearance to the static image, wherein the interactive visualization includes a dynamic visualization of an editable chart generated from the declarative description and rendered with graphic data.

11. The system of claim 9 , wherein said using image recognition to locate one or more visual borders of pre-defined chart elements within the static image comprises:

transforming the static image into abstract shapes using known image recognition techniques; and

comparing an assembly of the abstract shapes against a number of known pre-defined chart elements and selecting a best matching chart element.

12. The system of claim 9 , wherein said pre-defined chart elements include axes, elements, gridlines, and text.

13. The system of claim 9 , wherein the declarative description is directly translated into computer executable code.

14. The system of claim 9 , wherein visual properties include color, line color, line width, font size, and label format.

15. A computer program product for generating a computer executable chart visualization from a static image of said chart visualization, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause a device to perform the method comprising:

identifying said static image of said chart visualization;

identifying a set of pre-defined chart elements;

identifying one or more portions of said static image as corresponding to one or more respective pre-defined chart elements, wherein the identifying one or more portions comprises:

using image recognition to locate one or more visual borders of said one or more pre-defined chart elements within the static image; and

receiving, from a user, input regarding accuracy and completeness of the one or more visual borders of said one or more pre-defined chart elements within the static image, wherein input regarding completeness includes one or more annotations regarding missing chart elements;

determining a mapping between said one or more portions of the static image and said one or more respective pre-defined chart elements;

determining visual properties of each of said one or more portions of said static image; and

generating a declarative description of the chart visualization by setting the visual properties for each of said one or more respective pre-defined chart elements.

16. The computer program product of claim 15 , wherein the method further comprises rendering the declarative description to create an interactive visualization corresponding in appearance to the static image, wherein the interactive visualization includes a dynamic visualization of an editable chart generated from the declarative description and rendered with graphic data.

17. The computer program product of claim 15 , wherein said using image recognition to locate one or more visual borders of pre-defined chart elements within the static image comprises:

transforming the static image into abstract shapes using known image recognition techniques; and

comparing an assembly of the abstract shapes against a number of known pre-defined chart elements and selecting a best matching chart element.

18. The computer program product of claim 15 , wherein said pre-defined chart elements include axes, elements, gridlines, and text.

19. The computer program product of claim 15 , wherein the declarative description is directly translated into computer executable code.

20. The computer program product of claim 15 , wherein visual properties include color, line color, line width, font size, and label format.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: AIRBNB, INC.
Reel/Frame 056427/0193 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 31, 2015
From: BROADBENT, PETER R.; COOLEN, LEO A.; VAN HAM, FRANCISCUS J. J.; WILLS, GRAHAM J.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 036456/0001 →