IP Library Granted Patent US 10,282,878
Granted Patent B2
US 10,282,878 · App. 15/188,423 · Granted May 7, 2019

Method and apparatus for annotating a graphical output

Inventor: Ehud Baruch Reiter (Aberdeen, GB)
Assignee: ARRIA DATA2TEXT LIMITED
G06T11/60G06F3/04842G06F17/2229G06F17/241G06F17/248G06F17/2881G06T11/206G16H15/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,282,878
App. No.
15/188,423
Granted
May 7, 2019
Kind
B2
Abstract

Various methods are provided for generating and annotating a graph. One example method may include determining one or more key patterns in a primary data channel, wherein the primary data channel is derived from raw input data in response to a constraint being satisfied. A method may further include determining one or more significant patterns in one or more related data channels. A method may further include generating a natural language annotation for at least one of the one or more key patterns or the one or more significant patterns. A method may further include generating a graph that is configured to be displayed in a user interface, the graph having at least a portion of the one or more key patterns, the one or more significant patterns and the natural language annotation.

Claims (70)

1. A computer-implemented method for transforming raw input data that is at least partially expressed in a non-linguistic format into a format that can be expressed linguistically in one or more phrases with a graphical representation of the raw input data, the method comprising:

detecting, by a processor, one or more patterns in a data channel derived from raw input data;

identifying, by the processor, one or more patterns in another data channel also derived from the raw input data;

determining, by the processor, from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generating, using a natural language generation system that is configured to execute on the processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by:

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generating, using the processor, a graphical output for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

2. A method according to claim 1 , further comprising:

detecting, by the processor, one or more patterns in the data channel by:

identifying, by the processor, one or more patterns wherein a pattern is at least one of a trend, spike or step in the data channel;

assigning, by the processor, an importance level to the one or more patterns; and

identifying, by the processor, one or more key patterns of the one or more patterns, wherein a key pattern is a pattern that exceeds a predefined importance level.

3. A method according to claim 2 , further comprising:

determining, by the processor, a time period to be displayed in a graph, wherein the time period chosen for the graph is the time period in which the one or more key patterns are displayed.

4. A method according to claim 3 , further comprising:

identifying, by the processor, one or more patterns in the another data channel by:

assigning, by the processor, an importance level to one or more unexpected patterns; and

identifying, by the processor, one or more significant patterns of the one or more unexpected patterns, wherein a significant pattern is an unexpected pattern.

5. A method according to claim 4 , further comprising:

determining, by the processor, that the one or more patterns identified in an another data channel violate a predetermined constraint; and

determining, by the processor, that the one or more patterns are one or more unexpected patterns.

6. A method according to claim 4 , wherein the one or more unexpected patterns are identified within the time period of the graph.

7. A method according to claim 4 , further comprising:

generating, by the processor, the graph based on the graphical output, wherein the graph comprises at least a portion of the data channel that contains the one or more key events, the another data channel that contains the one or more significant events and the one or more contextual channels, the at least a portion of the one or more key events, the one or more significant events and the one or more contextual channels being annotated by the one or more phrases;

generating, by the processor, a narrative; and

causing, by the processor, the graph to be displayed in a user interface.

8. A method according to claim 7 , wherein additional information related to the one or more phrases is displayed in response to a user interaction with the user interface.

9. A method according to claim 7 , wherein the one or more key patterns and the one or more significant patterns are highlighted in response to a user interaction with the one or more phrases in the user interface.

10. A method according to claim 1 , wherein the context information provides at least background or circumstance information having influenced the one or more patterns in the data channel or the one or more patterns in the another data channel.

11. A method according to claim 1 , wherein the data channel is selected from one or more data channels derived from the raw input data, the data channel is selected based on an indication received from at least one of a user, an alarm or another computing device.

12. A method according to claim 1 , further comprising:

receiving, by the processor, an indication of a determined scale from a user via a user interface; and

generating, by the processor, the graphical output based on the data channel, the another data channel and the one or more phrases in accordance with the determined scale.

13. A method according to claim 1 , further comprising:

receiving, by the processor, a selection of one or more of the primary data channel, the another data channel, or a contextual channel from a user via a user interface; and

generating, by the processor, the graphical output for display in the user interface, based on the selection of the one or more of the primary data channel, the another data channel, or the contextual channel and the natural language annotation for the selection.

14. An apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

detect one or more patterns in a data channel derived from raw input data;

identify one or more patterns in another data channel also derived from the raw input data;

determine from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generate, using a natural language generation system that is configured to execute on a processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by:

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generate a graphical output, for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

15. An apparatus according to claim 14 , wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

identify one or more patterns in the primary data channel;

assign an importance level to the one or more patterns; and

identify one or more key patterns of the one or more patterns in an instance in which the importance level of a pattern of the one or more patterns exceeds a threshold defined by a domain model.

16. An apparatus according to claim 15 , wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine a time period to be represented by the graphical output, wherein the time period is configured such that at least a portion of the one or more key patterns occur within the time period.

17. An apparatus according to claim 15 , wherein the at least one memory including the computer program code is further configured to, with the at least one processor, cause the apparatus to:

determine one or more patterns in one or more related data channels that corresponds to an occurrence of the one or more key patterns; and

determine one or more significant patterns of the one or more patterns, wherein a significant pattern is a pattern that represents an anomaly.

18. An apparatus according to claim 14 , wherein context information provides at least background or circumstance information having influenced the one or more key patterns in the primary data channel or the one or more significant patterns in the one or more related data channels.

19. An apparatus according to claim 14 , wherein additional information related to the natural language annotation is displayed in the user interface in response to a user interaction with the user interface.

20. A computer program product comprising:

at least one computer readable non-transitory memory medium having program code instructions stored thereon, the program code instructions which when executed by an apparatus causes the apparatus at least to:

detect one or more patterns in a data channel derived from raw input data;

identify one or more patterns in another data channel also derived from the raw input data;

determine from one or more contextual channels context information for at least one of the one or more patterns in the data channel or the one or more patterns in the another data channel;

generate, using a natural language generation system that is configured to execute on a processor, one or more phrases describing the one or more patterns in the data channel and the one or more patterns in the another data channel, the one or more phrases generated by:

generating at least one message when a pattern from the one or more patterns in the data channel or a pattern from the one or more patterns in the another data channel satisfies one or more message requirements,

selecting one or more words to express a concept or a relation in the at least one message, and

applying a grammar to the selected one or more words; and

generate a graphical output, for display in a user interface, based on the data channel, the another data channel, the one or more contextual channels, and the one or more phrases, wherein the one or more phrases are interactively annotated on the graphical output of the data channel and the another data channel.

Assignments (5)
SECURITY INTEREST Recorded Sep 9, 2025
From: ARRIA DATA2TEXT LIMITED
To: COLLATERAL HOLDINGS, INC.
Reel/Frame 072202/0480 →
RELEASE OF SECURITY INTEREST Recorded Jun 24, 2022
From: AONAS HOLDINGS S.A.
To: ARRIA DATA2TEXT LIMITED
Reel/Frame 061212/0089 →
SECURITY INTEREST Recorded Mar 17, 2017
From: ARRIA DATA2TEXT LIMITED
To: AONAS HOLDINGS S.A.
Reel/Frame 041618/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 21, 2016
From: REITER, EHUD BARUCH
To: DATA2TEXT LIMITED
Reel/Frame 038975/0099 →
CHANGE OF NAME Recorded Jun 21, 2016
From: DATA2TEXT LIMITED
To: ARRIA DATA2TEXT LIMITED
Reel/Frame 038975/0166 →
Continuity (3)
Continuation 14634035 · Feb 27, 2015
Continuation PCTUS2012053128 · Aug 30, 2012
Related Publication 20170018107A1 · Jan 19, 2017