IP Library › Granted Patent US 12,417,569
Granted Patent B2
US 12,417,569 · App. 18/525,940 · Granted Sep 16, 2025

Computer implemented method and system for integrative causal modeling and transfer

Inventors: Wlodek Zadrozny (Charlotte, NC); Wenwen Dou (Charlotte, NC); Victor Zitian Chen (Charlotte, NC); Seethalakshmi Gopalakrishnan (Charlotte, NC)
Assignee: The University Of North Carolina At Charlotte
G06T11/206G06N3/08G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,417,569
App. No.
18/525,940
Granted
Sep 16, 2025
Kind
B2
Abstract

A hypergraph user interface for interaction with a system for integrative causal modeling from natural language text and including a natural language text input element configured to receive a designation of a natural language text file or such a file. The input element causes the designation or natural language text file to be communicated to at least one natural language processing module. The hypergraph user interface further includes a causal hypergraph visualization element representative of directed hypergraph data generated from the at least one natural language processing module. Selection of a hypergraph element of the causal hypergraph representation causes the hypergraph user interface to communicate an indication of the hypergraph element to a querying module and generate an informative element indicative of at least one of a causal link, a causal statement, or a sentence of the at least one natural text file on which the hypergraph element is based.

Claims (34)

1. A hypergraph user interface for interaction with a system for integrative causal modeling from natural language text, the hypergraph user interface comprising:

a natural language text input element configured to receive at least one of a designation of a natural language text file or a natural language text file and, in response to the received at least one designation or natural language text file, cause the at least one designation or natural language text file to be communicated to at least one natural language processing module of the system; and

a causal hypergraph visualization element representative of directed hypergraph data generated from the at least one natural language processing module of the system,

wherein, in response to selection of a hypergraph element of the causal hypergraph representation, the hypergraph user interface is configured to communicate an indication of the hypergraph element to a querying module of the system and generate, in association with the causal hypergraph visualization, an informative element indicative of at least one of a causal link, a causal statement, or a sentence of the at least one natural text file on which the hypergraph element is based.

2. The hypergraph user interface of claim 1 , wherein, when the directed hypergraph data includes multivariable hypergraph data, at least one hypergraph element of the casual hypergraph visualization indicates multivariable, directed causality.

3. The hypergraph user interface of claim 1 , wherein the at least one natural language processing module of the system includes at least one module suitable for transfer learning.

4. The hypergraph user interface of claim 1 , wherein, in response to the selection of the hypergraph element, the hypergraph user interface is configured to reorient the casual hypergraph visualization element to emphasize the selected hypergraph element.

5. The hypergraph user interface of claim 1 , wherein the informative element includes at least one of a table or chart.

6. The hypergraph user interface of claim 1 , wherein the causal hypergraph visualization element includes a plurality of nodes associated with a plurality of taxonomy categories.

7. The hypergraph user interface of claim 6 , wherein the causal hypergraph visualization element includes a plurality of directed edges connecting two or more nodes of the plurality of nodes, wherein each directed edge includes an indicator indicative of at least one of a number of causal statements associated with the directed edge, the strength of a causal association of at least one causal statement associated with the directed edge, or a strength of the association between at least one of the connected nodes and at least one casual statement associated with the directed edge.

8. The hypergraph user interface of claim 6 , wherein the causal hypergraph visualization element includes a first-order nodal diagram, wherein the first-order nodal diagram includes a performance node representing the nodes of the directed hypergraph data associated with the taxonomy categories and a non-performance node representing one or more portions of causal statements not associated with at least one of the taxonomy categories.

9. The hypergraph user interface of claim 8 , wherein the hypergraph user interface is configured to generate, in response to a selection of one of the performance node or the non-performance node, a second-order causal hypergraph visualization element including nodes associated with the selected node of the first-order nodal diagram.

10. The hypergraph user interface of claim 9 , wherein the second-order causal hypergraph visualization element includes a plurality of edges, each edge connecting at least one node associated with a taxonomy category to at least one other node, the plurality of edges indicating multivariable, directed hypergraph data.

11. The hypergraph user interface of claim 1 , wherein the hypergraph user interface is configured such that at least one of a layout of the hypergraph user interface including at least one of the causal hypergraph visualization element or the informative element or a user-initiated change in the at least one of the causal hypergraph visualization element or the informative element is provided as visualization training data to a visualization module configured to generate the causal hypergraph visualization element or the informative element.

12. The hypergraph user interface of claim 11 , wherein the visualization module comprises a deep learning network, and wherein the visualization training data is utilized to train the deep learning network.

13. A system for integrative causal modeling from natural language text, the system comprising:

a causal identification module comprising instructions stored in at least one memory and executable by one or more processors to cause the causal identification module to identify, from at least one natural language text file, a plurality of causal links utilizing natural language processing, each causal link including a causal portion and an output portion;

a normalization module comprising instructions stored in at least one memory and executable by one or more processors to cause the normalization module to identify whether each causal portion and each output portion of each causal link is associated with a taxonomy category of a plurality of predefined taxonomy categories; and

a visualization and querying module comprising instructions stored in at least one memory and executable by one or more processors to cause the visualization and querying module to generate a causal hypergraph visualization utilizing interface hardware associated with the system, the causal hypergraph visualization generated from directed hypergraph data representing the causal statements and associations with the predefined taxonomy categories of the at least one natural language text file.

14. The system of claim 13 , wherein the visualization and querying module further comprising instructions stored in the at least one memory and executable by the one or more processors to cause the visualization and querying module to:

receive a selection of an element of the causal hypergraph visualization; and

generate, in response to the selected element of the causal hypergraph visualization, at least one of a causal link indicated by the selected element, at least one causal statement on which the at least one causal link is based, at least one sentence of the at least one natural language text file on which the at least one causal link is based, or a combination of the preceding.

15. The system of claim 13 , wherein the causal identification module further comprising instructions stored in the at least one memory and executable by the one or more processors to cause the causal identification module to:

receive the at least one natural language text file; and

identify a plurality of causal statements included in the at least one natural language text file utilizing natural language processing,

wherein the casual links are identified from the identified plurality of causal statements.

16. The system of claim 13 , wherein the visualization and querying module further comprising instructions stored in the at least one memory and executable by the one or more processors to cause the visualization and querying module to:

generate the causal hypergraph visualization utilizing at least one of a haptic interface device, a virtual reality interface device, or a multimodal interface device; or

receive a selection of an element of the causal hypergraph visualization communicated from at least one of the haptic interface device, the virtual reality interface device, or the multimodal interface device.

17. The system of claim 13 , wherein the directed hypergraph data comprises multivariable, directed hypergraph data.

18. The system of claim 13 , further comprising a hypergraph data generation module comprising instructions stored in at least one memory and executable by one or more processors to cause the hypergraph data generation module to:

generate the directed hypergraph data based on the identified plurality of causal links and the associations of the causal links with the plurality of predefined taxonomy categories.

19. The system of claim 13 , further comprising a translation module communicatively coupled to the visualization and querying module, the at least one natural language text file, and at least one data file including the directed hypergraph data and data indicative of the casual links normalized with respect to the predefined taxonomy categories.

20. The system of claim 13 , wherein each of the causal identification module and the normalization module includes at least one module suitable for transfer learning.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 1, 2023
From: ZADROZNY, WLODEK; DOU, WENWEN; CHEN, VICTOR ZITIAN; GOPALAKRISHNAN, SEETHALAKSHMI
To: THE UNIVERSITY OF NORTH CAROLINA AT CHARLOTTE
Reel/Frame 065727/0929 →
Continuity (2)
Provisional Application 63429586 · Dec 2, 2022
Related Publication 20240185491A1 · Jun 6, 2024
References Cited (26)
US 7139752B2 · Broder et al. · 2006 [cited by applicant]
US 8887286B2 · Dupont et al. · 2014 [cited by applicant]
US 11593631B2 · Dalli et al. · 2023 [cited by applicant]
US 20180260474A1 · Surdeanu et al. · 2018 [cited by applicant]
US 20190005395A1 · Dutkowski · 2019 [cited by applicant]
US 20220121939A1 · Evans et al. · 2022 [cited by applicant]
US 20220188654A1 · Knuff et al. · 2022 [cited by applicant]
US 20230079455A1 · Dasgupta et al. · 2023 [cited by applicant]
US 20230297720A1 · Harvey · 2023 [cited by applicant]
US 20240028997A1 · Cheek, Jr · 2024 [cited by examiner]
CN 113569572A · 2021 [cited by applicant]
WO 2017037326A1 · 2017 [cited by applicant]
Sinha M, Tadepalli P, Ramsey SA (2021) Voting-based integration algorithm improves causal network learning from interventional and observational data: An application to cell signaling network inference. Plos One 16(2): … [cited by applicant]
Xiaojun Chang, Pengzhen Ren, Pengfei Xu, Zhihui Li, Xiaojiang Chen, and Alex Hauptmann (2021) A Comprehensive Survey of Scene Graphs: Generation and Application. See: arXiv:2104.01111. [cited by applicant]
Elif Hilal Korkut and Elif Surer (2022) “Visualization in Virtual Reality: a Systematic Review”. See arXiv:2203.07616. [cited by applicant]
Kortemeyer, G. (2022). Virtual-Reality graph visualization based on Fruchterman-Reingold using Unity and SteamVR. Information Visualization, vol. 21[2], 143-152. [cited by applicant]
N. Capece, U. Erra and J. Grippa (2018) “GraphVR: A Virtual Reality Tool for the Exploration of Graphs with HTC Vive System,” 2018 22nd International Conference Information Visualisation (IV), Fisciano, Italy, 2018, pp.… [cited by applicant]
Y. Gao, et al. (2020) “Hypergraph Learning: Methods and Practices” in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 44, No. 05, pp. 2548-2566. [cited by applicant]
M. Fischer, et al. (2020) “Visual analytics for temporal hypergraph model exploration.” IEEE Transactions on Visualization and Computer Graphics. vol. 27, No. 2, pp. 550-560. [cited by applicant]
Yang, J., Han, S.C. & Poon, J. (2022) A survey on extraction of causal relations from natural language text. Knowl Inf Syst 64, 1161-1186. [cited by applicant]
Klamt S, Haus UU, Theis F (2009) Hypergraphs and Cellular Networks. PLOS Computational Biology 5(5): e1000385. [cited by applicant]
Ilaria Tiddi, Stefan Schlobach (2022) Knowledge graphs as tools for explainable machine learning: A survey, Artificial Intelligence, vol. 302, 103627. [cited by applicant]
Li, L., Jiang, H., Wen, G. et al. (2022) TE-HI-GCN: An Ensemble of Transfer Hierarchical Graph Convolutional Networks for Disorder Diagnosis. Neuroinform 20, 353-375. [cited by applicant]
Hematialam, H. (2021). Knowledge extraction and analysis of medical text with particular emphasis on medical guidelines. Unc Charlotte Electronic Theses And Dissertations. [cited by applicant]
Hossein Hematialam, Wlodek W. Zadrozny. Identifying Condition-action Statements in Medical Guidelines: Three Studies using Machine Learning and Domain Adaptation, May 11, 2021, Preprint (Version 1) available at Research… [cited by applicant]
B. Du et al (2021) “Hypergraph Pre-training with Graph Neural Networks” See arXiv:2105.10862. [cited by applicant]