IP Library Granted Patent US 11,159,559
Granted Patent B2
US 11,159,559 · App. 16/950,509 · Granted Oct 26, 2021

Systems and methods for importing diagrams for automated threat modeling

Inventors: Anuraag Agarwwal (Jersey City, NJ); Nikunj Nagalia (Metuchen, NJ)
Assignee: THREATMODELER SOFTWARE INC.
H04L63/1433G06F21/577H04L41/145H04L63/1416H04L63/1425H04L63/1441
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,159,559
App. No.
16/950,509
Granted
Oct 26, 2021
Kind
B2
Abstract

Automated diagram import methods include providing one or more servers and one or more data stores communicatively coupled with the server(s). The data store(s) may include a plurality of computing environment assets and a plurality of connections between the assets. The method may include receiving a digital image of the hand drawn diagram and identifying a plurality of shapes and one or more links in the received digital image. The method further includes, for each component, identifying a text label for the component and classifying the component as an asset. The method further includes, for each link, determining a text label for the link and identifying two components connected by the link. The method may also include generating a diagram and displaying the diagram on a user interface. Automated hand drawn diagram import systems include systems configured to carry out automated importing of the hand drawn diagram.

Claims (76)

1. A method of automatically importing a hand drawn diagram for use in generating a threat model for a computing environment, comprising:

providing one or more servers;

providing one or more data stores communicatively coupled with the one or more servers, the one or more data stores storing information on:

a plurality of computing environment assets; and

a plurality of connections between the plurality of computing environment assets, wherein each stored connection of the plurality of connections connects two assets of the plurality of computing environment assets;

receiving a digital image of the hand drawn diagram;

identifying a plurality of shapes and one or more links in the received digital image, the plurality of shapes defining a plurality of components of the computing environment, the one or more links each defining a connection between two components of the plurality of components;

for each component of the plurality of components:

identifying a text label associated with the each component in the received digital image; and

classifying the each component as an asset of the plurality of computing environment assets based at least on the information in the one or more data stores and the identified text label;

for each link of the one or more links:

determining a text label associated with the each link; and

identifying two components in the received digital image connected by the each link;

generating, based at least on the classified plurality of components and the one or more links identifying connected components in the received digital image, a diagram including a text label for each component of the plurality of components and for each link of the one or more links; and

displaying the diagram on one or more user interfaces of one or more computing devices, using visual representations of one or more of the stored plurality of computing environment assets corresponding to the classified plurality of components.

2. The method of claim 1 , wherein each connection of the stored plurality of connections defines a communication protocol that connects the two assets of the plurality of computing environment assets.

3. The method of claim 1 , further comprising, in a case where the text label associated with the each component does not match stored information on the plurality of computing environment assets:

identifying at least one other component of the plurality of components connected to the each component by a link of the one or more links; and

changing the text label associated with the each component based at least on the stored information on the at least one other component and the stored information on the plurality of connections.

4. The method of claim 1 , wherein the step of identifying the two components in the received digital image connected by the each link further comprises, in a case where the each link is not connected to a source component or a destination component, identifying a closest component of the plurality of components that is closer to an unconnected end of the each link than other components of the plurality of components.

5. The method of claim 4 , further comprising verifying that the identified closest component can be validly connected to another component of the plurality of components by the each link based at least on the stored information on the plurality of connections.

6. The method of claim 5 , further comprising, in a case where the identified closest component cannot be validly connected to the another component by the each link:

identifying a next closest component of the plurality of components that is closer to the unconnected end of the each link than other components of the plurality of components not including the identified closest component; and

verifying that the identified next closest component can be validly connected to another component of the plurality of components by the each link based at least on the stored information on the plurality of connections.

7. The method of claim 1 , further comprising permitting a user to change the text label associated with at least one component of the plurality of components or at least one link of the one or more links.

8. The method of claim 1 , further comprising:

identifying one or more additional stored computing environment assets associated with the displayed plurality of components; and

updating the diagram displayed on the one or more user interfaces by adding undrawn components corresponding to the identified additional stored computing environment assets and adding one or more links connecting each undrawn component to at least one identified component or one other undrawn component.

9. The method of claim 1 , further comprising:

generating the threat model based at least upon the generated diagram; and

generating a threat report displaying one or more threats associated with one or more components of the plurality of components based at least on the generated threat model.

10. A hand drawn diagram import system, comprising:

one or more data stores configured to store information on a plurality of computing environment assets and a plurality of connections between the plurality of computing environment assets, wherein each stored connection of the plurality of connections connects two assets of the plurality of computing environment assets;

one or more memories configured to store instructions; and

one or more computing devices communicatively connected to the one or more data stores and the one or more memories and configured to execute the stored instructions to:

receive a digital image of the hand drawn diagram;

identify a plurality of shapes and one or more links in the received digital image, the plurality of shapes defining a plurality of components of the computing environment, the one or more links each defining a connection between two components of the plurality of components;

for each component of the plurality of components:

identify a text label associated with the each component in the received digital image; and

classify the each component as an asset of the plurality of computing environment assets based at least on the information in the one or more data stores and the identified text label;

for each link of the one or more links:

determine a text label associated with the each link; and

identify two components in the received digital image connected by the each link;

generate, based at least on the classified plurality of components and the one or more links identifying connected components in the received digital image, a diagram including a text label for each component of the plurality of components and for each link of the one or more links; and

display the diagram on one or more user interfaces of one or more computing devices, using visual representations of one or more of the stored plurality of computing environment assets corresponding to the classified plurality of components.

11. The system of claim 10 , wherein each connection of the stored plurality of connections defines a communication protocol that connects the two assets of the plurality of computing environment assets.

12. The system of claim 10 , wherein the one or more computing devices are further configured to execute the stored instructions to, in a case where the text label associated with the each component does not match stored information on the plurality of computing environment assets:

identify at least one other component of the plurality of components connected to the each component by a link of the one or more links; and

change the text label associated with the each component based at least on the stored information on the at least one other component and the stored information on the plurality of connections.

13. The system of claim 10 , wherein the one or more computing devices are further configured to execute the stored instructions to, in a case where the each link is not connected to a source component or a destination component, identify a closest component of the plurality of components that is closer to an unconnected end of the each link than other components of the plurality of components.

14. The system of claim 13 , wherein the one or more computing devices are further configured to execute the stored instructions to verify that the identified closest component can be validly connected to another component of the plurality of components by the each link based at least on the stored information on the plurality of connections.

15. The system of claim 14 , wherein the one or more computing devices are further configured to execute the stored instructions to, in a case where the identified closest component cannot be validly connected to the another component by the each link:

identify a next closest component of the plurality of components that is closer to the unconnected end of the each link than other components of the plurality of components not including the identified closest component; and

verify that the identified next closest component can be validly connected to another component of the plurality of components by the each link based at least on the stored information on the plurality of connections.

16. The system of claim 10 , wherein the one or more computing devices are further configured to execute the stored instructions to permit a user to change the text label associated with at least one component of the plurality of components or at least one link of the one or more links.

17. The system of claim 10 , wherein the one or more computing devices are further configured to execute the stored instructions to:

identify one or more additional stored computing environment assets associated with the displayed plurality of components; and

update the diagram displayed on the one or more user interfaces by adding undrawn components corresponding to the identified additional stored computing environment assets and adding one or more links connecting each undrawn component to at least one identified component or one other undrawn component.

18. The system of claim 10 , wherein the one or more computing devices are further configured to execute the stored instructions to:

generate a threat model based at least upon the generated diagram; and

generate a threat report displaying one or more threats associated with one or more components of the plurality of components based at least on the generated threat model.

19. A non-transitory computer readable storage medium configured to a store a program that executes a method of automatically importing a hand drawn diagram for use in generating a threat model for a computing environment, the method comprising:

providing one or more servers;

providing one or more data stores communicatively coupled with the one or more servers, the one or more data stores storing information on:

a plurality of computing environment assets; and

a plurality of connections between the plurality of computing environment assets, wherein each stored connection of the plurality of connections connects two assets of the plurality of computing environment assets;

receiving a digital image of the hand drawn diagram;

identifying a plurality of shapes and one or more links in the received digital image, the plurality of shapes defining a plurality of components of the computing environment, the one or more links each defining a connection between two components of the plurality of components;

for each component of the plurality of components:

identifying a text label associated with the each component in the received digital image; and

classifying the each component as an asset of the plurality of computing environment assets based at least on the information in the one or more data stores and the identified text label;

for each link of the one or more links:

determining a text label associated with the each link; and

identifying two components in the received digital image connected by the each link;

generating, based at least on the classified plurality of components and the one or more links identifying connected components in the received digital image, a diagram including a text label for each component of the plurality of components and for each link of the one or more links; and

displaying the diagram on one or more user interfaces of one or more computing devices, using visual representations of one or more of the stored plurality of computing environment assets corresponding to the classified plurality of components.

Assignments (2)
SECURITY INTEREST Recorded Dec 30, 2025
From: THREATMODELER SOFTWARE, INC.
To: WESTERN ALLIANCE BANK
Reel/Frame 073340/0598 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2020
From: AGARWWAL, ANURAAG; NAGALIA, NIKUNJ
To: THREATMODELER SOFTWARE INC.
Reel/Frame 054394/0140 →
Continuity (10)
Continuation In Part 16947798 · Aug 17, 2020
Continuation In Part 16664679 · Oct 25, 2019
Continuation In Part 16228738 · Dec 20, 2018
Continuation In Part 15922856 · Mar 15, 2018
Continuation In Part 15888021 · Feb 3, 2018
Provisional Application 62507691 · May 17, 2017
Provisional Application 62520954 · Jun 16, 2017
Provisional Application 62527671 · Jun 30, 2017
Provisional Application 62530295 · Jul 10, 2017
Related Publication 20210294901A1 · Sep 23, 2021
Cited By (1)
US 12,306,959