IP Library Granted Patent US 11,080,905
Granted Patent B1
US 11,080,905 · App. 16/854,715 · Granted Aug 3, 2021

System and method of image analysis of user-illustrated process flow to automate generation of executable code instructions

Inventors: Dennis Matthew McCarty (Pennsauken, NJ); Tiffany T. Powley (Plymouth Meeting, PA); Thang Ngoc Mai (Downington, PA); Christopher R. Taylor (Marysville, OH)
Assignee: Dell Products, LP
G06T11/206G06T7/543G06T11/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,080,905
App. No.
16/854,715
Granted
Aug 3, 2021
Kind
B1
Abstract

An information handling system operating an image analysis integration flow creation system may comprise a network interface device receiving a captured image of an illustrated integration process flow chart connecting process step illustrations in a user-specified pattern, and a processor determining a process flow plot connecting visual element placeholders corresponding to the process step illustrations according to the user-specified pattern. The processor may identify an image shape within each process step illustration and image shape-identifying parameters for that image shape, apply a neural network to determine a type of integration process visual element represented by each process step illustration, based on the image shape-identifying parameters, and generate an integration process flow model displayed in a GUI by inserting the type of integration process visual element represented by each process step illustration into the visual element placeholder corresponding to that process step illustration.

Claims (44)

1. An information handling system operating an image analysis integration flow creation system comprising:

a network interface device receiving a captured image of an illustrated integration process flow chart connecting a plurality of process step illustrations to one another in a user-specified pattern;

a processor determining a process flow plot connecting each of an identified plurality of visual element placeholders, each corresponding to one of the plurality of process step illustrations, according to the user-specified pattern;

the processor analyzing the captured image to identify, for each of the plurality of process step illustrations, an image shape within the process step illustration, and a plurality of image shape-identifying parameters for the image shape and applying a neural network to determine a type of integration process visual element represented by the process step illustration, based on the plurality of image shape-identifying parameters; and

the processor generating an integration process flow model displayed in a graphical user interface (GUI) by inserting, for each of the plurality of process step illustrations, the type of integration process visual element represented by the process step illustration into the visual element placeholder of the process flow plot corresponding to the process step illustration.

2. The information handling system of claim 1 further comprising:

the neural network determining the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a position of each of the visual element placeholders associated with each of the plurality of process step illustrations within the process flow plot.

3. The information handling system of claim 1 further comprising:

the neural network determining the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a determined likelihood that a value of one of the plurality of image-shape identifying parameters differentiates one of the plurality of process step illustrations from a remainder of the plurality of process step illustrations.

4. The information handling system of claim 3 , wherein the likelihood that the value of one of the plurality of image shape-identifying parameters differentiates one of the plurality of process step illustrations from the remainder of the plurality of process step illustrations is determined based on crowd-sourced image shape-identifying parameter values.

5. The information handling system of claim 1 , wherein the illustrated integration process flow chart is at least partially a hand-drawn or manually-formed illustration.

6. The information handling system of claim 1 , wherein the illustrated integration process flow chart is a computer-assisted sketch drawn by a user.

7. The information handling system of claim 1 further comprising:

the processor associating each integration process visual element displayed within the integration process flow model with one of a plurality of custom connector code sets and generating a runtime engine; and

the network interface device transmitting the plurality of custom connector code sets and runtime engine to an enterprise system for remote execution.

8. A method of image analysis integration flow creation comprising:

receiving, via a network interface device, a captured image of an illustrated integration process flow chart connecting a plurality of process step illustrations to one another in a user-specified pattern;

determining, via a processor, a process flow plot connecting each of an identified plurality of visual element placeholders, each corresponding to one of the plurality of process step illustrations, according to the user-specified pattern;

analyzing, via the processor, the captured image to identify, for each of the plurality of process step illustrations, an image shape within the process step illustration, and a plurality of image shape-identifying parameters for the image shape and applying a neural network to determine a type of integration process visual element represented by the process step illustration, based on the plurality of image shape-identifying parameters;

generating an integration process flow model displayed in a graphical user interface (GUI) GUI by inserting, via a processor, for each of the plurality of process step illustrations, the type of integration process visual element represented by the process step illustration into the visual element placeholder of the process flow plot corresponding to the process step illustration.

9. The method of claim 8 further comprising:

determining, via the neural network, the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a position of each of the visual element placeholders associated with each of the plurality of process step illustrations within the process flow plot.

10. The method of claim 8 further comprising:

determining, via the neural network, the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a determined likelihood that a value of one of the plurality of image-shape identifying parameters differentiates one of the plurality of process step illustrations from a remainder of the plurality of process step illustrations.

11. The method of claim 10 , wherein the likelihood that the value of one of the plurality of image shape-identifying parameters differentiates one of the plurality of process step illustrations from the remainder of the plurality of process step illustrations is determined based on crowd-sourced image shape-identifying parameter values.

12. The method of claim 8 , wherein the illustrated integration process flow chart is at least partially a hand-drawn or manually-formed illustration.

13. The method of claim 8 , wherein the illustrated integration process flow chart is a computer-assisted sketch drawn by a user.

14. The method of claim 8 further comprising:

associating, via the processor, each integration process visual element displayed within the integration process flow model with one of a plurality of custom connector code sets and generating a runtime engine; and

transmitting, via the network interface device, the plurality of custom connector code sets and runtime engine to an enterprise system for remote execution.

15. An information handling system operating an image analysis integration flow creation system comprising:

a network interface device receiving a captured image of an illustrated integration process flow chart connecting a plurality of process step illustrations to one another in a user-specified pattern;

a processor determining a process flow plot connecting each of an identified plurality of visual element placeholders, each corresponding to one of the plurality of process step illustrations, according to the user-specified pattern;

the processor analyzing the captured image to identify, for each of the plurality of process step illustrations, an image shape within the process step illustration, and a plurality of image shape-identifying parameters for the image shape and applying a neural network to determine a type of integration process visual element represented by the process step illustration, based on the plurality of image shape-identifying parameters;

the processor generating an integration process flow model displayed in a graphical user interface (GUI) GUI by inserting, for each of the plurality of process step illustrations, the type of integration process visual element represented by the process step illustration into the visual element placeholder of the process flow plot corresponding to the process step illustration;

the processor associating each integration process visual element displayed within the integration process flow model with one of a plurality of custom connector code sets and generating a runtime engine; and

the network interface device transmitting the plurality of custom connector code sets and runtime engine to an enterprise system for remote execution.

16. The information handling system of claim 15 further comprising:

the neural network determining the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a position of each of the visual element placeholders associated with each of the plurality of process step illustrations within the process flow plot.

17. The information handling system of claim 15 further comprising:

the neural network determining the type of integration process visual element illustrated by each of the plurality of process step illustrations, based on a determined likelihood that a value of one of the plurality of image-shape identifying parameters differentiates one of the plurality of process step illustrations from a remainder of the plurality of process step illustrations.

18. The information handling system of claim 17 , wherein the likelihood that the value of one of the plurality of image shape-identifying parameters differentiates one of the plurality of process step illustrations from the remainder of the plurality of process step illustrations is determined based on crowd-sourced image shape-identifying parameter values.

19. The information handling system of claim 15 , wherein the illustrated integration process flow chart is at least partially a hand-drawn or manually-formed illustration.

20. The information handling system of claim 15 , wherein the illustrated integration process flow chart is a computer-assisted sketch drawn by a user.

Assignments (4)
SECURITY INTEREST Recorded Nov 12, 2024
From: SIXTH STREET SPECIALTY LENDING, INC.
To: BLUE OWL CAPITAL CORPORATION
Reel/Frame 069342/0406 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2023
From: DELL PRODUCTS, LP
To: BOOMI, LP
Reel/Frame 063833/0555 →
SECURITY INTEREST Recorded Oct 1, 2021
From: BOOMI, LLC
To: SIXTH STREET SPECIALTY LENDING, INC., AS COLLATERAL AGENT
Reel/Frame 057679/0908 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 21, 2020
From: MCCARTY, DENNIS MATTHEW; POWLEY, TIFFANY T.; MAI, THANG NGOC; TAYLOR, CHRISTOPHER R.
To: DELL PRODUCTS, LP
Reel/Frame 052457/0136 →
Cited By (1)
US 12,619,623