IP Library Granted Patent US 11,645,733
Granted Patent B2
US 11,645,733 · App. 16/902,568 · Granted May 9, 2023

System and method for providing artificial intelligence architectures to people with disabilities

Inventor: Madhusudhanan Krishnamoorthy (Tamilnadu, IN)
Assignee: Bank of America Corporation
G06T3/0012G06F40/58G06N3/04G06N3/08G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,645,733
App. No.
16/902,568
Granted
May 9, 2023
Kind
B2
Abstract

A system is configured for converting an unstandardized architecture diagram into a braille language diagram is disclosed. The system receives the unstandardized architecture diagram which includes a plurality of architecture components. The system receives a standardized model that includes features to depict the architecture components of the unstandardized architecture diagram in a standard format. The system determines the architecture components, their connections, and their sequences from the unstandardized architecture diagram. The system determines the features to depict the architecture components of the unstandardized architecture diagram in the standard format. The system applies the identified features on the identified architecture components in the unstandardized architecture diagram. The system determines a standardized graphical representation of the unstandardized architecture diagram. The system converts the standardized graphical representation of the unstandardized architecture diagram into a braille language diagram.

Claims (107)

1. A system for converting an unstandardized architecture diagram image into a braille language diagram, comprising:

a memory operable to store the unstandardized architecture diagram image;

a processor operably coupled to the memory and configured to:

receive the unstandardized architecture diagram image, comprising:

a first layer that comprises a first plurality of components, wherein the first plurality of components includes inputs of an unstandardized architecture diagram in the unstandardized architecture diagram image; and

a second layer that comprises a second plurality of components, wherein:

the second plurality of components include outputs of the unstandardized architecture diagram;

a plurality of functions operates on the first plurality of components; and

the first layer is connected to the second layer;

receive a standardized model image that comprises features to depict the first plurality of components, the second plurality of components, and the plurality of functions in a standardized format;

determine the first layer, the second layer, the plurality of functions, connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image;

generate a first vector representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer from the unstandardized architecture diagram image;

determine features to represent the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image;

generate a second vector representing the features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized format from the standardized model image;

generate a third vector by applying the features to depict the standardized model image from the second vector on the first vector;

determine a standardized graphical representation of the unstandardized architecture diagram image by changing a dimension of the third vector;

based on the standardized graphical representation of the unstandardized architecture diagram image, convert each of the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer into a corresponding braille symbol.

2. The system of claim 1 , wherein determining the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image by performing a convolution operation on the unstandardized architecture diagram image;

generating a first matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

determining the sequence of the first layer and the second layer of the unstandardized architecture diagram image by performing a long short term memory (LSTM) operation on the first matrix;

generating a second matrix representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer in the unstandardized architecture diagram image; and

performing a flattening operation on the second matrix to generate the first vector.

3. The system of claim 1 , wherein determining features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image by performing a convolution operation on the standardized model image;

generating a third matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

performing a flattening operation on the third matrix to generate the second vector.

4. The system of claim 1 , wherein determining the standardized graphical representation of the unstandardized architecture diagram image comprises:

performing a convolution with an upsampling operation on the third vector, wherein the upsampling operation unifies a size of the standardized graphical representation of the unstandardized architecture diagram image.

5. The system of claim 1 , wherein the standardized graphical representation comprises:

a first standardized layer comprising the first plurality of components depicted in a standardized format; and

a second standardized layer comprising the second plurality of components depicted in a standardized format, wherein the plurality of functions is operated on the first plurality of components, the first standardized layer is connected to the second standardized layer.

6. The system of claim 1 , wherein converting the plurality of functions into a corresponding braille symbol comprises:

converting a first function into a first braille symbol using a mapping of functions to braille symbols stored in the memory; and

converting a second function into a second braille symbol different from the first braille symbol using the mapping of functions to braille symbols.

7. The system of claim 6 , wherein converting the first layer and the second layer into a corresponding braille symbol comprises:

converting the first layer into a third braille symbol using a mapping of layers to braille symbols stored in the memory; and

converting the second layer into a fourth braille symbol using a mapping of layers to braille symbols.

8. A method for converting an unstandardized architecture diagram image into a braille language diagram, comprising:

receiving the unstandardized architecture diagram image, comprising:

a first layer that comprises a first plurality of components, wherein the first plurality of components includes inputs of an unstandardized architecture diagram in the unstandardized architecture diagram image; and

a second layer that comprises a second plurality of components, wherein:

the second plurality of components include outputs of the unstandardized architecture diagram;

a plurality of functions operates on the first plurality of components; and

the first layer is connected to the second layer;

receiving a standardized model image that comprises features to depict the first plurality of components, the second plurality of components, and the plurality of functions in a standardized format;

determining the first layer, the second layer, the plurality of functions, connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image;

generating a first vector representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer from the unstandardized architecture diagram image;

determining features to represent the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image;

generating a second vector representing the features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized format from the standardized model image;

generating a third vector by applying the features to depict the standardized model image from the second vector on the first vector;

determining a standardized graphical representation of the unstandardized architecture diagram image by changing a dimension of the third vector;

based on the standardized graphical representation of the unstandardized architecture diagram image, converting each of the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer into a corresponding braille symbol.

9. The method of claim 8 , wherein determining the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image by performing a convolution operation on the unstandardized architecture diagram image;

generating a first matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

determining the sequence of the first layer and the second layer of the unstandardized architecture diagram image by performing a long short term memory (LSTM) operation on the first matrix;

generating a second matrix representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer in the unstandardized architecture diagram image; and

performing a flattening operation on the second matrix to generate the first vector.

10. The method of claim 8 , wherein determining features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image by performing a convolution operation on the standardized model image;

generating a third matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

performing a flattening operation on the third matrix to generate the second vector.

11. The method of claim 8 , wherein determining the standardized graphical representation of the unstandardized architecture diagram image comprises:

performing a convolution with an upsampling operation on the third vector, wherein the upsampling operation unifies a size of the standardized graphical representation of the unstandardized architecture diagram image.

12. The method of claim 8 , wherein the standardized graphical representation comprises:

a first standardized layer comprising the first plurality of components depicted in a standardized format; and

a second standardized layer comprising the second plurality of components depicted in a standardized format, wherein the plurality of functions is operated on the first plurality of components, the first standardized layer is connected to the second standardized layer.

13. The method of claim 8 , wherein converting the plurality of functions into a corresponding braille symbol comprises:

converting a first function into a first braille symbol using a mapping of functions to braille symbols stored in a memory; and

converting a second function into a second braille symbol different from the first braille symbol using the mapping of functions to braille symbols.

14. The method of claim 8 , wherein converting the first layer and the second layer into a corresponding braille symbol comprises:

converting the first layer into a third braille symbol using a mapping of layers to braille symbols stored in a memory; and

converting the second layer into a fourth braille symbol using a mapping of layers to braille symbols.

15. A computer program comprising executable instructions stored in a non-transitory computer-readable medium that when executed by a processor causes the processor to:

receive an unstandardized architecture diagram image, comprising:

a first layer that comprises a first plurality of components, wherein the first plurality of components includes inputs of an unstandardized architecture diagram in the unstandardized architecture diagram image; and

a second layer that comprises a second plurality of components, wherein:

the second plurality of components include outputs of the unstandardized architecture diagram;

a plurality of functions operates on the first plurality of components; and

the first layer is connected to the second layer;

receive a standardized model image that comprises features to depict the first plurality of components, the second plurality of components, and the plurality of functions in a standardized format;

determine the first layer, the second layer, the plurality of functions, connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image;

generate a first vector representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer from the unstandardized architecture diagram image;

determine features to represent the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image;

generate a second vector representing the features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized format from the standardized model image;

generate a third vector by applying the features to depict the standardized model image from the second vector on the first vector;

determine a standardized graphical representation of the unstandardized architecture diagram image by changing a dimension of the third vector;

based on the standardized graphical representation of the unstandardized architecture diagram image, convert each of the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer into a corresponding braille symbol.

16. The computer program of claim 15 , wherein determining the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and a sequence of the first layer and the second layer in the unstandardized architecture diagram image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image by performing a convolution operation on the unstandardized architecture diagram image;

generating a first matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

determining the sequence of the first layer and the second layer of the unstandardized architecture diagram image by performing a long short term memory (LSTM) operation on the first matrix;

generating a second matrix representing the first layer, the second layer, the plurality of functions, the connections between the first layer and the second layer, and the sequence of the first layer and the second layer in the unstandardized architecture diagram image; and

performing a flattening operation on the second matrix to generate the first vector.

17. The computer program of claim 15 , wherein determining features to depict the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image comprises:

determining the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the standardized model image by performing a convolution operation on the standardized model image;

generating a third matrix representing the first layer, the second layer, the plurality of functions, and the connections between the first layer and the second layer in the unstandardized architecture diagram image;

performing a flattening operation on the third matrix to generate the second vector.

18. The computer program of claim 15 , wherein determining the standardized graphical representation of the unstandardized architecture diagram image comprises:

performing a convolution with an upsampling operation on the third vector, wherein the upsampling operation unifies a size of the standardized graphical representation of the unstandardized architecture diagram image.

19. The computer program of claim 15 , wherein the standardized graphical representation comprises:

a first standardized layer comprising the first plurality of components depicted in a standardized format; and

a second standardized layer comprising the second plurality of components depicted in a standardized format, wherein the plurality of functions is operated on the first plurality of components, the first standardized layer is connected to the second standardized layer.

20. The computer program of claim 15 , wherein converting the plurality of functions into a corresponding braille symbol comprises:

converting a first function into a first braille symbol using a mapping of functions to braille symbols stored in a memory; and

converting a second function into a second braille symbol different from the first braille symbol using the mapping of functions to braille symbols.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2020
From: KRISHNAMOORTHY, MADHUSUDHANAN
To: BANK OF AMERICA CORPORATION
Reel/Frame 052950/0091 →
Continuity (1)
Related Publication 20210390656A1 · Dec 16, 2021