Conversion of tabular format data to machine readable text for QA operations

1. A method comprising:

converting, by a processor of a computing system, a table containing text in tabular form to an image;

labeling, by the processor, each text area of the image with a bounding box;

determining, by the processor, for each bounding box, a position information, a semantic information, and an image information;

reconstructing, by the processor, the image into a graph form having a plurality of nodes, wherein each node represents the bounding box of the text areas of the image;

inputting, by the processor, the position information into a first trained neural network for position embedding, the semantic information into a second trained neural network for text embedding, and the image information into a third trained neural network for image embedding;

inputting, by the processor, at least two nodes into a trained neural network to determine a relative relationship between the at least two nodes;

building, by the processor, a knowledge graph using the relative relationship of the at least two nodes; and

translating, by the processor, the knowledge graph into machine readable natural language.

2. The method of claim 1 , further comprising classifying, by the processor, the plurality of nodes into a node type.

3. The method of claim 2 , wherein the node type includes: a content node, a row name node, and a column name node.

4. The method of claim 1 , wherein the translating is performed by using a natural language generation method based on data of the knowledge graph in the graph form to organize a semantic information of each node of the knowledge graph into a continuous natural language temporary text, which is then translated into the machine readable natural language using a natural language style transfer method.

5. The method of claim 1 ,

wherein the position embedding, the text embedding, and the image embedding are matrix structures with a same shape.

6. The method of claim 1 , further comprising:

in response to the converting, performing, by the processor, an optical character recognition of the image to extract a text contained within the image.

7. The method of claim 1 , further comprising:

parsing, by the processor, the machine readable natural language in response to a query; and

outputting, by the processor, a response to the query.

8. A computing system, comprising:

a processor;

a memory device coupled to the processor; and

a computer readable storage device coupled to the processor, wherein the storage device contains program code executable by the processor via the memory device to implement a method comprising:

converting, by the processor, a table containing text in tabular form to an image;

labeling, by the processor, each text area of the image with a bounding box;

determining, by the processor, for each bounding box, a position information, a semantic information, and an image information;

reconstructing, by the processor, the image into a graph form having a plurality of nodes, wherein each node represents the bounding box of the text areas of the image;

inputting, by the processor, the position information into a first trained neural network for position embedding, the semantic information into a second trained neural network for text embedding, and the image information into a third trained neural network for image embedding;

inputting, by the processor, at least two nodes into a trained neural network to determine a relative relationship between the at least two nodes;

building, by the processor, a knowledge graph using the relative relationship of the at least two nodes; and

translating, by the processor, the knowledge graph into machine readable natural language.

9. The computer system of claim 8 , further comprising classifying, by the processor, the plurality of nodes into a node type.

10. The computer system of claim 9 , wherein the node type includes: a content node, a row name node, and a column name node.

11. The computer system of claim 8 , wherein the translating is performed by using a natural language generation method based on data of the knowledge graph in the graph form to organize a semantic information of each node of the knowledge graph into a continuous natural language temporary text, which is then translated into the machine readable natural language using a natural language style transfer method.

12. The computer system of claim 8 ,

wherein the position embedding, the text embedding, and the image embedding are matrix structures with a same shape.

13. The computer system of claim 8 , further comprising:

in response to the converting, performing, by the processor, an optical character recognition of the image to extract a text contained within the image.

14. The computer system of claim 8 , further comprising:

parsing, by the processor, the machine readable natural language in response to a query; and

outputting, by the processor, a response to the query.

15. A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, the computer readable program code comprising an algorithm that when executed by a computer processor of a computing system implements a method comprising:

converting, by the processor, a table containing text in tabular form to an image;

labeling, by the processor, each text area of the image with a bounding box;

determining, by the processor, for each bounding box, a position information, a semantic information, and an image information;

reconstructing, by the processor, the image into a graph form having a plurality of nodes, wherein each node represents the bounding box of the text areas of the image;

inputting, by the processor, the position information into a first trained neural network for position embedding, the semantic information into a second trained neural network for text embedding, and the image information into a third trained neural network for image embedding;

inputting, by the processor, at least two nodes into a trained neural network to determine a relative relationship between the at least two nodes;

building, by the processor, a knowledge graph using the relative relationship of the at least two nodes; and

translating, by the processor, the knowledge graph into machine readable natural language.

16. The computer program product of claim 15 , further comprising classifying, by the processor, the plurality of nodes into a node type.

17. The computer program product of claim 16 , wherein the node type includes: a content node, a row name node, and a column name node.

18. The computer program product of claim 15 , wherein the translating is performed by using a natural language generation method based on data of the knowledge graph in the graph form to organize a semantic information of each node of the knowledge graph into a continuous natural language temporary text, which is then translated into the machine readable natural language using a natural language style transfer method.

19. The computer program product of claim 15 , further comprising:

wherein the position embedding, the text embedding, and the image embedding are matrix structures with a same shape.

20. The computer program product of claim 15 , further comprising:

in response to the converting, performing, by the processor, an optical character recognition of the image to extract a text contained within the image;

parsing, by the processor, the machine readable natural language in response to a query; and

outputting, by the processor, a response to the query.