Systems and methods for generating contextual table embeddings for tabular data

1 . A method for generating contextual embeddings for tabular data, comprising:

receiving, by a table embedding computer program, an input table comprising a plurality of cells;

separating, by the table embedding computer program, the plurality of cells in the input table by data type of each cell, wherein the plurality of cells are separated into cells of a text data type and cells of a numeric data type;

embedding, by the table embedding computer program and for each cell of the input table, the data type for the cell;

embedding, by the table embedding computer program and for each cell having the numeric data type, an embedding vector for a numeric value in the cell using linear projection;

enhancing, by the table embedding computer program, the cells of the input table based on a position by embedding a vector representing the position of each cell in the input table;

generating, by the table embedding computer program, contextual embeddings for the input table using an encoder of a table transformer; and

generating, by the table embedding computer program, a table summary for the contextual embeddings using a decoder for the table transformer.

2 . The method of claim 1 , further comprising:

embedding, by the table embedding computer program and for each cell having the text data type, a semantic representation of text present in the cell.

3 . The method of claim 1 , wherein the vector representing the position of each cell in the input table comprises a Fourier encoding of its position in the input table.

4 . The method of claim 1 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

5 . The method of claim 1 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.

6 . A system, comprising:

a data repository comprising a plurality of tables each comprising a plurality of cells;

an electronic device executing a table embedding computer program that is configured to receive an input table out of the plurality of tables from the data repository, separate the plurality of cells in the input table by data type of each cell, wherein the plurality of cells are separated into cells of a text data type and cells of a numeric data type, embed, for each cell of the input table, the data type for the cell, embed, for each cell having the numeric data type, an embedding vector for a numeric value in the cell using linear projection, enhance the cells of the input table based on a position by embedding a vector representing the position of each cell in the input table, generate contextual embeddings for the input table using an encoder of a table transformer, and generate a table summary for the contextual embeddings using a decoder for the table transformer; and

a downstream system that is configured to receive the contextual embeddings from the table embedding computer program.

7 . The system of claim 6 , wherein the table embedding computer program is further configured to embed, for each cell having the text data type, a semantic representation of text present in the cell.

8 . The system of claim 6 , wherein the vector representing the position of each cell in the input table comprises a Fourier encoding of its position in the input table.

9 . The system of claim 6 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

10 . The system of claim 6 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.

11 . A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:

receiving an input table comprising a plurality of cells;

separating the plurality of cells in the input table by data type of each cell, wherein the plurality of cells are separated into cells of a text data type and cells of a numeric data type;

embedding, for each cell of the input table, the data type for the cell;

embedding, for each cell having the numeric data type, an embedding vector for a numeric value in the cell using linear projection;

enhancing the cells of the input table based on a by embedding a vector representing a position of each cell in the input table;

generating contextual embeddings for the input table using an encoder of a table transformer; and

generating a table summary for the contextual embeddings using a decoder for the table transformer.

12 . The non-transitory computer readable storage medium of claim 11 , further including instructions stored thereon, which when read and executed by the one or more computer processors, cause the one or more computer processors to perform steps comprising:

embedding, for each cell having the text data type, a semantic representation of text present in the cell.

13 . The non-transitory computer readable storage medium of claim 11 , wherein the vector representing the position of each cell in the input table comprises a Fourier encoding of its position in the input table.

14 . The non-transitory computer readable storage medium of claim 11 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

15 . The non-transitory computer readable storage medium of claim 11 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.