Row-column formatting for relational databases
Embodiments of the present disclosure provide methods and devices for formatting a database. The method can include clustering, into each block, a set of sequence of elements from each column of a database; generating a file storing one or more blocks having the clustered set of sequences of elements; providing a first table storing file schema information that associates the elements with the file; and providing a second table for storing block schema information that associates the columns with the one or more blocks.
1. A method for formatting a database, comprising:
clustering, into each block of a set of blocks representing the database, a set of sequence of elements from each column of the database, wherein each block of the set of blocks corresponds to a different set of rows from the database, and the clustered set of sequence of elements is stored in a columnar format in each block of the set of blocks;
generating a file storing one or more blocks having the clustered set of sequences of elements;
providing a first table storing file schema information that associates the elements with the file; and
providing a second table for storing block schema information that associates the columns with the one or more blocks.
2. The method of claim 1 , wherein the file schema information includes at least one of a total number of rows in the file, a total number of blocks, or a total size of the file.
3. The method of claim 1 , wherein the block schema information includes at least one of a starting row of each block in the file, a total number of rows in each block, a total number of deleted rows in each block, a delete bitmap, statistics of a column, or position information of a column.
4. The method of claim 3 , wherein the delete bitmap indicates deleted rows of the database.
5. The method of claim 3 , wherein a block of the one or more blocks is filtered out in data retrieval in response to a determination that the block does not contain elements that satisfy a predicate of a query based on comparison between the statistics of the block and the predicate.
6. The method of claim 3 , wherein the statistics of a column includes at least one of a maximum value of elements in a column, or a minimum value of elements in a column.
7. The method of claim 1 , further comprising:
providing a third table storing changes of the elements, the changes being caused by a data operation.
8. The method of claim 1 , wherein the first table and the second table support Atomicity, Consistency, Isolation, and Durability (ACID).
9. A database system for formatting a database, comprising:
a memory storing a set of instructions; and
a processor configured to execute the set of instructions to cause the database system to:
cluster, into each block of a set of blocks representing the database, a set of sequence of elements from each column of a relational database, wherein each block of the set of blocks corresponds to a different set of rows from the database and the set of sequence of elements of each column has a same number of elements, and the clustered set of sequence of elements is stored in a columnar format in each block of the set of blocks;
generate a file storing one or more blocks having the clustered set of sequences of elements;
provide a first table storing file schema information that associates the elements with the file; and
provide a second table for storing block schema information that associates the columns with the one or more blocks.
10. The database system of claim 9 , wherein the file schema information includes at least one of a total number of rows in the file, a total number of blocks, or a total size of the file.
11. The database system of claim 9 , wherein the block schema information includes at least one of a starting row of each block in the file, a total number of rows in each block, a total number of deleted rows in each block, a delete bitmap, statistics of a column, or position information of a column.
12. The database system of claim 11 , wherein the delete bitmap indicates deleted rows of the database.
13. The database system of claim 11 , wherein a block of the one or more blocks is filtered out in data retrieval in response to a determination that the block does not contain elements that satisfy a predicate of a query based on comparison between the statistics of the block and the predicate.
14. The database system of claim 11 , wherein the statistics of a column include at least one of a maximum value of elements in a column, or a minimum value of elements in a column.
15. The database system of claim 9 , wherein the processor is further configured to cause the database system to:
provide a third table storing changes of the elements, the changes being caused by a data operation.
16. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer to cause the computer to perform a method for formatting a relational database, the method comprising:
clustering, into each block of a set of blocks representing the database, a set of sequence of elements from each column of a relational database, wherein each block of the set of blocks corresponds to a different set of rows from the database and the set of sequence of elements of each column has a same number of elements, and the clustered set of sequence of elements is stored in a columnar format in each block of the set of blocks;
generating a file storing one or more blocks having the clustered set of sequences of elements;
providing a first table storing file schema information that associates the elements with the file; and
providing a second table for storing block schema information that associates the columns with the one or more blocks.
17. The computer readable medium of claim 16 , wherein the file schema information includes at least one of a total number of rows in the file, a total number of blocks, or a total size of the file.
18. The computer readable medium of claim 16 , wherein the block schema information includes at least one of a starting row of each block in the file, a total number of rows in each block, a total number of deleted rows in each block, a delete bitmap, statistics of a column, or position information of a column.
19. The computer readable medium of claim 18 , wherein the delete bitmap indicates deleted rows of the database.
20. The computer readable medium of claim 18 , wherein a block of the one or more blocks is filtered out in data retrieval in response to a determination that the block does not contain elements that satisfy a predicate of a query based on comparison between the statistics of the block and the predicate.
21. The computer readable medium of claim 16 , wherein the set of instructions that is executable by one or more processors of the computer to cause the computer to further perform:
provide a third table storing changes of the elements, the changes being caused by a data operation.