Data storage method and data storage engine
A data storage method includes the following steps: storing data to be stored into a mutable table, the mutable table being located in a RAM. If a storage content of the mutable table is greater than or equal to a preset storage content threshold, modifying the mutable table into an immutable table and stopping receiving storage of the data to be stored; if a storage content of a lowest level in a disk is greater than or equal to a storage content threshold of a present level, and next level higher in the disk is stored with columnar format, storing data of the lowest level to the next level in the columnar format; and if the lowest level is stored with row format, storing data in the immutable table to the lowest level in the row format.
1. A data storage method, comprising:
storing data to be stored into a mutable table, the mutable table being located in a RAM;
if a storage content of the mutable table is greater than or equal to a preset storage content threshold, modifying the mutable table into an immutable table and stopping receiving storage of the data to be stored;
when a storage content of a lowest level in a disk is greater than or equal to a storage content threshold of a present level, and a next level higher in the disk is stored with a columnar format, storing data of the lowest level to the next level in the columnar format; and
when the lowest level is stored with a row format, storing data in the immutable table to the lowest level in the row format;
wherein the step of storing the data of the lowest level to the next level in the columnar format comprises:
extracting a value of each attribute from each data of the lowest level; and
for the each attribute, storing the value of the each attribute to an attribute storage file corresponding to an attribute in the next level.
2. The method according to claim 1 , further comprising:
when the storage content of the lowest level in the disk is greater than or equal to the storage content threshold of the present level, and the next level higher in the disk is stored with the row format, storing the data of the lowest level to the next level.
3. The method according to claim 2 , further comprising:
when a storage content of one level stored with the row format in the disk is greater than or equal to the storage content threshold of the present level, and a level next to the one level is stored with the columnar format, storing data of the one level stored with the row format to the next level in the columnar format;
when the storage content of the one level stored with the row format in the disk is greater than or equal to the storage content threshold of the present level, and the level next to the one level is stored with the row format, storing the data of the one level stored with the row format to the next level; and
when the storage content of the one level stored with the columnar format in the disk is greater than or equal to the storage content threshold of the present level, and a level next to the level is stored with the columnar format, storing the data of the one level stored with the columnar format to the next level.
4. The method according to claim 3 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
5. The method according to claim 2 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
6. The method according to claim 1 , further comprising:
when the storage content of the lowest level is smaller than the storage content threshold of the present level, and the lowest level is stored with the row format, storing the data in the immutable table to the lowest level in the row format.
7. The method according to claim 6 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
8. The method according to claim 1 , wherein after the step of modifying the mutable table into the immutable table, the method further comprises:
creating a new mutable table, and storing the data to be stored into the new mutable table.
9. The method according to claim 8 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
10. The method according to claim 1 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
11. The method according to claim 1 , wherein the data is stored in each level in SST files, and the method further comprises:
dynamically modifying a storage format of the each SST file in the each level.
12. An electronic device, comprising:
a processor,
a memory, and
a computer program stored on the memory and operable on the processor,
wherein the processor executes the program to perform operations comprising:
storing data to be stored into a mutable table, the mutable table being located in a RAM;
if a storage content of the mutable table is greater than or equal to a preset storage content threshold, modifying the mutable table into an immutable table and stopping receiving storage of the data to be stored;
when a storage content of a lowest level in a disk is greater than or equal to a storage content threshold of a present level, and a next level higher in the disk is stored with a columnar format, storing data of the lowest level to the next level in the columnar format; and
when the lowest level is stored with a row format, storing data in the immutable table to the lowest level in the row format;
wherein the step of storing the data of the lowest level to the next level in the columnar format comprises:
extracting a value of each attribute from each data of the lowest level; and
for the each attribute, storing the value of the each attribute to an attribute storage file corresponding to an attribute in the next level.
13. The electronic device according to claim 12 , further comprising:
when the storage content of the lowest level in the disk is greater than or equal to the storage content threshold of the present level, and the next level higher in the disk is stored with the row format, storing the data of the lowest level to the next level.
14. The electronic device according to claim 13 , further comprising:
when a storage content of one level stored with the row format in the disk is greater than or equal to the storage content threshold of the present level, and a level next to the one level is stored with the columnar format, storing data of the one level stored with the row format to the next level in the columnar format;
when the storage content of the one level stored with the row format in the disk is greater than or equal to the storage content threshold of the present level, and the level next to the one level is stored with the row format, storing the data of the one level stored with the row format to the next level; and
when the storage content of the one level stored with the columnar format in the disk is greater than or equal to the storage content threshold of the present level, and a level next to the level is stored with the columnar format, storing the data of the one level stored with the columnar format to the next level.
15. The electronic device according to claim 12 , further comprising:
when the storage content of the lowest level is smaller than the storage content threshold of the present level, and the lowest level is stored with the row format, storing the data in the immutable table to the lowest level in the row format.
16. The electronic device according to claim 12 , wherein after the step of modifying the mutable table into the immutable table, the method further comprises:
creating a new mutable table, and storing the data to be stored into the new mutable table.