A multi-level storage architecture and method of performing a partial merge are disclosed. A main store is partitioned into a passive main part and an active main part, the active main part being empty at a start of the partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge. A values index corresponding to a sorted dictionary of the passive main part is set to a cardinality of n. The data records of a second level storage structure is merged into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part.
1. In a unified table architecture of an in-memory computing system having a multi-level storage architecture, the multi-level storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage, a method of performing a partial merge, the method comprising:
partitioning the main store into a passive main part and an active main part, the active main part being empty at a start of the partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge;
setting a values index corresponding to a sorted dictionary of the passive main part to a cardinality of n; and
merging the data records of the second level storage structure into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part, the merging of the data records of the second level storage structure into the active main part being the partial merge.
2. The method in accordance with claim 1 , further comprising resolving a point access operation within the sorted dictionary of the passive main part of the main store.
3. The method in accordance with claim 1 , further comprising resolving a range access operation within both the sorted dictionary of the passive main part and the dictionary of the active main part.
4. A computer implemented method comprising:
providing a multi-level storage architecture, the multi-level storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage;
partitioning the main store into a passive main part and an active main part, the active main part being empty at a start of a partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge;
setting a values index corresponding to a sorted dictionary of the passive main part to a cardinality of n; and
merging the data records of the second level storage structure into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part, the merging of the data records of the second level storage structure into the active main part being the partial merge.
5. The computer implemented method in accordance with claim 4 , further comprising resolving a point access operation within the sorted dictionary of the passive main part of the main store.
6. The computer implemented method in accordance with claim 4 , further comprising resolving a range access operation within both the sorted dictionary of the passive main part and the dictionary of the active main part.
7. The computer implemented method in accordance with claim 4 , wherein the partitioning, the setting, and the merging are performed by one more processors.
8. A system comprising:
at least one programmable processor;
a multi-level storage architecture, the multi-level storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage; and
a machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising:
partitioning the main store into a passive main part and an active main part, the active main part being empty at a start of a partial merge, the passive main part storing encoded data records of the main store that are not subject to the partial merge;
setting a values index corresponding to a sorted dictionary of the passive main part to a cardinality of n; and
merging the data records of the second level storage structure into the active main part, the active main part having a dictionary that starts with a value of n+1, such that the merging into the active main part continues an encoding scheme according to the values index of the passive main part, the merging of the data records of the second level storage structure into the active main part being the partial merge.
9. The system in accordance with claim 8 , further comprising resolving a point access operation within the sorted dictionary of the passive main part of the main store.
10. The system in accordance with claim 8 , further comprising resolving a range access operation within both the sorted dictionary of the passive main part and the dictionary of the active main part.