Logless atomic data movement
View Patent ↗A system and method of logless atomic data movement. An internal transaction is started within a multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure. Committed data is read from a first level storage structure of the multi-level storage architecture as specified by the internal transaction. The committed data from the first level storage structure is inserted into a second level storage structure in a bulk insertion process, and the committed data is marked as being deleted from the first level storage. The internal transaction is then committed to the multi-level storage architecture when the committed data has been inserted into the second level storage structure.
1. In a unified table architecture of an in-memory computing system having a multi-level storage architecture, the storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage, a method of log-less atomic data movement, the method comprising:
starting an internal transaction within the multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure;
reading committed data from the first level storage structure as specified by the internal transaction;
inserting the committed data from the first level storage structure into the second level storage structure in a bulk insertion process;
marking the committed data as being deleted from the first level storage; and
committing the internal transaction to the multi-level storage when the committed data has been inserted into the second level storage structure.
2. The method in accordance with claim 1 , wherein the bulk insertion process includes:
generating a page identifier for the committed data; and
creating an undo log for the page identifier.
3. The method in accordance with claim 1 , further comprising marking the committed data as being deleted from the first level storage further includes adding a page delete version of the committed data to the first level storage.
4. A computer implemented method comprising:
providing a unified table architecture of an in-memory computing system, the unified table architecture having a multi-level storage architecture, the storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage;
starting an internal transaction within the multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure;
reading committed data from the first level storage structure as specified by the internal transaction;
inserting the committed data from the first level storage structure into the second level storage structure in a bulk insertion process;
marking the committed data as being deleted from the first level storage; and
committing the internal transaction to the multi-level storage when the committed data has been inserted into the second level storage structure.
5. The computer implemented method in accordance with claim 4 , wherein the bulk insertion process includes:
generating a page identifier for the committed data; and
creating an undo log for the page identifier.
6. The computer implemented method in accordance with claim 4 , further comprising marking the committed data as being deleted from the first level storage further includes adding a page delete version of the committed data to the first level storage.
7. The computer implemented method in accordance with claim 4 , wherein the starting, the reading, the inserting, the marking, and the committing are performed by one or more processors.
8. A system comprising:
at least one programmable processor;
a multi-level storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage, a method of log-less atomic data movement; and
a non-transitory machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising:
starting an internal transaction within the multi-level storage architecture, the internal transaction to merge data from the first level storage structure to the second level storage structure;
reading committed data from the first level storage structure as specified by the internal transaction;
inserting the committed data from the first level storage structure into the second level storage structure in a bulk insertion process;
marking the committed data as being deleted from the first level storage; and
committing the internal transaction to the multi-level storage when the committed data has been inserted into the second level storage structure.
9. The system in accordance with claim 8 , wherein the bulk insertion process includes:
generating a page identifier for the committed data; and
creating an undo log for the page identifier.
10. The system in accordance with claim 8 , wherein the operations further comprise marking the committed data as being deleted from the first level storage further includes adding a page delete version of the committed data to the first level storage.