Storing large objects on disk and not in main memory of an in-memory database system
View Patent ↗A method, computer program product and system are provided. The method, computer program product and system execute a process for determining a size of an object, the object having raw data that is operable upon by one or more physical operators. If the object is smaller than a threshold size, the object is stored in main memory of an in-memory database system. If the object is equal to or larger than the threshold size, the object is stored in a persistency of a disk storage, where storing the object in a disk storage further includes generating a global container identifier (ID) for the object, the global container ID referencing raw data of the object stored in the persistency of the disk storage.
1. A method comprising:
determining a size of an object, the object having raw data that is operable upon by one or more physical operators;
if the object is smaller than a threshold size, storing the object in main memory of a first host system of a plurality of host systems in an in-memory database system, the first host system including a first column store, a first row store and a first persistency associated with a first disk storage, the plurality of host systems further comprising a second host system that includes a second column store, a second row store and a second persistency associated with a second disk storage; and
if the object is equal to or larger than the threshold size, storing the object in the first persistency, the storing of the object in the first persistency comprising generating a global container identifier (ID) for the object stored in the first persistency, the global container ID referencing raw data of the object stored in the first persistency, the global container ID including a volume ID specifying the first persistency where the object is stored and a container ID that references a file of a plurality of files inside the first persistency, the container ID being separate from the volume ID, the global container ID for the first host system being stored in the first column store and the first row store, the global container ID being allowed to be moved from the first row store to the second row store so that the second host system can access the object stored in the first persistency within the first host system.
2. The method in accordance with claim 1 , wherein:
the main memory of the first host system comprises the first column store and the first row store; and
a main memory of the second host system comprises the second column store and the second row store.
3. The method in accordance with claim 1 , wherein the global container ID of the object stored in the first persistency is stored in the first column store and the second column store so that the main memory of the first host system has access to the object stored in the first persistency.
4. The method in accordance with claim 1 , wherein the object stored in the first persistency is accessed in response to a select operation specifying the global container ID.
5. A non-transitory computer program product comprising a machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
determine a size of an object, the object having raw data that is operable upon by one or more physical operators;
if the object is smaller than a threshold size, store the object in main memory of a first host system of a plurality of host systems in an in-memory database system, the first host system including a first column store, a first row store and a first persistency associated with a first disk storage, the plurality of host systems further comprising a second host system that includes a second column store, a second row store and a second persistency associated with a second disk storage; and
if the object is equal to or larger than the threshold size, store the object in the first persistency, the storing of the object in the first persistency comprising generating a global container identifier (ID) for the object stored in the first persistency, the global container ID referencing raw data of the object stored in the first persistency, the global container ID including a volume ID specifying the first persistency where the object is stored and a container ID that references a file of a plurality of files inside the first persistency, the container ID being separate from the volume ID, the global container ID for the first host system being stored in the first column store and the first row store, the global container ID being allowed to be moved from the first row store to the second row store, the global container ID in the second store allowing the second host system to access the object stored in the first persistency within the first host system.
6. The non-transitory computer program product in accordance with claim 5 , wherein:
the main memory of the first host system comprises the first column store and the first row store; and
a main memory of the second host system comprises the second column store and the second row store.
7. The non-transitory computer program product in accordance with claim 5 , wherein the global container ID of the object stored in the first persistency is stored in the first column store and the second column store so that the main memory of the first host system has access to the object stored in the first persistency.
8. The non-transitory computer program product in accordance with claim 5 , wherein the object stored in the first persistency is accessed in response to a select operation specifying the global container ID.
9. A system comprising:
at least one programmable processor; and
a machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising:
determining a size of an object, the object having raw data that is operable upon by one or more physical operators;
if the object is smaller than a threshold size, storing the object in main memory of a first host system of a plurality of host systems in an in-memory database system, the first host system including a first column store, a first row store and a first persistency associated with a first disk storage, the plurality of host systems further comprising a second host system that includes a second column store, a second row store and a second persistency associated with a second disk storage; and
if the object is equal to or larger than the threshold size, storing the object in the first persistency, the storing of the object in the first persistency comprising generating a global container identifier (ID) for the object stored in the first persistency, the global container ID referencing raw data of the object stored in the first persistency, the global container ID including a volume ID specifying the first persistency where the object is stored and a container ID that references a file of a plurality of files inside the first persistency, the container ID being different from the volume ID, the global container ID for the first host system being stored in the first column store and the first row store, the global container ID being allowed to be moved from the first row store to the second row store so that the second host system can access the object stored in the first persistency within the first host system.
10. The system in accordance with claim 9 , wherein:
the main memory of the first host system comprises the first column store and the first row store; and
a main memory of the second host system comprises the second column store and the second row store.
11. The system in accordance with claim 9 , wherein the global container ID of the object stored in the first persistency is stored in the first column store and the second column store so that the main memory of the first host system has access to the object stored in the first persistency.
12. The system in accordance with claim 9 , wherein the object stored in the first persistency is accessed in response to a select operation specifying the global container ID.