Hybrid column store providing both paged and memory-resident configurations
Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.
1. A database system, comprising:
a hybrid column store, comprising:
an in-memory store including a buffer cache configured to store a representation of a database object;
an on-disk store comprising an in-memory primitive store and a paged primitive store including one or more pages located outside the buffer cache, and configured to store a primitive for the representation of the database object, wherein the primitive is saved as an in-memory form primitive in the in-memory primitive store as a compressed version of the representation of the database object by a contiguous block of data or a paged form primitive in the paged primitive store as a compressed version of the representation of the database object on the one or more pages of the paged primitive store, wherein the in-memory form primitive and the paged form primitive are a byte-compatible representation of the database object to provide a unified persistence format for the database object.
2. The database system of claim 1 , wherein the representation of the database object includes the database object or a substructure of the database object.
3. The database system of claim 2 , wherein the substructure of the database object includes a dictionary, a data vector, an index, one or more values, or one or more value identifiers corresponding to the one or more values.
4. The database system of claim 1 , wherein the database object includes a database column containing one or more values.
5. The database system of claim 1 , wherein the in-memory form primitive and the paged form primitive have a same application program interface (API).
6. The database system of claim 1 , wherein the primitive contains a compressed version of the representation of the database object that is derived by using one or more compression schemes selected from an n-bit compression, a range compression, and an offset compression.
7. The database system of claim 1 , wherein the in-memory primitive store is configured to save the in-memory form primitive.
8. The database system of claim 1 , wherein the hybrid column store is included in a primary database, and the database system further comprises a secondary database provides only an in-memory representation or a paged representation of the database object.
9. The database system of claim 1 , wherein the in-memory store is configured to further include a tag for the database object to indicate that the representation of the database object is stored as the in-memory form primitive or the paged form primitive.
10. The database system of claim 1 , wherein the in-memory store further includes a buffer cache storing one or more pages retrieved from the paged primitive store.
11. A method performed by a database system, comprising:
performing one or more compression schemes on a database object or a substructure of the database object to derive a primitive for the database object or the substructure of the database object, wherein the database object or the substructure of the database object is stored in an in-memory store; and
saving the primitive as an in-memory form primitive in an in-memory primitive store or a paged form primitive in a paged primitive store, wherein the in-memory form primitive and the paged form primitive are a byte-compatible representation of the database object or the substructure of the database object to provide a unified persistence format for the database object.
12. The method of claim 11 , further comprising:
loading one or more pages of the paged primitive store into a buffer cache of the in-memory store; and
decompressing one or more database objects stored in the buffer cache.
13. The method of claim 11 , further comprising:
creating the substructure of the database object to represent the database object; and,
deriving the primitive for the substructure of the database object.
14. The method of claim 11 , further comprising:
receiving a load configuration;
determining, based on the load configuration, whether to save the primitive as the in-memory form primitive in the in-memory primitive store or the paged form primitive in the paged primitive store.
15. The method of claim 11 , wherein the in-memory form primitive and the paged form primitive have a same application program interface (API).
16. The method of claim 11 , wherein the substructure of the database object includes a dictionary, a data vector, an index, one or more values, or one or more value identifiers corresponding to the one or more values.
17. A non-transitory computer-readable device having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:
performing one or more compression schemes on a database object or a substructure of the database object to derive a primitive for the database object or the substructure of the database object, wherein the database object or the substructure of the database object is stored in an in-memory store; and
saving the primitive as an in-memory form primitive in an in-memory primitive store or a paged form primitive in a paged primitive store, wherein the in-memory form primitive and the paged form primitive are a byte-compatible representation of the database object or the substructure of the database object to provide a unified persistence format for the database object.
18. The non-transitory computer-readable device of claim 17 , wherein the in-memory form primitive and the paged form primitive have a same application program interface (API).
19. The non-transitory computer-readable device of claim 18 , wherein the substructure of the database object includes a dictionary, a data vector, an index, one or more values, or one or more value identifiers corresponding to the one or more values.
20. The non-transitory computer-readable device of claim 17 , wherein the database object includes a database column containing one or more values.