Integrating database management system and external cache
Example systems and methods for integrating a primary data store (e.g., a source-of-truth relational database management system) and a secondary data store (e.g., external cache) are described. The approach implements a modified read-through/write-through data access pattern that integrates read and write flows in order to support high-concurrency environments while maintaining immediate consistency between stores. Writes are handled using a three-phase flow that avoids concurrency-related race conditions and the need to block in the secondary store in order to maintain cross-store consistency. Reads are never dirty and will repair the secondary store as needed without conflicting with repairs that may have been initiated by other sessions.
1. A method comprising:
receiving, by one or more processors of one or more servers associated with a data access logic (DAL) library, a request for a put operation of an object, the DAL library managing persistence of a secondary data store associated with a primary data store;
determining whether the secondary data store includes the object; and
in response to a determination that the secondary data store includes the object:
setting a cache optimistic locking value in metadata of the object, the cache optimistic locking value representing a first unique identifier generated for the object to manage optimistic locking at the secondary data store;
updating the object in the primary data store with the cache optimistic locking value; and
updating content data of the object in the secondary data store in response to a determination that the cache optimistic locking value has not been changed;
setting a check and set (CAS) value for the object;
setting a time-to-live (TTL) for the object;
setting a database optimistic locking value representing a second unique identifier generated for the object to manage optimistic locking at the primary data store;
updating the metadata in the primary data store, the metadata including the database optimistic locking value;
determining whether the database optimistic locking value has been changed; and
in response to a determination that the database optimistic locking value has not been changed, determining whether the CAS value has not been changed during the TTL.
2. The method of claim 1 , further comprising:
in response to a determination that the secondary data store does not include the object:
updating the content data of the object in the secondary data store; and
inserting the content data of the object in the primary data store.
3. The method of claim 1 , further comprising storing the object in the primary data store in response to a determination that the CAS value has not been changed.
4. The method of claim 1 , wherein updating the content data of the object in the secondary data store comprises updating the content data of the object in the secondary data store in response to the determination that the CAS value has not been changed during the TTL.
5. The method of claim 1 , wherein the secondary data store is an external cache that is physically separate from the primary data store.
6. A method comprising:
receiving, by one or more processors of one or more servers in a data access logic (DAL) framework, a request for a get operation of an object wherein the DAL framework manages cache persistence associated with a cache and a database, and wherein the cache is a level 2 (L2);
determining whether the cache, that is associated with the database, includes the object; and
in response to a determination that the cache does not include the object:
retrieving content data of the object from the database;
writing the content data of the object to the cache, wherein writing the content data of the object to the cache comprises:
setting a cache optimistic locking value for the object, the cache optimistic locking value representing a unique identifier generated for the object to manage optimistic locking at the cache;
retrieving a current cache optimistic locking value associated with the object;
determining whether the cache optimistic locking value has not been changed based on the current cache optimistic locking value; and
in response to a determination that the cache optimistic locking value has not been changed, writing the content data of the object to the cache; and
returning the content data of the object.
7. The method of claim 6 , wherein the cache is an external cache that is physically separate from the database.
8. The method of claim 6 , further comprising, in response to a determination that the cache includes the object, returning the content data of the object.
9. An apparatus comprising:
a memory configured to store data and one or more sets of instructions; and
one or more processors coupled to the memory, the one or more processors configured to execute the one or more sets of instructions and perform operations comprising:
receiving a request for a write operation of an object;
determining whether a secondary data store associated with a database includes the object;
in response to a determination that the secondary data store includes the object:
setting a cache optimistic locking value associated with the object in metadata of the object in the secondary data store, the cache optimistic locking value representing a unique identifier generated for the object to manage optimistic locking at the secondary data store;
updating the object in the database with the cache optimistic locking value; and
updating content data of the object in the secondary data store in response to a determination that the cache optimistic locking value has not been changed;
setting a check and set (CAS) value for the object;
setting a time-to-live (TTL) for the object, wherein the metadata includes the cache optimistic locking value and the TTL;
setting a database optimistic locking value representing a second unique identifier generated for the object to manage optimistic locking at the database;
writing the metadata in the database, the metadata including the database optimistic locking value;
determining whether the database optimistic locking value has been changed; and
in response to a determination that the database optimistic locking value has not been changed, determining whether the CAS value has not been changed during the TTL.
10. The apparatus of claim 9 , wherein the apparatus comprises a data access layer (DAL) that manages cache persistence associated with the secondary data store and the database.
11. The apparatus of claim 9 , wherein the operations further comprise:
in response to a determination that the secondary data store does not include the object:
updating the content data of the object in the secondary data store; and
updating the content data of the object in the database.
12. The apparatus of claim 9 wherein the operations further comprise storing the object in the database in response to a determination that the CAS value has not been changed.
13. The apparatus of claim 9 wherein the updating the content data of the object in the secondary data store comprises updating the content data of the object in the secondary data store in response to the determination that the CAS value has not been changed during the TTL.
14. The apparatus of claim 9 , wherein the secondary data store is an external cache that is physically separate from the database.