Persistent key-value repository with a pluggable architecture to abstract physical storage
View Patent ↗A system and techniques are described for managing key-value pairs using an architecture that does not restrict its users to any specific platform or storage subsystem. According to one aspect of the invention, the repository is portable in that the architecture need not change based on the environment and platform in which the repository is used. Rather, the portion of the architecture that is platform-specific is limited to plug-ins at a storage abstraction layer of the repository. The plug-ins expose the same storage abstraction interface to the other layers of the repository, but are implemented to interact with different platforms and storage subsystems. Consequently, in response to being ported from one platform to another, the repository may simply change the plug-in to invoke for persistent storage operations.
1. A method for storing key-value pairs, the method comprising:
exposing, at an application programming layer, a repository interface through which clients may make calls to store and access key-value pair information in a repository;
receiving, through said repository interface, at a first component on a first node, from a client on the first node, a request to perform an operation, wherein the first component is associated with the first client;
in response to receiving the request at the first component, determining whether the first component has been designated as a caching component for the first node;
in response to determining that the first component has not been designated as a caching component for the first node, determining whether a second component on the first node has been designated as a caching component for the first node, wherein the second component is associated with a second client on the first node;
in response to determining that the second component has been designated as a caching component for the first node, determining, at the second component, whether information required by said request is contained in a local cache on the first node;
in response to determining that the information required by said request is not contained in the local cache on the first node, determining whether a third component on a second node has been designated as a master component, wherein the second node is separate from the first node, wherein the third component is associated with a third client on the second node;
in response to determining that the third component has been designated as the master component, performing the following steps at the third component:
servicing calls made through said repository interface by calling one of a plurality of plug-ins through a common interface of a storage abstraction layer;
determining which particular plug-in of said plurality of plug-ins corresponds to a type of storage subsystem to access during execution of said operation;
dynamically loading the particular plug-in that corresponds to said type of storage subsystem; and
performing said operation by making one or more calls through said common interface to said particular plug-in;
wherein each plug-in of said plurality of plug-ins exposes said common interface for accessing durably stored key-value pair information;
wherein each plug-in of said plurality of plug-ins is designed to interact with a particular type of storage subsystem in response to calls made through said common interface; and
wherein the storage subsystem with which each plug-in interacts is a different type of storage subsystem than the storage subsystem with which each other of said plurality of plug-ins interacts.
2. The method of claim 1 , further comprising the steps of:
designating the third component of said repository as the master component;
receiving, at components spread across a plurality of nodes of a cluster, calls through said repository interface from clients residing on said plurality of nodes;
directing all calls that require access to persistent storage to said master component;
wherein the master component is the only component of said repository permitted to perform the step of calling plug-ins through said common interface.
3. The method of claim 1 , further comprising the steps of:
receiving a request from a client through said repository interface;
prior to making a call through said common interface for said information required by said request, search for said information in a master cache.
4. The method of claim 3 , wherein:
the first client resides on the first node and the master cache resides on the second node; and
the method further comprises the step of, prior to searching for said information in said master cache, searching for said information on the local cache on said first node.
5. The method of claim 4 wherein:
the first node includes a plurality of components of said repository;
the method further comprises the steps of establishing only one component of said plurality of components as the caching component; and
if said first component is not said caching component, then causing a message to be sent from said first component to said caching component to cause said caching component to search said local cache for said information.
6. The method of claim 1 further comprising the steps of:
creating a code library that contains routines that implement said repository interface;
linking said code library to client code to allow routines in said client code to use said repository by calling said routines in said code library.
7. The method of claim 1 further comprising the steps of:
receiving a call from a client through a routine that implements a portion of said repository interface; and
based on a value of a parameter in said call, determining whether to establish a component of said repository that is associated with said client as a caching component responsible for managing a cache for said repository.
8. A computer-readable storage medium carrying instructions for storing key-value pairs, the instructions comprising instructions for performing the steps of:
exposing, at an application programming layer, a repository interface through which clients may make calls to store and access key-value pair information in a repository;
receiving, through said repository interface, at a first component on a first node, from a client on the first node, a request to perform an operation, wherein the first component is associated with the first client;
in response to receiving the request at the first component, determining whether the first component has been designated as a caching component for the first node;
in response to determining that the first component has not been designated as a caching component for the first node, determining whether a second component on the first node has been designated as a caching component for the first node, wherein the second component is associated with a second client on the first node;
in response to determining that the second component has been designated as a caching component for the first node, determining, at the second component, whether information required by said request is contained in a local cache on the first node;
in response to determining that the information required by said request is not contained in the local cache on the first node, determining whether a third component on a second node has been designated as a master component, wherein the second node is separate from the first node, wherein the third component is associated with a third client on the second node;
in response to determining that the third component has been designated as the master component, performing the following steps at the third component:
servicing calls made through said repository interface by calling one of a plurality of plug-ins through a common interface of a storage abstraction layer;
determining which particular plug-in of said plurality of plug-ins corresponds to a type of storage subsystem to access during execution of said operation;
dynamically loading the particular plug-in that corresponds to said type of storage subsystem; and
performing said operation by making one or more calls through said common interface to said particular plug-in;
wherein each plug-in of said plurality of plug-ins exposes said common interface for accessing durably stored key-value pair information;
wherein each plug-in of said plurality of plug-ins is designed to interact with a particular type of storage subsystem in response to calls made through said common interface; and
wherein the storage subsystem with which each plug-in interacts is a different type of storage subsystem than the storage subsystem with which each other of said plurality of plug-ins interacts.
9. The computer-readable storage medium of claim 8 further comprising instructions for performing the steps of:
designating a the third component of said repository as the master component;
receiving, at components spread across a plurality of nodes of a cluster, calls through said repository interface from clients residing on said plurality of nodes;
directing all calls that require access to persistent storage to said master component;
wherein the master component is the only component of said repository permitted to perform the step of calling plug-ins through said common interface.
10. The computer-readable storage medium of claim 8 further comprising instructions for performing the steps of:
receiving a request from a client through said repository interface;
prior to making a call through said common interface for said information required by said request, searching for said information in a master cache.
11. The computer-readable storage medium of claim 10 wherein:
the first client resides on the first node and the master cache resides on the second node; and
the tangible computer-readable medium further comprises instructions for performing the step of, prior to searching for said information in said master cache, searching for said information on the local cache on said first node.
12. The computer-readable storage medium of claim 11 wherein:
the first node includes a plurality of components of said repository;
the tangible computer-readable medium further comprises instructions for performing the steps of establishing only one component of said plurality of components as the caching component; and
if said first component is not said caching component, then causing a message to be sent from said first component to said caching component to cause said caching component to search said local cache for said information.
13. The computer-readable storage medium of claim 8 further comprising instructions for performing the steps of:
creating a code library that contains routines that implement said repository interface;
linking said code library to client code to allow routines in said client code to use said repository by calling said routines in said code library.
14. The computer-readable storage medium of claim 8 further comprising instructions for performing the steps of:
receiving a call from a client through a routine that implements a portion of said repository interface; and
based on a value of a parameter in said call, determining whether to establish a component of said repository that is associated with said client as a caching component responsible for managing a cache for said repository.