Enhanced caching operations based on a coherency metric
A global policy-driven framework for managing, revising, and implementing data coherency policies in a distributed data storage system. An edge node of the distributed data storage system receives a request from an application to store a data object in the system. The edge node, in response to receiving the request to store the data object, generates a coherency policy request, which is then submitted to the primary node of the system. The primary node of the system generates a coherency policy response, which is returned to the edge node. The edge node then stores the data object in the system in accordance with the coherency policy response.
1 . A method of operating a distributed data storage system, the method comprising:
receiving, at an edge node of the distributed data storage system, a request to store a data object;
transmitting, by the edge node, a request to obtain a coherency policy from a primary node of the distributed data storage system, wherein the request identifies one or more characteristics of the data object, and wherein the primary node selects the coherency policy based on the characteristics of the data object;
receiving, from the primary node, a response that identifies the coherency policy, wherein:
the coherency policy is one of a number of coherency policies, and
the number of coherency policies comprises strictly coherent caching operations and eventually coherent caching operations; and
storing, by the edge node, the data object in accordance with the coherency policy.
2 . The method of claim 1 , wherein:
strictly coherent caching operations comprise caching operations that synchronize copies of the data object without a delay in response to a modification of the data object; and
eventually coherent caching operations comprise caching operations that synchronize copies of the data object after a delay in response to a modification of the data object.
3 . The method of claim 1 , the method further comprising receiving, at the edge node, the request to store the data object from an application remote to the edge node.
4 . The method of claim 3 , the method further comprising identifying, by the edge node, the one or more characteristics of the data object based on metadata associated with the data object.
5 . The method of claim 1 , wherein the one or more characteristics of the data object comprise an object type of the data object and a corresponding application for the data object.
6 . The method of claim 1 , the method further comprising determining the coherency policy based on the one or more characteristics of the data object.
7 . The method of claim 6 , wherein determining the coherency policy based on the one or more characteristics of the data object comprises:
generating a vector for the data object, and
submitting the vector to a machine learning model trained to receive the vector as an input and, in response, to return the coherency policy.
8 . The method of claim 1 , the method further comprising:
generating, by the edge node, a lock request for the data object, the lock request comprising a request to lock one or more copies of the data object on each of one or more other edge nodes of the distributed data storage system; and
transmitting, by the edge node, the lock request to the primary node.
9 . The method of claim 1 , the method further comprising:
receiving, by the edge node, a lock instruction for the data object, the lock instruction comprising instructions to lock the data object on the edge node; and
locking, based on the lock instruction, the data object.
10 . A computing device comprising:
one or more non-transitory computer readable storage media;
one or more processors operatively coupled with the one or more computer readable storage media; and
a distributed data storage system comprising program instructions stored on the one or more computer readable storage media, wherein the program instructions, when executed by the one or more processors, direct the computing device to at least:
receive, at a primary node of the distributed data storage system, a request to obtain a coherency policy from the primary node, wherein:
the request to obtain the coherency policy corresponds to a request to store a data object in the distributed data storage system; and
the request to obtain the coherency policy identifies one or more characteristics of the data object,
determine the coherency policy based on the one or more characteristics of the data object, wherein:
the coherency policy is one of a number of coherency policies; and
the number of coherency policies comprises strictly coherent caching operations and eventually coherent caching operations, and
transmit a response that identifies the coherency policy to an edge node of the distributed data storage system.
11 . The computing device of claim 10 , wherein:
strictly coherent caching operations comprise caching operations that synchronize copies of the data object without a delay in response to a modification of the data object; and
eventually coherent caching operations comprise caching operations that synchronize copies of the data object after a delay in response to a modification of the data object.
12 . The computing device of claim 10 , wherein the program instructions directing the computing device to receive the request to obtain the coherency policy further comprise instructions that, when executed, direct the computing device to:
receive the request to obtain the coherency policy from the edge node of the distributed data storage system.
13 . The computing device of claim 10 , wherein the one or more characteristics of the data object comprise an object type of the data object and a corresponding application for the data object.
14 . The computing device of claim 13 , wherein the program instructions directing the computing device to determine the coherency policy based on the one or more characteristics of the data object further comprise instructions that, when executed, direct the computing device to:
determine the coherency policy based on the object type of the data object and the corresponding application for the data object.
15 . The computing device of claim 10 , wherein the program instructions directing the computing device to determine the coherency policy based on the one or more characteristics of the data object further comprise instructions that, when executed, direct the computing device to:
generate a vector for the data object, the vector having features that correspond to the one or more characteristics of the data object, and
submit the vector to a machine learning model trained to receive the vector as an input and, in response, to return the coherency policy.
16 . The computing device of claim 10 , wherein the program instructions further comprise instructions that, when executed, direct the computing device to:
receive a lock request for the data object, the lock request comprising a request to lock one or more copies of the data object on each of one or more other edge nodes of the distributed data storage system;
generate, based on the lock request for the data object, a lock instruction comprising instructions that direct each of one or more other edge nodes to lock one or more copies of the data object therein; and
transmit the lock instruction to the each of one or more other edge nodes.
17 . The computing device of claim 10 , wherein the program instructions further comprise instructions that, when executed, direct the computing device to:
receive, at the edge node, a lock instruction for the data object, the lock instruction comprising instructions to lock the data object on the edge node; and
lock, based on the lock instruction, the data object.
18 . The computing device of claim 10 , wherein the program instructions further comprise instructions that, when executed, direct the computing device to:
distribute the response to one or more other edge nodes of the distributed data storage system.
19 . The computing device of claim 18 , wherein the program instructions further comprise instructions that, when executed, direct the computing device to:
acquire, at the primary node, a current version of the data object; and
distribute the current version of the data object to each of the one or more other edge nodes.
20 . A method of operating a distributed data storage system, the method comprising:
receiving, at a primary node of the distributed data storage system, a request to obtain a coherency policy from the primary node, wherein:
the request to obtain the coherency policy corresponds to a request to store a data object in the distributed data storage system; and
the request to obtain the coherency policy identifies one or more characteristics of the data object,
determining, by the primary node, the coherency policy based on the one or more characteristics of the data object, wherein:
the coherency policy is one of a number of coherency policies, and
the number of coherency policies comprises strictly coherent caching operations and eventually coherent caching operations, and
transmitting, from the primary node to an edge node of the distributed data storage system, a response that identifies the coherency policy.