Decentralized data protection system using local metadata
In a decentralized system of nodes configured to provide data protection functionality, wherein at least a subset of the nodes store and share data using content-addresses managed via a distributed hash table in each of the subset of nodes, a given one of the subset of nodes locally stores: a data protection policy to be implemented by the given node; data protected on the given node; and metadata comprising information indicating placement of a given data set on one or more other ones of the subset of nodes. The given node accesses the locally stored metadata to manage protection of the given data set on the one or more other ones of the subset of nodes.
1. A method comprising:
in a decentralized system of nodes configured to provide data protection functionality, wherein at least a subset of the nodes store and share data using content-addresses managed via a distributed hash table in each of the subset of nodes;
locally storing in a given one of the subset of nodes: a data protection policy to be implemented by the given node; data protected on the given node; and metadata comprising information indicating placement of a given data set on one or more other ones of the subset of nodes;
the given one of the subset of nodes accessing the locally stored metadata to manage protection of the given data set on the one or more other ones of the subset of nodes;
the given one of the subset of nodes obtaining an up-to-date version of the data protection policy from a distributed ledger;
the given one of the subset of nodes, upon communication with another node, negotiating the up-to-date version of the data protection policy to ensure both nodes are implementing the same data protection policy; and
responsive to the other node of the subset of nodes having an out-of-date policy version of the data protection policy, rejecting the other node.
2. The method of claim 1 , further comprising updating the locally stored metadata to reflect the up-to-date version of the data protection policy that the given one of the subset of nodes is currently applying for the given data set.
3. The method of claim 1 , further comprising the given one of the subset of nodes executing a data placement process upon initial receipt of the given data set, wherein the data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.
4. The method of claim 1 , further comprising the given one of the subset of nodes receiving a query to access the given data set on one or more nodes on which the given data set is placed.
5. The method of claim 4 , further comprising the given one of the subset of nodes accessing the locally stored metadata to determine on which nodes the given data set is placed.
6. The method of claim 5 , further comprising the given one of the subset of nodes communicating with the nodes on which the given data set is placed.
7. The method of claim 1 , wherein the data protection policy comprises one or more data storage methods.
8. The method of claim 7 , wherein the one or more data storage methods comprise one of a duplication function, an archive function, and a RAID function.
9. The method of claim 7 , wherein the data protection policy comprises one or more data placement methods.
10. The method of claim 9 , wherein the one or more data placement methods comprise a consistent hash-based method.
11. The method of claim 9 , wherein the one or more data placement methods comprise one or more of a geographic property and a cloud provider property.
12. The method of claim 1 , further comprising the given one of the subset of nodes re-executing a data placement process upon checking that a failure occurred with respect to at least one of the nodes upon which the given data set is placed; wherein the re-executed data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.
13. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device of a given node causes the given node to perform steps of:
in a decentralized system of nodes, including the given node, configured to provide data, protection functionality, wherein at least a subset of the nodes store and share data using content-addresses managed via a distributed hash table in each of the subset of nodes;
locally storing at the given node: a data protection policy to be implemented by the given node; data protected on the given node; and metadata comprising information indicating placement of a given data set on one or more other ones of the subset of nodes;
accessing at the given node the locally stored metadata to manage protection of the given data set on the one or more other ones of the subset of nodes;
the given one of the subset of nodes obtaining an up-to-date version of the data protection policy from a distributed ledger;
the given one of the subset of nodes, upon communication with another node, negotiating the up-to-date version of the data protection policy to ensure both nodes are implementing the same data protection policy; and
responsive to the other node of the subset of nodes having an out-of-date policy version of the data protection policy, rejecting the other node.
14. The article of claim 13 , further comprising the given node executing a data placement process upon initial receipt of the given data set, wherein the data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.
15. The article of claim 13 , wherein the given node further performs the step of re-executing a data placement process upon checking that a failure occurred with respect to at least one of the nodes upon which the given data set is placed; wherein the re-executed data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.
16. An apparatus comprising at least one processing device, wherein the at least one processing device comprises a processor coupled to a memory to form a given node configured to:
in a decentralized system of nodes, including the given node, configured to provide data protection functionality, wherein at least a subset of the nodes store and share data using content-addresses managed via a distributed hash table in each of the subset of nodes;
locally store at the given node: a data protection policy to be implemented by the given node; data protected on the given node; and metadata comprising information indicating placement of a given data set on one or more other ones of the subset of nodes;
access at the given node the locally stored metadata to manage protection of the given data set on the one or more other ones of the subset of nodes;
the given one of the subset of nodes obtaining an up-to-date version of the data protection policy from a distributed ledger;
the given one of the subset of nodes, upon communication with another node, negotiating the up-to-date version of the data protection policy to ensure both nodes are implementing the same data protection policy; and
responsive to the other node of the subset of nodes having an out-of-date policy version of the data protection policy, reject the other node.
17. The apparatus of claim 16 , wherein the given node is further configured to execute a data placement process upon initial receipt of the given data set, wherein the data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.
18. The apparatus of claim 16 , wherein the given node is further configured to update the locally stored metadata to reflect the up-to-date version of the data protection policy that the given node is currently applying for the given data set.
19. The apparatus of claim 16 , wherein the given node is further configured to receive a query to access the given data set on one or more nodes on which the given data set is placed.
20. The apparatus of claim 16 , wherein the given node is further configured to re-execute a data placement process upon checking that a failure occurred with respect to at least one of the nodes upon which the given data set is placed; wherein the re-executed data placement process determines placement of the given data set on the one or more other ones of the subset of nodes in accordance with the data protection policy.