Decentralized data management across highly distributed systems

1. A method comprising:

in a system environment comprising a plurality of computing resources, each computing resource comprising a computing platform, wherein at least a portion of the computing resources are mobile, the system environment comprising a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources comprises a given messaging node and a given data node;

managing transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing at least a portion of the plurality of computing resources to obtain the policy file;

obtaining the policy file at the portion of the plurality of computing resources in accordance with the message; and

implementing the one or more policies of the policy file at the portion of the plurality of computing resources;

wherein the step of managing transfer of the data set further comprises assigning a content address to the policy file and specifying a pointer to the content address in the message to direct the portion of the plurality of computing resources where to obtain the policy file;

wherein the one or more policies of the policy file include instructions relating to at least one of download, distribution, storage or accessibility of at least portions of the data set; and

wherein the method is implemented via one or more processing devices each comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the step of managing transfer of the data set further comprises logging a confirmation message indicating implementation of the one or more policies of the policy file by the portion of the plurality of computing resources as the computing resources implements the one or more policies.

3. The method of claim 2 , wherein the confirmation messages are retrievable to audit the percentage of the portion of the plurality of computing resources that have implemented the one or more policies.

4. The method of claim 1 , wherein the policy file specifies an amount of the data set that given types of the computing resources are instructed to store.

5. The method of claim 4 , wherein a first type of computing resource is instructed to store the entire data set.

6. The method of claim 5 , wherein the first type of computing resource is a stationary computing resource.

7. The method of claim 5 , wherein a second type of computing resource is instructed to store at least a predetermined fraction of the entire data set.

8. The method of claim 7 , wherein the second type of computing resource is a mobile computing resource.

9. The method of claim 1 , wherein the step of managing transfer of the data set further comprises partitioning the data set into chunks.

10. The method of claim 9 , wherein the step of managing transfer of the data set further comprises inserting one or more of the chunks into the decentralized data network and inserting the message into the decentralized messaging network, the message instructing the portion of the plurality of computing resources to obtain the one or more chunks.

11. The method of claim 10 , wherein the step of managing transfer of the data set further comprises assigning the content address to each of the chunks and inserting the pointer to the content address of each chunk into the decentralized messaging network to direct the portion of the plurality of computing resources where to obtain the one or more chunks.

12. The method of claim 10 , wherein the step of managing transfer of the data set further comprises a given one of a given type of the plurality of computing resources sharing one or more chunks of the data set which it currently stores with one or more others of the given type of the plurality of computing resources that do not currently store the one or more chunks.

13. The method of claim 12 , wherein the sharing step is implemented using a distributed hash table architecture.

14. The method of claim 10 , wherein the step of managing transfer of the data set further comprises managing an instance wherein a given computing resource does not have the storage capacity to store the one or more chunks.

15. The method of claim 1 , wherein the step of managing transfer of the data set further comprises maintaining a reachable status for each of the plurality of computing resources and setting one of the one or more policies in the policy file to ensure that the entire data set is reachable by any given computing resource.

16. The method of claim 1 , wherein the step of managing transfer of the data set further comprises aggregating functionalities from two or more of the computing resources to ensure that the aggregation of two or more computing resources obtains the entire data set.

17. The method of claim 1 , wherein the plurality of computing resources comprises a cloud platform at a first level of the system environment, a plurality of edge servers coupled to the cloud platform at a second level of the system environment, and a plurality of mobile compute platforms coupled to the plurality of edge servers at a third level of the system environment.

18. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by one or more processing device causes the one or more processing devices to perform steps of:

in a system environment comprising a plurality of computing resources, each computing resource comprising a computing platform, wherein at least a portion of the computing resources are mobile, the system environment comprising a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources comprises a given messaging node and a given data node; and

managing transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing at least a portion of the plurality of computing resources to obtain the policy file;

obtaining the policy file at the portion of the plurality of computing resources in accordance with the message; and

implementing the one or more policies of the policy file at the portion of the plurality of computing resources;

wherein the step of managing transfer of the data set further comprises assigning a content address to the policy file and specifying a pointer to the content address in the message to direct the portion of the plurality of computing resources where to obtain the policy file; and

wherein the one or more policies of the policy file include instructions relating to at least one of download, distribution, storage or accessibility of at least portions of the data set.

19. A system comprising:

one or more processing devices coupled to a memory and configured to:

in a system environment comprising a plurality of computing resources, each computing resource comprising a computing platform, wherein at least a portion of the computing resources are mobile, the system environment comprising a decentralized messaging network of interconnected messaging nodes and a decentralized data network of interconnected data nodes, wherein each of the plurality of computing resources comprises a given messaging node and a given data node; and

manage transfer of a data set between the plurality of computing resources in association with the decentralized messaging network and the decentralized data network, wherein managing transfer of the data set comprises inserting a policy file into the decentralized data network specifying one or more policies for managing the transfer of the data set, and inserting a message into the decentralized messaging network instructing at least a portion of the plurality of computing resources to obtain the policy file;

obtain the policy file at the portion of the plurality of computing resources in accordance with the message; and

implement the one or more policies of the policy file at the portion of the plurality of computing resources;

wherein managing transfer of the data set further comprises assigning a content address to the policy file and specifying a pointer to the content address in the message to direct the portion of the plurality of computing resources where to obtain the policy file; and

wherein the one or more policies of the policy file include instructions relating to at least one of download, distribution, storage or accessibility of at least portions of the data set.

20. The article of manufacture of claim 18 , wherein:

the step of managing transfer of the data set further comprises logging a confirmation message indicating implementation of the one or more policies of the policy file by the portion of the plurality of computing resources as the computing resources implement the one or more policies; and

wherein the confirmation messages are retrievable to audit the percentage of the portion of the plurality of computing resources that have implemented the one or more policies.

21. The system of claim 19 , wherein:

managing transfer of the data set further comprises logging a confirmation message indicating implementation of the one or more policies of the policy file by the portion of the plurality of computing resources as the computing resources implement the one or more policies; and

wherein the confirmation messages are retrievable to audit the percentage of the portion of the plurality of computing resources that have implemented the one or more policies.