Minimizing data loss in a computer storage environment with non-guaranteed continuous network connectivity
Embodiments for preventing data loss in a computing storage environment by a processor. One or more offline events in a locality failing to have continuous network connectivity may be detected. An ad-hoc distributed file system, having one or more edge devices associated with the locality, may be created for storing data produced during the one or more offline events.
1. A method, by a processor, for preventing data loss in a computing storage environment, comprising:
maintaining a plurality of edge devices in a locality, wherein the plurality of edge devices are connected to a network and comprise Internet of Things (IoT) devices streaming data produced respectively thereon directly to a cloud computing storage system notwithstanding whether any local storage exists on at least some of the IoT devices;
detecting one or more offline events in the locality as a loss of network connectivity of the network between the plurality of edge devices in the locality and the cloud computing storage system;
establishing a distributed file system master node in response to the detected one or more offline events;
responsive to receiving the notification of the detected one or more offline events, grouping, by the distributed file system master node, a set of the plurality of edge devices into a region, wherein the region is a new region inclusive of the set of the plurality of edge devices having local storage;
transmitting distributed file system initiation requests, by the distributed file system master node to each of the set of the plurality of edge devices of the region, notifying each of the set of the plurality of edge devices to act as participating nodes in an ad-hoc distributed file system, wherein each of the participating nodes were not part of any distributed file system prior to the notification; and
responsive to receiving the distributed file system initiation requests, creating the ad-hoc distributed file system, having the set of the plurality of edge devices associated with the locality, for storing offline data produced during the one or more offline events.
2. The method of claim 1 , wherein the distributed file system master node is a gateway device of the locality.
3. The method of claim 1 , further including writing the offline data to the ad-hoc distributed file system by one or more agents operating on the set of the plurality of edge devices.
4. The method of claim 1 , further including reading the offline data produced during the one or more offline events from the ad-hoc distributed file system by a gateway device.
5. The method of claim 1 , further including determining the network connectivity between the locality and the cloud computing storage system is restored.
6. The method of claim 1 , further including forwarding the offline data to a gateway device by one or more agents on the set of the plurality of edge devices.
7. The method of claim 1 , further including forwarding the offline data read by a gateway device during the one or more offline events from the ad-hoc distributed file system to the cloud computing storage system.
8. A system for preventing data loss in a computing storage environment, comprising:
one or more computers with executable instructions that when executed cause the system to:
maintain a plurality of edge devices in a locality, wherein the plurality of edge devices are connected to a network and comprise Internet of Things (IoT) devices streaming data produced respectively thereon directly to a cloud computing storage system notwithstanding whether any local storage exists on at least some of the IoT devices;
detect one or more offline events in the locality as a loss of network connectivity of the network between the plurality of edge devices in the locality and the cloud computing storage system;
establish a distributed file system master node of the detected one or more offline events;
responsive to receiving the notification of the detected one or more offline events, group, by the distributed file system master node, a set of the plurality of edge devices into a region, wherein the region is a new region inclusive of the set of the plurality of edge devices having local storage;
transmit distributed file system initiation requests, by the distributed file system master node to each of the set of the plurality of edge devices of the region, notifying each of the set of the plurality of edge devices to act as participating nodes in an ad-hoc distributed file system, wherein each of the participating nodes were not part of any distributed file system prior to the notification; and
responsive to receiving the distributed file system initiation requests, create the ad-hoc distributed file system, having the set of the plurality of edge devices associated with the locality, for storing offline data produced during the one or more offline events.
9. The system of claim 8 , wherein the distributed file system master node is a gateway device of the locality.
10. The system of claim 8 , wherein the executable instructions that when executed cause the system to write the offline data to the ad-hoc distributed file system by one or more agents operating on the set of the plurality of edge devices.
11. The system of claim 8 , wherein the executable instructions that when executed cause the system to read the offline data produced during the one or more offline events from the ad-hoc distributed file system by a gateway device.
12. The system of claim 8 , wherein the executable instructions that when executed cause the system to
determine the network connectivity between the locality and the cloud computing storage system is restored.
13. The system of claim 8 , wherein the executable instructions that when executed cause the system to forward the offline data to a gateway device by one or more agents on the set of the plurality of edge devices.
14. The system of claim 8 , wherein the executable instructions that when executed cause the system to forward the offline data read by a gateway device during the one or more offline events from the ad-hoc distributed file system to the cloud computing storage system.
15. A computer program product for, by a processor, preventing data loss in a computing storage environment, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion that maintains a plurality of edge devices in a locality, wherein the plurality of edge devices are connected to a network and comprise Internet of Things (IoT) devices streaming data produced respectively thereon directly to a cloud computing storage system notwithstanding whether any local storage exists on at least some of the IoT devices;
an executable portion that detects one or more offline events in the locality as a loss of network connectivity of the network between the plurality of edge devices in the locality and the cloud computing storage system;
an executable portion that establishes a distributed file system master node of the detected one or more offline events;
an executable portion that, responsive to receiving the notification of the detected one or more offline events, groups, by the distributed file system master node, a set of the plurality of edge devices into a region, wherein the region is a new region inclusive of the set of the plurality of edge devices having local storage;
an executable portion that transmits distributed file system initiation requests, by the distributed file system master node to each of the set of the plurality of edge devices of the region, notifying each of the set of the plurality of edge devices to act as participating nodes in an ad-hoc distributed file system, wherein each of the participating nodes were not part of any distributed file system prior to the notification; and
an executable portion that, responsive to receiving the distributed file system initiation requests, creates the ad-hoc distributed file system, having the set of the plurality of edge devices associated with the locality, for storing offline data produced during the one or more offline events.
16. The computer program product of claim 15 , wherein the distributed file system master node is a gateway device of the locality.
17. The computer program product of claim 15 , further including an executable portion that:
writes the offline data to the ad-hoc distributed file system by one or more agents operating on the set of the plurality of edge devices; or
reads the offline data produced during the one or more offline events from the ad-hoc distributed file system by a gateway device.
18. The computer program product of claim 15 , further including an executable portion that determines the network connectivity between the locality and the cloud computing storage system is restored.
19. The computer program product of claim 15 , further including an executable portion that forwards the offline data to a gateway device by one or more agents on the set of the plurality of edge devices.
20. The computer program product of claim 15 , further including an executable portion that forwards the offline data read by a gateway device during the one or more offline events from the ad-hoc distributed file system to the cloud computing storage system.