Gateway for cloud-based secure storage
View Patent ↗The systems and methods disclosed herein transparently provide an improved scalable cloud-based dynamically adjustable or configurable storage volume. In one aspect, a gateway provides a dynamically or configurably adjustable storage volume, including a local cache. The storage volume may be transparently adjusted for the amount of data that needs to be stored using available local or cloud-based storage. The gateway may use caching techniques and block clustering to provide gains in access latency compared to existing gateway systems, while providing scalable off-premises storage.
1. A method for providing data recovery to a client computer system using cloud-based storage, the method comprising:
detecting a request to capture a snapshot of a local file system of the client computer system at a first timestamp, wherein one or more data files associated with the client computer system are transparently stored to a storage volume, the storage volume comprising a local cache memory communicatively coupled to the client computer system and a cloud library comprising one or more remote storage devices;
in response to detecting the request, sending a snapshot capture indicator including the first timestamp to a gateway manager associated with the storage volume;
generating, using the gateway manager, a first capture of a state of the local cache memory at the first timestamp;
requesting, using the gateway manger, a second capture of a state of one or more cluster blocks stored by the one or more remote storage devices at the first timestamp, the cluster blocks including the one or more data files;
generating a capture version number for the first and second capture based on the snapshot capture indicator;
causing the storage volume to store the first capture, the second capture and the capture version number; and
wherein causing the storage volume to store the first or the second capture comprises:
applying, at the local cache memory, a first cryptographic operation to the first or the second capture based on a first encryption key, and
applying, at a cloud interface, a second cryptographic operation based on a second encryption key to the first or the second capture that is encrypted with the first encryption key.
2. The method of claim 1 , further comprising:
presenting, to the client computer system, the second capture in synchronization with the first capture in response to a second request from the client computer system to restore the state of the file system associated with the first timestamp.
3. The method of claim 1 , further comprising:
detecting a data access request to recover a version of the one or more data files associated with the first timestamp; and
transparently accessing the first or the second capture based on the first timestamp from the storage volume; and
transparently retrieving the version of the one or more data files from the first or the second capture.
4. The method of claim 1 , further comprising storing the first encryption key, the second encryption key, or both in a separate storage location from the first or the second capture.
5. The method of claim 1 , wherein causing the storage volume to store the first or the second capture comprises:
causing the first or the second capture to be distributed in a plurality of data shares located in the one or more remote storage devices.
6. The method of claim 5 , wherein causing the first or the second capture to be distributed in a plurality of data shares comprises:
generating a plurality of cluster blocks from the first or the second capture;
splitting each cluster block into a plurality of secondary data units and causing each secondary data unit to be placed into one of the plurality of data shares, wherein each cluster block is restorable by recombining a subset less than all of the secondary data units from the plurality of data shares.
7. The method of claim 1 , wherein causing the storage volume to store the first or the second capture comprises:
causing the first or the second capture associated with the first timestamp and the version number to be stored in a designated data recovery folder.
8. The method of claim 1 , further comprising:
causing the storage volume to store the first or the second capture associated with the first timestamp without overwriting a prior capture associated with an earlier timestamp.
9. A system for providing data recovery to a client computer system using cloud-based storage, the system comprising:
a user communication interface to detect a request to capture a snapshot of a local file system of the client computer system at a first timestamp, wherein one or more data files associated with the client computer system are transparently stored to a storage volume, the storage volume comprising a local cache memory communicatively coupled to the client computer system and a cloud library comprising one or more remote storage devices;
a gateway manager, communicatively coupled to the user communication interface, to:
in response to detecting the request, receive a snapshot capture indicator including the first timestamp with the storage volume, and
generate a first capture of a state of the local cache memory at the first timestamp;
a cloud interface to send a request for a second capture of a state of one or more cluster blocks stored by the one or more remote storage devices at the first timestamp, the cluster blocks including the one or more data files,
wherein a capture version number for the first and second capture is generated based on the snapshot capture indicator, and the first capture, the second capture and the capture version number are stored at the storage volume; and
wherein the gateway manager is further configured to:
apply, at the local cache memory, a first cryptographic operation to the first or the second capture based on a first encryption key, and
apply, at a cloud interface, a second cryptographic operation based on a second encryption key to the first or the second capture that is encrypted with the first encryption key.
10. The system of claim 9 , wherein the user communication interface is further configured to present, to the client computer system, the second capture in synchronization with the first capture in response to a second request from the client computer system to restore the state of the file system associated with the first timestamp.
11. The system of claim 9 , wherein the gateway manager is further configured to:
detect a data access request to recover a version of the one or more data files associated with the first timestamp; and
transparently access the first or the second capture based on the first timestamp from the storage volume; and
transparently retrieve the version of the one or more data files from the first or the second capture.
12. The system of claim 9 , wherein the gateway manager is further configured to store the first encryption key, the second encryption key, or both in a separate storage location from the first or the second capture.
13. The system of claim 9 , wherein the gateway manager is further configured to:
cause the first or the second capture to be distributed in a plurality of data shares located in the one or more remote storage devices.
14. The system of claim 13 , wherein the gateway manager is further configured to:
generate a plurality of cluster blocks from the first or the second capture;
split each cluster block into a plurality of secondary data units and causing each secondary data unit to be placed into one of the plurality of data shares, wherein each cluster block is restorable by recombining a subset less than all of the secondary data units from the plurality of data shares.
15. The system of claim 9 , wherein the gateway manager is further configured to:
cause the first or the second capture associated with the first timestamp and the version number to be stored in a designated data recovery folder.
16. The system of claim 9 , wherein the gateway manager is further configured to cause the storage volume to store the first or the second capture associated with the first timestamp without overwriting a prior capture associated with an earlier timestamp.
17. A non-transitory computer-readable medium storing processor-executable instructions for providing data recovery to a client computer system using cloud-based storage, comprising instructions executable by one or more computer processors to:
detect a request to capture a snapshot of a local file system of the client computer system at a first timestamp, wherein one or more data files associated with the client computer system are transparently stored to a storage volume, the storage volume comprising a local cache memory communicatively coupled to the client computer system and a cloud library comprising one or more remote storage devices;
in response to detecting the request, send a snapshot capture indicator including the first timestamp to a gateway manager associated with the storage volume;
generate a first capture of a state of the local cache memory at the first timestamp;
request a second capture of a state of one or more cluster blocks stored by the one or more remote storage devices at the first timestamp, the cluster blocks including the one or more data files;
generate a capture version number for the first and second capture based on the snapshot capture indicator;
cause the storage volume to store the first capture, the second capture and the capture version number; and
wherein causing the storage volume to store the first or the second capture comprises:
applying, at the local cache memory, a first cryptographic operation to the first or the second capture based on a first encryption key, and
applying, at a cloud interface, a second cryptographic operation based on a second encryption key to the first or the second capture that is encrypted with the first encryption key.
18. The non-transitory computer-readable medium of claim 17 , further comprising:
instructions executable by the one or more computer processors to present, to the client computer system, the second capture in synchronization with the first capture in response to a second request from the client computer system to restore the state of the file system associated with the first timestamp.
19. The non-transitory computer-readable medium of claim 17 , further comprising instructions executable by the one or more computer processors to:
detect a data access request to recover a version of the one or more data files associated with the first timestamp; transparently access the first or the second capture based on the first timestamp from the storage volume; and
transparently retrieve the version of the one or more data files from the first or the second capture.
20. The non-transitory computer-readable medium of claim 17 , further comprising instructions executable by the one or more computer processors to store the first encryption key, the second encryption key, or both in a separate storage location from the first or the second capture.
21. The non-transitory computer-readable medium of claim 17 , further comprising instructions executable by the one or more computer processors to cause the storage volume to store the first or the second capture by:
causing the first or the second capture to be distributed in a plurality of data shares located in the one or more remote storage devices.
22. The non-transitory computer-readable medium of claim 21 , further comprising instructions executable by the one or more computer processors to cause the first or the second capture to be distributed in a plurality of data shares by:
generating a plurality of cluster blocks from the first or the second capture; and
splitting each cluster block into a plurality of secondary data units and causing each secondary data unit to be placed into one of the plurality of data shares, wherein each cluster block is restorable by recombining a subset less than all of the secondary data units from the plurality of data shares.
23. The non-transitory computer-readable medium of claim 17 , further comprising instructions executable by the one or more computer processors to cause the storage volume to store the first or the second capture by:
causing the first or the second capture associated with the first timestamp and the version number to be stored in a designated data recovery folder.
24. The non-transitory computer-readable medium of claim 17 , further comprising:
instructions executable by the one or more computer processors to cause the storage volume to store the first or the second capture associated with the first timestamp without overwriting a prior capture associated with an earlier timestamp.