Cloud-native global file system with reshapable caching
A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include a reshapable caching scheme for the local filer. Like striped caches, the approach uses a stripe, but the striping is implemented via a true RAID 0 (disk striping) rather than as a striped LV (logical volume) device. This approach allows for a “reshape” operation to convert from a n-way stripe set to a n+1-way stripe set. Preferably, a reshape involves redistributing each block on disk to its new calculated home. For example, going from a single disk to a two disk set would move every other block from disk 1 to disk 2, and rearrange the blocks on disk 1 to fill in the “holes”. Performance after the reshape matches that of a striped cache. In one embodiment, the cache is structured as a “degraded” RAID 4.
1. A method to provide storage to an enterprise, comprising:
providing a filer configured to be hosted on a physical or virtual machine associated with the enterprise, the filer being configured to represent, to the enterprise, a local file system whose data is stored in an object store associated with a cloud-based storage service provider;
configuring a cache in the filer, the cache initially provisioned with a minimal configuration corresponding to a single disk RAID 0 or degraded RAID 4;
reshaping the cache by:
receiving an indication that a new disk is associated with a physical or virtual machine, and
in response to receiving the indication, automatically re-organizing data in the cache to utilize the new disk efficiently, thereby reshaping the cache;
wherein provisioning the cache with the minimal configuration enables the reshaping to be performed without erasing and re-loading the data in the cache and without requiring a pause in service during reshaping.
2. The method as described in claim 1 wherein reshaping is implemented upon a first boot of the physical or virtual machine.
3. The method as described in claim 1 wherein the filer automatically reorganizes data by:
enumerating all free disks associated with the physical or virtual machine;
assigning a smallest of the free disks as a copy-on-write (COW) disk; and
performing an n-way stripe over all remaining free disks, using as much of those disks as possible.
4. The method as described in claim 1 further including growing a size of an existing disk in the cache.
5. A computer program product in a non-transitory computer-readable media, the computer program product providing a filer configured to be hosted and executed on a physical or virtual machine associated with the enterprise, the filer being configured to represent, to the enterprise, a local file system whose data is stored in an object store associated with a cloud-based storage service provider, the computer program product comprising program code configured to:
instantiate a cache, the cache initially provisioned with a minimal configuration corresponding to a single disk RAID 0 or degraded RAID 4; and
reshape the cache by:
receiving an indication that a new disk is associated with a physical or virtual machine, and
in response to receiving the indication, automatically re-organizing data in the cache to utilize the new disk efficiently, thereby reshaping the cache;
wherein provisioning the cache with the minimal configuration enables the reshaping to be performed without erasing and re-loading the data in the cache and without requiring a pause in service during reshaping.
6. The computer program product as described in claim 5 wherein the program code configured to reshape the cache is implemented upon a first boot of the physical or virtual machine.
7. The computer program product as described in claim 5 wherein the program code configured to automatically reorganize data includes program code configured to:
enumerate all free disks associated with the physical or virtual machine;
assign a smallest of the free disks as a copy-on-write (COW) disk; and
perform an n-way stripe over all remaining free disks, using as much of those disks as possible.
8. The computer program product as described in claim 5 further including program code to grow a size of an existing disk in the cache.