Cloud-native global file system with reshapable caching
A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include a reshapable caching scheme for the local filer. Like striped caches, the approach uses a stripe, but the striping is implemented via a true RAID 0 (disk striping) rather than as a striped LV (logical volume) device. This approach allows for a “reshape” operation to convert from a n-way stripe set to a n+1-way stripe set. Preferably, a reshape involves redistributing each block on disk to its new calculated home. For example, going from a single disk to a two disk set would move every other block from disk 1 to disk 2, and rearrange the blocks on disk 1 to fill in the “holes”. Performance after the reshape matches that of a striped cache. In one embodiment, the cache is structured as a “degraded” RAID 4.
1. An apparatus, comprising:
one or more hardware processors;
physical memory or storage associated with the one or more hardware processors for storing computer program code configured to be executed by the one or more hardware processors to provide storage for an enterprise, the computer program code configured as a filer associated with the enterprise, wherein the filer is configured to represent, to the enterprise, a local file system whose data is stored in an object store associated with a cloud-based storage service provider, the filer having a cache associated therewith, and wherein the cache is initially provisioned with a minimal configuration corresponding to a single disk RAID 0 or degraded RAID 4;
the filer being further configured to receive an indication that a new disk is associated with a physical or virtual machine, and in response to receiving the indication, automatically re-organize data in the cache to utilize the new disk efficiently, thereby reshaping the cache;
wherein provisioning the cache with the minimal configuration enables the reshaping to be performed without erasing and re-loading the data in the cache and without requiring a pause in service during reshaping.
2. The apparatus as described in claim 1 wherein the reshaping is implemented upon a first boot of the physical or virtual machine.
3. The apparatus as described in claim 1 wherein the filer automatically re-organizes data by:
enumerating all free disks associated with the physical or virtual machine;
assigning a smallest of the free disks as a copy-on-write (COW) disk; and
performing an n-way stripe over all remaining free disks, using as much of those disks as possible.
4. The apparatus as described in claim 1 wherein the filer also is configured to grow a size of an existing disk in the cache.