Sequential file restore performance using filesystem redirection
A prefetch mechanism stores data in the memory allocated in a remote passive (remote) node in a High Availability active-passive node pair. A read-ahead cache that can host different parts of a file is created on both the active and the passive nodes. The system improves the performance of sequential restores by using memory available on a passive node to supplement the local (active node) memory to hold the prefetched data for the common case of large sequential restores. This extra memory overcomes issues when a current prefetch fails to keep up with the target restore performance. Filesystem redirection is thus accomplished by servicing an I/O request by sending it another node that itself caches the data.
1. A computer-implemented method of restoring files in a data backup system, comprising:
receiving a restore request;
determining that the received restore request comprises a long sequential read and not small file-level restores;
setting up a local read-ahead cache on an active node and a remote read-ahead cache on a passive node coupled to the active node through a fast interconnect link;
defining an index relating a file offset to memory storing file data and containing an indication of a location of the memory as stored locally or remotely;
receiving a prefetch request to aid retrieval of file data restored from a storage system, wherein the prefetch request issues after a read request and comprises a prefetch request issued at a prefetch horizon on the order of megabytes in size from a read offset, and wherein the prefetch horizon covers an amount of time between stalls of input/output (I/O) operations of reads from the memory and slower reads from disk, and wherein the time between the stalls comprises a read highway;
first determining if the read request accesses data contained in the index, and second determining if the indexed data is stored locally or remotely; and
accessing the read data from the local read-ahead cache if stored locally or accessing the read data from the remote cache if stored remotely to effectively lengthen the read highway to thereby reduce the stalls of I/O operations.
2. The computer-implemented method of claim 1 wherein prior to first determining if the read request accesses data contained in the index, determining that the file data is of a size exceeding a minimum size threshold.
3. The computer-implemented method of claim 2 wherein prior to first determining if the read request accesses data contained in the index, determining that the read request accesses the file data in a minimally sequential access pattern.
4. The computer-implemented method of claim 1 wherein each of the local read-ahead cache and the remote read-ahead cache comprises a pool of pages that are uniquely addressable.
5. The computer-implemented method of claim 4 wherein the pages of the pool are used to track memory pool in a filesystem of the storage system.
6. The computer-implemented method of claim 5 wherein the index comprises a hash table storing a relationship of a file offset to a page number for the file being read by the read request.
7. The computer-implemented method of claim 1 wherein the prefetch request prefetches at least some data of the file into a system read-ahead cache comprising both the local and remote read-ahead caches.
8. The computer-implemented method of claim 7 wherein the prefetch request comprises a prefetch operation including a hint to the filesystem allowing lower layers of the filesystem stack to read-ahead the portion of the file stored in the system read-ahead cache.
9. The computer-implemented method of claim 8 wherein the restore process is implemented using a Multi-Streamed Restore (MSR) process that opens up a plurality of internal streams for the read operation and uses the system read ahead cache to serve user the I/O operations.
10. The computer-implemented method of claim 9 wherein the storage system comprises part of a deduplication backup process executed by a data storage server running a Data Domain File System (DDFS).