Synthesizing full content using file extents without namespace impact
Improving extent-based read performance using pre-fetches by utilizing synthesized sequential extent files in a deduplication storage system. Synthesized content for the extents is created on a server. The synthesized content is headed by a content handle and stored in a stateful context data structure for lookup by the client. This enables an extent read workflow to be supported on a storage system without an added cache required to achieve sufficient random read performance by synthesizing content into a inode-less file. It improves extent reads at random offsets by synthesizing the extents to leverage the sequential restore performance, and leverages delta files to improve read performance for extent reads.
1 . A computer-implemented method for improving extent-based read performance of a file using data pre-fetches in a client-server network, comprising:
accessing backup files represented by a Merkle tree comprising file data as variable sized segments grouped in hierarchical levels from a bottom level to a top level;
obtaining, by a hardware-based prefetch processing component, extent information for delta changes between a first generation backup file and a second generation backup file;
receiving, through a network interface in a filesystem server and from a client, an instruction to create a new synthesized file corresponding to the extent information, wherein the extent information comprises a sequence of offsets and lengths, with each offset and length pair defining a corresponding extent of data added to the first generation backup file to synthesize the second generation backup file;
first creating, by the hardware-based prefetch processing component, upon receiving this instruction, a new synthesized file comprising extents of the extent information as contiguous data that can be read sequentially to leverage the benefits of the pre-fetches, wherein the pre-fetches move data from the extents into a read-ahead cache to be sent to an application of the client in response to a read request, and further wherein a prefetch generated by a pre-fetch request comprises a hint that a read input/output (I/O) operation is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation for the read request;
second creating a new Merkle tree to host synthesized contents for the extent information;
heading the synthesized file by a content handle that is pointed to by a file handle that identifies the synthesized file, wherein the content handle is stored outside of the namespace for the synthesized file in a stateful context data structure, and that used to read the content of the synthesized file without having an entry in the namespace, thus effectively comprising an inode-less file that does not impact the namespace;
maintaining a server-side state with the content handle for use when reading data of the file through a read operation;
storing, in physical storage media in the filesystem server, the synthesized file in a stateful context data structure, which can be looked up by the client through accessing the content handle from the stateful context; and
destroying, after the read operation is complete, the server-side state.
2 . The method of claim 1 wherein the benefits of the pre-fetches comprise at least one of: preventing wasted input/output operations created by attempting to pre-fetch data beyond an end of an extent, or failing to pre-fetch any data at a beginning of an extent.
3 . The method of claim 2 wherein the sequence of offsets comprise an extent map, with each offset defining a corresponding extent.
4 . The method of claim 1 wherein the storage comprises part of a deduplication backup process executed by a data storage server running a Data Domain filesystem (DDFS), and wherein the client comprises a DDBoost client, and further wherein the stateful context data structure can be looked up using a DDBoost connection.
5 . A computer-implemented method for improving read performance of a file using data pre-fetches in a client-server network, comprising:
accessing backup files represented by a Merkle tree comprising file data as variable sized segments grouped in hierarchical levels from a bottom level to a top level;
first creating, by a hardware-based prefetch processing component, a synthesized file comprising extents of extent information for delta changes between a first generation backup file and a second generation backup file organized as contiguous data that can be read sequentially to leverage the benefits of the pre-fetches, wherein the extent information comprises a sequence of offsets and lengths, with each offset and length pair defining a corresponding extent of data added to the first generation backup file to synthesize the second generation backup file, and further wherein the pre-fetches move data from the extents into a read-ahead cache to be sent to an application of the client in response to a read request, and further wherein a prefetch generated by a pre-fetch request comprises a hint that a read input/output (I/O) operation is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation for the read request;
second creating a new Merkle tree to host synthesized contents for the extent information;
heading the synthesized file by a content handle that is pointed to by a file handle that identifies the synthesized file, wherein the content handle is stored outside of the namespace for the synthesized file in a stateful context data structure, and that used to read the content of the synthesized file without having an entry in the namespace, thus effectively comprising an inode-less file that does not impact the namespace;
maintaining a server-side state with the content handle for use when reading data of the file through a read operation;
storing, in physical storage media in a filesystem server, the synthesized file in a stateful context data structure, which can be looked up by a client when reading data associated with the synthesized file through accessing the content handle from the stateful context; and
destroying, upon completion of the reading, the server-side content handle.
6 . The method of claim 5 wherein the benefits of the pre-fetches comprise at least one of: preventing wasted input/output operations created by attempting to pre-fetch data beyond an end of an extent, or failing to pre-fetch any data at a beginning of an extent.
7 . The method of claim 6 wherein the sequence of offsets comprise an extent map, with each offset defining a corresponding extent.
8 . The method of claim 7 wherein the server and client comprise part of a deduplication backup process executed by a data storage server running a Data Domain filesystem (DDFS), and wherein the client comprises a DDBoost client, and further wherein the stateful context data structure can be looked up using a DDBoost connection.
9 . A system for improving read performance of a file using data pre-fetches in a client-server network, comprising:
a server hosting a filesystem storing data in storage for an application executed in the network, the data represented by a Merkle tree comprising file data as variable sized segments grouped in hierarchical levels from a bottom level to a top level;
a network client hosting the application;
a hardware-based backup processing component obtaining extent information for delta changes between a first generation backup file and a second generation backup file;
a hardware-based server component receiving, from the network client, an instruction to create a new synthesized file corresponding to the extent information, and first creating, upon receiving this instruction, a new synthesized file comprising extents of the extent information as contiguous data that can be read sequentially to leverage the benefits of the pre-fetches, and second creating a new Merkle tree to host synthesized contents for the extent information, wherein the synthesized file is headed by a content handle that is pointed to by a file handle that identifies the synthesized file, wherein the content handle is stored outside of the namespace for the synthesized file in a stateful context data structure, and that used to read the content of the synthesized file without having an entry in the namespace, thus effectively comprising an inode-less file that does not impact the namespace, the component further maintaining a server-side state with the content handle for use when reading data of the file through a read operation, wherein the extent information comprises a sequence of offsets and lengths, with each offset and length pair defining a corresponding extent of data added to the first generation backup file to synthesize the second generation backup file, and further wherein the pre-fetches move data from the extents into a read-ahead cache to be sent to an application of the client in response to a read request, and further wherein a prefetch generated by a pre-fetch request comprises a hint that a read input/output (I/O) operation is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation for the read request; and
a physical storage of the server storing the synthesized file in a stateful context data structure, which can be looked up by a client when reading data associated with the synthesized file through accessing the content handle from the stateful context, wherein, after the read operation is complete, the server-side state is destroyed.
10 . The system of claim 9 wherein the storage comprises part of a deduplication backup process executed by a data storage server running a Data Domain filesystem (DDFS), and wherein the client comprises a DDBoost client, and further wherein the stateful context data structure can be looked up using a DDBoost connection.