CONCURRENT TRANSMISSION OF MULTIPLE EXTENTS DURING BACKUP OF EXTENT-ELIGIBLE FILES
An information management system supports the concurrent transfer of multiple extents from a client computing device having one or more files stored in a primary storage device. The client computing device determines whether one or more of the files are extent-eligible by comparing a file size of a file with a predetermine file size threshold. Based on the comparison, the client computing device determines that the file is extent-eligible, and determines a plurality of extents for the extent-eligible file. The client computing device may then concurrently transmit one or more of the extents of the extent-eligible file to one or more nodes in communication with the client computing device. The one or more nodes may index the received extents for later retrieval during a restore operation of the extent-eligible file. The one or more nodes may also validate the extents to ensure that the extents are not corrupted.
1 . A method for backing up file system data as a plurality of extents, the method comprising:
receiving, at a data agent being executed on a client computing device, an instruction to perform a backup of file system data of the client computing device, wherein the file system data is stored in a plurality of files;
determining whether a file from the plurality of files meets or exceeds a predetermined file size threshold;
determining a plurality of extents for the file based on a determination that the file meets or exceeds the predetermined file threshold, wherein each extent includes a portion of the data for the file and less than all of the data for the file;
transmitting a list of the plurality of extents for the file to a coordinating worker node selected from a plurality of worker nodes, wherein the coordinating worker node coordinates backup and restoration operations among the plurality of worker nodes in communication with the client computing device;
receiving an instruction from the storage manager to transmit the plurality of extents to the plurality of worker nodes; and
transmitting the plurality of extents to the plurality of worker nodes, wherein at least a first extent of the plurality of extents is transmitted to a first worker node and at least a second extent of the plurality of extents is transmitted to a second worker node.
2 . The method of claim 1 , further comprising:
creating a volume snapshot of a primary data storage device of the client computing device; and wherein the determination of the plurality of extents of the file is performed on the file created from the volume snapshot.
3 . The method of claim 1 , further comprising:
determining whether the file has changed during the transmission of the plurality of extents of the file; and
in response to a determination that the file has changed:
stopping the transmission of the plurality of extents; and
identifying that the file is to be backed up when a subsequent request is received to perform a backup of the file system data.
4 . The method of claim 3 , wherein the determination of the whether the file has changed is performed by:
comparing a first time at which the file changed with a second time at which the file changed; and
determining that the file has changed based on a difference in the first time with the second time.
5 . The method of claim 1 , further comprising:
receiving a failure message that at least one extent of the plurality of extents has failed; and
identifying that the file is to be backed up when a subsequent request is received to perform a backup of the file system data in response to the received failure message.
6 . The method of claim 1 , further comprising:
receiving the plurality of extents from the plurality of worker nodes during a restore operation, wherein the first extent is received from the first worker node and the second extent is received from the second worker node; and
reconstructing the file from the plurality of extents.
7 . The method of claim 1 , wherein:
a size of each extent from the plurality of extents is less than the predetermined file size threshold.
8 . The method of claim 1 , wherein the plurality of extents is transmitted substantially concurrent to corresponding worker nodes of the plurality of worker nodes.
9 . The method of claim 1 , wherein at least one worker node of the plurality of worker nodes comprises a media agent configured to update a media agent index based on a corresponding extent of the plurality of extents, wherein the media agent index comprises information for restoring the corresponding extent from a secondary storage device in communication with the media agent.
10 . The method of claim 1 , wherein the same extent selected from the plurality of extents is transmitted to at least two different worker nodes selected from the plurality of worker nodes.
11 . A system for backing up file system data as a plurality of extents, the system comprising:
one or more non-transitory, computer-readable medium having computer-executable instructions stored thereon; and
one or more processors that, having executed the computer-executable instructions, configure the system to perform a plurality of operations comprising:
receiving, at a data agent being executed on a client computing device, an instruction to perform a backup of file system data of the client computing device, wherein the file system data is stored in a plurality of files;
determining whether a file from the plurality of files meets or exceeds a predetermined file size threshold;
determining a plurality of extents for the file based on a determination that the file meets or exceeds the predetermined file threshold, wherein each extent includes a portion of the data for the file and less than all of the data for the file;
transmitting a list of the plurality of extents for the file to a coordinating worker node selected from a plurality of worker nodes, wherein the coordinating worker node coordinates backup and restoration operations among the plurality of worker nodes in communication with the client computing device;
receiving an instruction from the storage manager to transmit the plurality of extents to the plurality of worker nodes; and
transmitting the plurality of extents to the plurality of worker nodes, wherein at least a first extent of the plurality of extents is transmitted to a first worker node and at least a second extent of the plurality of extents is transmitted to a second worker node.
12 . The system of claim 11 , wherein the plurality of operations further comprises:
creating a volume snapshot of a primary data storage device of the client computing device; and wherein
the determination of the plurality of extents of the file is performed on the file created from the volume snapshot.
13 . The system of claim 11 , wherein the plurality of operations further comprises:
determining whether the file has changed during the transmission of the plurality of extents of the file; and
in response to a determination that the file has changed:
stopping the transmission of the plurality of extents; and
identifying that the file is to be backed up when a subsequent request is received to perform a backup of the file system data.
14 . The system of claim 13 , wherein the determination of the whether the file has changed is performed by:
comparing a first time at which the file changed with a second time at which the file changed; and
determining that the file has changed based on a difference in the first time with the second time.
15 . The system of claim 11 , wherein the plurality of operations further comprises:
receiving a failure message that at least one extent of the plurality of extents has failed; and
identifying that the file is to be backed up when a subsequent request is received to perform a backup of the file system data in response to the received failure message.
16 . The system of claim 11 , wherein the plurality of operations further comprises:
receiving the plurality of extents from the plurality of worker nodes during a restore operation, wherein the first extent is received from the first worker node and the second extent is received from the second worker node; and
reconstructing the file from the plurality of extents.
17 . The system of claim 11 , wherein:
a size of each extent from the plurality of extents is less than the predetermined file size threshold.
18 . The system of claim 11 , wherein the plurality of extents is transmitted substantially concurrent to corresponding worker nodes of the plurality of worker nodes.
19 . The system of claim 11 , wherein at least one worker node of the plurality of worker nodes comprises a media agent configured to update a media agent index based on a corresponding extent of the plurality of extents, wherein the media agent index comprises information for restoring the corresponding extent from a secondary storage device in communication with the media agent.
20 . The system of claim 11 , wherein the same extent selected from the plurality of extents is transmitted to at least two different worker nodes selected from the plurality of worker nodes.