Managing files according to categories
According to various embodiments, with respect to a target set of files being managed (e.g., protected by data snapshots), each file in the target set of files is classified into one of two or more filesets (discontiguous filesets), where each of these filesets comprises one or more files that are related to each other by one or more factors, such as frequency of file change or purpose of existence (e.g., used by a software application). Once classified, files within the target set of files can be uniquely processed by a data management operation (e.g., incremental data snapshot process) according to their association to a discontiguous fileset.
1. A method comprising:
generating, by one or more hardware processors, a plurality of data snapshots of a target set of files stored on a filesystem of a remote computing device, the plurality of data snapshots comprising at least one incremental data snapshot;
periodically classifying, by the one or more hardware processors, files in the target set of files to one of a plurality of fileset classes based on a set of file ranks for the target set of files, the set of file ranks being determined based on a set of ranking factors that comprises a frequency of file data change over prior data snapshots, individual fileset classes of the plurality of fileset classes being associated with different time intervals in a plurality of snapshot generation time intervals that determine frequencies at which data snapshots are generated with respect to files associated with the individual fileset classes; and
for individual fileset classes of the plurality of fileset classes, causing, by the one or more hardware processors, periodic generation of an incremental data snapshot for a given subset of files of the target set of files according to a given snapshot generation time interval from the plurality of snapshot generation time intervals, the given subset of files and the given snapshot generation time interval being associated with the individual fileset class, and the causing periodic generation of the incremental data snapshot for the given subset of files comprises, for individual files in the given subset of files:
obtaining, from the remote computing device, metadata for the individual file;
in response to obtaining the metadata, determining, based on the metadata, whether the individual file changed since a last data snapshot; and
obtaining, from the remote computing device, at least changed data for the individual file responsive to determining that the individual file changed since the last data snapshot.
2. The method of claim 1 , wherein the periodically classifying each file in the target set of files to one of the plurality of fileset classes based on the set of file ranks for the target set of files further comprises:
associating each individual file in the target set of files to a given fileset class from the plurality of fileset classes based on a respective file rank from the set of file ranks for the individual file and based on a given range of ranks associated with the given fileset class, each fileset class of the plurality of fileset classes being associated with a different range of ranks.
3. The method of claim 1 , wherein the set of ranking factors comprises an association of a given file with a software application.
4. The method of claim 1 , wherein the at least changed data comprises at least one of a set of changed data blocks of the individual file or changed file metadata of the individual file.
5. The method of claim 1 , wherein the obtaining, from the remote computing device, the metadata for the individual file comprises:
obtaining, from a remote agent installed on the remote computing device, the metadata for the individual file.
6. The method of claim 1 , wherein the obtaining, from the remote computing device, the metadata for the individual file comprises:
sending a set of instructions to the remote computing device that causes the remote computing device to perform a stat( ) system call with respect to the individual file.
7. The method of claim 1 , wherein the obtaining, from the remote computing device, the at least changed data for the individual file comprises:
obtaining, from a remote agent installed on the remote computing device, the at least changed data for the individual file.
8. The method of claim 1 , wherein the set of rankin factors comprises an association of a given file with a software application and, wherein the software application is associated with a significance rank.
9. The method of claim 8 , wherein the significance rank is user-defined.
10. The method of claim 1 , wherein the causing periodic generation of the incremental data snapshot for the given subset of files further comprises:
adjusting policy data that determines periodic generation of the incremental data snapshot for the given subset of files.
11. The method of claim 1 , wherein each file in the target set of files is initially associated with a same fileset class from the plurality of fileset classes.
12. A system comprising:
a memory storing instructions; and
one or more hardware processors communicatively coupled to the memory and configured by the instructions to perform operations comprising:
generating a plurality of data snapshots of a target set of files stored on a filesystem of a remote computing device, the plurality of data snapshots comprising at least one incremental data snapshot;
periodically classifying files in the target set of files to one of a plurality of fileset classes based on a set of file ranks for the target set of files. the set of file ranks being determined based on a set of ranking factors that comprises a frequency of file data change over prior data snapshots, individual fileset classes of the plurality of fileset classes being associated with different time intervals in a plurality of snapshot generation time intervals that determine frequencies at which data snapshots are generated with respect to files associated with the individual fileset classes; and
for individual fileset classes of the plurality of fileset classes, causing periodic generation of an incremental data snapshot for a given subset of files of the target set of files according to a given snapshot generation time interval from the plurality of snapshot generation time intervals, the given subset of files and the given snapshot generation time interval being associated with the individual fileset class, and the causing periodic generation of the incremental data snapshot for the given subset of files comprises, for individual files in the given subset of files:
obtaining, from the remote computing device, metadata for he individual file;
in response to obtaining the metadata, determining, based on the metadata, whether the individual file changed since a last data snapshot; and
obtaining, from the remote computing device, at least changed data for the individual file responsive to determining that the individual file changed since the last data snapshot.
13. The system of claim 12 , wherein the at least changed data comprises at least one of a set of changed data blocks of the individual file or changed file metadata of the individual file.
14. The system of claim 12 , wherein the obtaining, from the remote computing device, the metadata for the individual file comprises:
obtaining, from a remote agent installed on the remote computing device, the metadata for the individual file.
15. The system of claim 12 , wherein the obtaining, from the remote computing device, the metadata for the individual file comprises:
sending a set of instructions to the remote computing device that causes the remote computing device to perform a stat( ) system call with respect to the individual file.
16. The system of claim 12 , wherein the obtaining, from the remote computing device, the at least changed data for the individual file comprises:
obtaining, from a remote agent installed on the remote computing device, the at least changed data for the individual file.
17. The system of claim 12 , wherein the causing periodic generation of the incremental data snapshot for the given subset of files further comprises:
adjusting policy data that determines periodic generation of the incremental data snapshot for the given subset of files.
18. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
generating a plurality of data snapshots of a target set of files stored on a filesystem of a remote computing device, the plurality of data snapshots comprising at least one incremental data snapshot;
periodically classifying files in the target set of files to one of a plurality of fileset classes based on a set of file ranks for the target set of files, the set of file ranks being determined based on a set of ranking factors that comprises a frequency of file data change over prior data snapshots, individual fileset classes of the plurality of fileset classes being associated with different time intervals in a plurality of snapshot generation time intervals that determine frequencies at which data snapshots are generated with respect to files associated with the individual fileset classes; and
for individual fileset classes of the plurality of fileset classes, causing periodic generation of an incremental data snapshot for a given subset of files of the target set of files according to a given snapshot generation time interval from the plurality of snapshot generation time intervals, the given subset of files and the given snapshot generation time interval being associated with the individual fileset class, and the causing periodic generation of the incremental data snapshot for the given subset of files comprises, for individual files in the given subset of files:
obtaining, from the remote computing device, metadata for the individual file;
in response to obtaining the metadata, determining, based on the metadata, whether the individual file changed since a last data snapshot; and
obtaining, from the remote computing device, at least changed data for the individual file responsive to determining that the individual file changed since the last data snapshot.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the periodically classifying each file in the target set of files to one of the plurality of fileset classes based on the set of file ranks for the target set of files further comprises:
associating each individual file in the target set of files to a given fileset class from the plurality of fileset classes based on a respective file rank from the set of file ranks for the individual file and based on a given range of ranks associated with the given fileset class, each fileset class of the plurality of fileset classes being associated with a different range of ranks.
20. The non-transitory computer-readable storage medium of claim 18 , wherein the set of ranking factors comprises an association of a given file with a software application.