Managing files according to categories
According to various embodiments, with respect to a target set of files being managed (e.g., protected by data snapshots), each file in the target set of files is classified into one of two or more filesets (discontiguous filesets), where each of these filesets comprises one or more files that are related to each other by one or more factors, such as frequency of file change or purpose of existence (e.g., used by a software application). Once classified, files within the target set of files can be uniquely processed by a data management operation (e.g., incremental data snapshot process) according to their association to a discontiguous fileset.
1. A method comprising:
determining, by one or more hardware processors, whether a plurality of existing data snapshots comprises a minimal number of data snapshots for a target set of files; and
after determining that the plurality of existing data snapshots comprises the minimal number of data snapshots:
periodically classifying the target set of files, by the one or more hardware processors, by periodically:
determining a rank of each file in the target set of files to generate a set of file ranks for the target set of files; and
associating each file in the target set of files to a given fileset class from a plurality of fileset classes based on a respective file rank from the set of file ranks and based on a given range of ranks associated with the given fileset class, each fileset class of the plurality of fileset classes being associated with a different time interval in a plurality of snapshot generation time intervals and each fileset class of the plurality of fileset classes being associated with a different range of ranks; and
periodically generating, by the one or more hardware processors, incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files, the periodically generating of the incremental data snapshots comprising storing delta data for each file in the target set of files that has experienced a data block change or a file metadata change.
2. The method of claim 1 , further comprising:
generating, by the one or more hardware processors, the plurality of existing data snapshots of the target set of files stored on a file system over time according to a defined snapshot generation time interval.
3. The method of claim 1 , wherein the determining of the rank of each file in the target set of files to generate the set of file ranks comprises:
ranking each file in the target set of files based on a set of ranking factors, the set of ranking factors comprising a frequency of file data change.
4. The method of claim 1 , wherein the periodically generating the incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files comprises:
for each individual fileset class of the plurality of fileset classes, causing periodic generation of an incremental data snapshot for a given subset of files of the target set of files according to a given snapshot generation time interval from the plurality of snapshot generation time intervals, the given subset of files and the given snapshot generation time interval being associated with the individual fileset class.
5. The method of claim 1 , wherein the periodically generating the incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files comprises:
adjusting, based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files, policy data that determines periodic generation of incremental data snapshots for the target set of files.
6. The method of claim 1 , wherein the plurality of fileset classes comprises a first fileset class, a second fileset class, and a third fileset class, wherein the plurality of snapshot generation time intervals comprises a first snapshot generation time interval associated with the first fileset class, a second snapshot generation time interval associated with the second fileset class, and a third snapshot generation time interval associated with the third fileset class, wherein the first snapshot generation time interval is greater than the second snapshot generation time interval, and wherein the second snapshot generation time interval is greater than the third snapshot generation time interval.
7. The method of claim 6 , further comprising:
generating, by the one or more hardware processors, a plurality of data snapshots of the target set of files stored on a file system over time according to the second snapshot generation time interval.
8. The method of claim 1 , wherein the periodically classifying each file in the target set of files is performed after generation of a new data snapshot.
9. The method of claim 8 , wherein the new data snapshot comprises a new incremental data snapshot.
10. The method of claim 1 , wherein each file in the target set of files is initially associated with a same fileset class from the plurality of fileset classes.
11. A system comprising:
a memory storing instructions; and
one or more hardware processors communicatively coupled to the memory and configured by the instructions to perform operations comprising:
determining whether a plurality of existing data snapshots comprises a minimal number of data snapshots for a target set of files; and
after determining that the plurality of existing data snapshots comprises the minimal number of data snapshots:
periodically classifying the target set of files by periodically:
determining a rank of each file in the target set of files to generate a set of file ranks for the target set of files; and
associating each file in the target set of files to a given fileset class from a plurality of fileset classes based on a respective file rank from the set of file ranks and based on a given range of ranks associated with the given fileset class, each fileset class of the plurality of fileset classes being associated with a different time interval in a plurality of snapshot generation time intervals and each fileset class of the plurality of fileset classes being associated with a different range of ranks; and
periodically generating incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files, the periodically generating of the incremental data snapshots comprising storing delta data for each file in the target set of files that has experienced a data block change or a file metadata change.
12. The system of claim 11 , wherein the operations further comprise:
generating the plurality of existing data snapshots of the target set of files stored on a file system over time according to a defined snapshot generation time interval.
13. The system of claim 11 , wherein the determining of the rank of each file in the target set of files to generate the set of file ranks comprises:
ranking each file in the target set of files based on a set of ranking factors, the set of ranking factors comprising a frequency of file data change.
14. The system of claim 11 , wherein the periodically generating the incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files comprises:
for each individual fileset class of the plurality of fileset classes, causing periodic generation of an incremental data snapshot for a given subset of files of the target set of files according to a given snapshot generation time interval from the plurality of snapshot generation time intervals, the given subset of files and the given snapshot generation time interval being associated with the individual fileset class.
15. The system of claim 11 , wherein the periodically generating the incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files comprises:
adjusting, based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files, policy data that determines periodic generation of incremental data snapshots for the target set of files.
16. The system of claim 11 , wherein the plurality of fileset classes comprises a first fileset class, a second fileset class, and a third fileset class, wherein the plurality of snapshot generation time intervals comprises a first snapshot generation time interval associated with the first fileset class, a second snapshot generation time interval associated with the second fileset class, and a third snapshot generation time interval associated with the third fileset class, wherein the first snapshot generation time interval is greater than the second snapshot generation time interval, and wherein the second snapshot generation time interval is greater than the third snapshot generation time interval.
17. The system of claim 11 , wherein the periodically classifying each file in the target set of files is performed after generation of a new data snapshot.
18. The system of claim 11 , wherein each file in the target set of files is initially associated with a same fileset class from the plurality of fileset classes.
19. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
determining whether a plurality of existing data snapshots comprises a minimal number of data snapshots for a target set of files; and
after determining that the plurality of existing data snapshots comprises the minimal number of data snapshots:
periodically classifying the target set of files by periodically:
determining a rank of each file in the target set of files to generate a set of file ranks for the target set of files; and
associating each file in the target set of files to a given fileset class from a plurality of fileset classes based on a respective file rank from the set of file ranks and based on a given range of ranks associated with the given fileset class, each fileset class of the plurality of fileset classes being associated with a different time interval in a plurality of snapshot generation time intervals and each fileset class of the plurality of fileset classes being associated with a different range of ranks; and
periodically generating incremental data snapshots for the target set of files based on the plurality of fileset classes and according to the plurality of snapshot generation time intervals as associated with the target set of files, the periodically generating of the incremental data snapshots comprising storing delta data for each file in the target set of files that has experienced a data block change or a file metadata change.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the determining of the rank of each file in the target set of files to generate the set of file ranks comprises:
ranking each file in the target set of files based on a set of ranking factors, the set of ranking factors comprising a frequency of file data change.