AI-driven dynamic data exclusion engine for efficient cloud tiering in storage systems

1 . A method comprising: collecting metadata about files present at a plurality of clients, the file metadata for each file comprising a plurality of features;

receiving the file metadata, including values for the plurality of features, at a data protection appliance coupled to the clients;

normalizing the values of the plurality of features, wherein the plurality of features comprises an access frequency, a modification time, a sensitivity level, a retention priority, and a file type;

wherein the access frequency indicates a number of times the file has been accessed over a period of time, the modification time indicates a time when the file was last modified, the sensitivity level indicates a degree of confidentiality that has been attached to content of the file, the retention priority indicates a duration of time for which the file is to be retained, and the file type indicates a format of the file;

calculating, using the normalized values, a centroid for each of the plurality of features associated with the files;

for each file, comparing a normalized value of a feature associated with a file against a respective centroid calculated for the feature;

based on the comparison, when any normalized value of a respective feature associated with the file falls below the respective centroid calculated for the respective feature, not backing up the file from a client at which the file is present;

when each normalized value of each respective feature associated with the file is equal to or above the respective centroid calculated for the respective feature, backing up the file to the data protection appliance; and

applying a machine learning model to files that have been backed up to the data protection appliance to tier the backed up files.

2 . The method of claim 1 further comprising:

training the machine learning model using the file metadata collected at the plurality of clients and classifications of the files, the classifications comprising one of a first or second label attached to each file, wherein the first label attached to a particular file indicates that the particular file is allowed to remain at the data protection appliance, and

wherein the second label attached to the particular file indicates that the particular file is to be moved from the data protection appliance to cloud storage.

3 . The method of claim 1 wherein tiering the files backed up to the data protection appliance comprises allowing some files to remain at the data protection appliance and moving other files from the data protection appliance to cloud storage.

4 . The method of claim 1 wherein the machine learning model comprises a random forest machine learning model.

5 . A system comprising: a processor; and memory configured to store one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of:

collecting metadata about files present at a plurality of clients, the file metadata for each file comprising a plurality of features;

receiving the file metadata, including values for the plurality of features, at a data protection appliance coupled to the clients;

normalizing the values of the plurality of features, wherein the plurality of features comprises an access frequency, a modification time, a sensitivity level, a retention priority, and a file type;

wherein the access frequency indicates a number of times the file has been accessed over a period of time, the modification time indicates a time when the file was last modified, the sensitivity level indicates a degree of confidentiality that has been attached to content of the file, the retention priority indicates a duration of time for which the file is to be retained, and the file type indicates a format of the file;

calculating, using the normalized values, a centroid for each of the plurality of features associated with the files;

for each file, comparing a normalized value of a feature associated with a file against a respective centroid calculated for the feature;

based on the comparison, when any normalized value of a respective feature associated with the file falls below the respective centroid calculated for the respective feature, not backing up the file from a client at which the file is present;

when each normalized value of each respective feature associated with the file is equal to or above the respective centroid calculated for the respective feature, backing up the file to the data protection appliance; and

applying a machine learning model to files that have been backed up to the data protection appliance to tier the backed up files.

6 . The system of claim 5 wherein the processor further carries out the steps of:

training the machine learning model using the file metadata collected at the plurality of clients and classifications of the files, the classifications comprising one of a first or second label attached to each file, wherein the first label attached to a particular file indicates that the particular file is allowed to remain at the data protection appliance, and

wherein the second label attached to the particular file indicates that the particular file is to be moved from the data protection appliance to cloud storage.

7 . The system of claim 5 wherein tiering the files backed up to the data protection appliance comprises allowing some files to remain at the data protection appliance and moving other files from the data protection appliance to cloud storage.

8 . The system of claim 5 wherein the machine learning model comprises a random forest machine learning model.

9 . A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method comprising: collecting metadata about files present at a plurality of clients, the file metadata for each file comprising a plurality of features;

receiving the file metadata, including values for the plurality of features, at a data protection appliance coupled to the clients;

normalizing the values of the plurality of features, wherein the plurality of features comprises an access frequency, a modification time, a sensitivity level, a retention priority, and a file type;

wherein the access frequency indicates a number of times the file has been accessed over a period of time, the modification time indicates a time when the file was last modified, the sensitivity level indicates a degree of confidentiality that has been attached to content of the file, the retention priority indicates a duration of time for which the file is to be retained, and the file type indicates a format of the file;

calculating, using the normalized values, a centroid for each of the plurality of features associated with the files;

for each file, comparing a normalized value of a feature associated with a file against a respective centroid calculated for the feature;

based on the comparison, when any normalized value of a respective feature associated with the file falls below the respective centroid calculated for the respective feature, not backing up the file from a client at which the file is present;

when each normalized value of each respective feature associated with the file is equal to or above the respective centroid calculated for the respective feature, backing up the file to the data protection appliance; and

applying a machine learning model to files that have been backed up to the data protection appliance to tier the backed up files.

10 . The computer program product of claim 9 wherein the method further comprises:

training the machine learning model using the file metadata collected at the plurality of clients and classifications of the files, the classifications comprising one of a first or second label attached to each file, wherein the first label attached to a particular file indicates that the particular file is allowed to remain at the data protection appliance, and

wherein the second label attached to the particular file indicates that the particular file is to be moved from the data protection appliance to cloud storage.

11 . The computer program product of claim 9 wherein tiering the files backed up to the data protection appliance comprises allowing some files to remain at the data protection appliance and moving other files from the data protection appliance to cloud storage.

12 . The computer program product of claim 9 wherein the machine learning model comprises a random forest machine learning model.