Artificial intelligence-based storage monitoring
Techniques are provided for artificial intelligence-based storage monitoring. In an example, a system determines structured and unstructured attributes of a folder in a file system and provides them to an trained artificial intelligence model that outputs whether the folder is interesting or not. The folders labelled interesting by the trained artificial intelligence model can be further refined to a subset of folders that are placed in a watch list, and monitored for changes.
1. A system, comprising:
a processor; and
a non-transitory memory that stores executable instructions that, when executed by the first processor, facilitate performance of operations, comprising:
determining a first attribute of a first folder in a computer file system;
converting the first attribute into a first vector of attributes;
outputting the first vector of attributes to an artificial intelligence classifier that outputs a first classification of the first folder, the first classification indicating that the first folder is assigned a first importance value; and
in response to determining that the first importance value is above a first defined threshold for the first folder, monitoring the first folder for changes.
2. The system of claim 1 , wherein the monitoring the first folder comprises including the first folder in a first watch group, and wherein the operations further comprise:
monitoring activity of a second folder that is a member of a second watch group, the second watch group being determined previously to determining the first watch group; and
including the second folder in the first watch group based on the monitoring activity of the second folder.
3. The system of claim 2 , wherein the activity of the second folder comprises a throughput relating to the second folder, an input/output operations per second relating to the second folder, or a latency of accessing data relating to the second folder.
4. The system of claim 2 , wherein the including the second folder in the first watch group comprises increasing a second importance value of the second folder, resulting in an increased second importance value, the increased second importance value being determined to be above a second defined threshold value.
5. The system of claim 2 , wherein the including the second folder in the first watch group comprises decreasing a second defined threshold value, resulting in a decreased second threshold value, a second importance value associated with the second folder being determined to be above the decreased second defined threshold value.
6. The system of claim 1 , wherein the monitoring the folder comprises including the first folder in a first watch group, and wherein the operations further comprise:
performing iterations of updating the first watch group at a defined time interval.
7. The system of claim 1 , further comprising:
training the artificial intelligence classifier with labeled training data, the labeled training data comprising historical folders and corresponding indications of whether each historical folder was monitored for changes.
8. A method, comprising:
converting, by a system comprising a processor, a first attribute of a first folder in a file system into a first vector of attributes;
providing, by the system, the first vector of attributes as input to an artificial intelligence classifier that outputs a first classification of the first folder, the first classification indicating that the first folder has a first importance value; and
in response to determining that the first importance value is above a first defined threshold for the first folder, monitoring, by the system, the first folder for changes.
9. The method of claim 8 , wherein the first folder belongs to a set of candidate folders, and further comprising:
reducing, by the system, the set of candidate folders to a set of watch list folders, a first number of folders in the set of candidate folders being larger than a second number of folders in the set of watch list folders; and
monitoring, by the system, the set of watch list folders for changes.
10. The method of claim 9 , wherein the reducing the set of candidate folders to the set of watch list folders comprises:
smoothing, by the system, changes to the set of watch list folders relative to a set of previous watch list folders.
11. The method of claim 9 , wherein the reducing the set of candidate folders to the set of watch list folders comprises:
implementing, by the system, a delay between the generating the set of watch list folders and the monitoring the set of watch list folders.
12. The method of claim 9 , further comprising:
monitoring, by the system, a second folder for changes in response to determining that the second folder is included in a defined number of iterations of the set of watch list folders.
13. The method of claim 8 , wherein the first attribute comprises at least a first structured attribute and a first unstructured attribute.
14. The method of claim 13 , wherein the first structured attribute comprises a numerical statistic of the folder.
15. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:
sending a first vector of attributes of a first folder in a file system to an artificial intelligence classifier that outputs a first classification of the first folder, the first classification indicating that the first folder has a first importance value; and
in response to determining that the first importance value is above a first predetermined threshold for the first folder, monitoring the first folder for changes.
16. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:
reducing a set of candidate folders to a set of watch group folders, a first number of folders in the set of candidate folders being larger than a second number of folders in the set of watch group folders; and
monitoring the set of watch group folders for changes.
17. The non-transitory computer-readable medium of claim 16 , wherein the reducing the set of candidate folders to the set of watch group folders comprises:
omitting a second folder that is in the set of candidate folders from the set of watch group folders in response to determining that the second folder and a third folder of the set of watch group folders have a respective file path that overlaps by at least a second threshold value.
18. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise:
determining to perform the omitting the second folder in response to determining that a first difference between a second importance value of the second folder and a second predetermined threshold of the second folder is less than a third importance value of the third folder and a third predetermined threshold of the third folder.
19. The non-transitory computer-readable medium of claim 15 , wherein the first attribute comprises an unstructured attribute, the unstructured attribute comprising a text string corresponding to the first folder.
20. The non-transitory computer-readable medium of claim 19 , wherein the text string corresponding to the first folder comprises a path name of the first folder.