Filesystem block sampling to identify user consumption of storage resources
Providing a statistical analysis of all files in a file system based on random sampling of data blocks to identify individual user consumption of file system resources and characteristics of the files stored in the file system. In one or more of the various embodiments, the file system is based on information for a plurality of cylinder groups. Also, each cylinder group may include at one or more known locations at least three types of data structures that enable reverse mapping of data blocks to root directories.
1. A method for managing consumption of data storage resources in a file system, wherein one or more processors execute instructions that perform the method comprising:
instantiating a sampling engine to perform actions, including:
(i) determining a total amount of blocks of data in each of a plurality of cylinder groups in the file system;
(ii) determining a total amount of allocated blocks in the plurality of cylinder groups;
(iii) providing a confidence level based on selecting a defined amount of blocks in the plurality of cylinder groups to be sampled;
(iv) randomly selecting and sampling one block in a cylinder group;
(v) determining a file ID for a file associated with sampled block based on a reverse block map;
(vi) employing an inode tree to find an inode for the file, wherein the file's parent pointer is employed to identify the file's parent directory;
(vii) storing a file name for the file associated with the sampled block;
(viii) when the file ID is non-equivalent to a root node, determining the file's child file in its parent directory and associate the file ID with the parent directory, wherein the method loops returns to performing actions starting at step (iv) again;
(ix) when the file ID is equivalent to the root node and one or more of the selected blocks remain to be sampled, the method loops back to performing starting at step (iv); and
(x) when the file ID is equivalent to the root node and all of the selected blocks have been sampled, performing statistical analysis of the stored file names and sampled blocks.
2. The method of claim 1 , wherein the actions of the sampling engine further comprise:
employing the stored file names to identify sampled blocks.
3. The method of claim 1 , wherein the actions of the sampling engine further comprise identifying one or more users of the files associated with the sampled blocks.
4. The method of claim 1 , wherein the actions of the sampling engine further comprise identifying characteristics of the files associated with the sampled blocks, wherein the characteristics include one or more of: size of file, type of file, author, last user that accessed the file, last time a file was accessed, or other copies of the file.
5. The method of claim 1 , wherein the actions of the sampling engine further comprise employing statistical analysis of sampled blocks and file characteristics to identify data storage resource consumption by identified users of the file system.
6. The method of claim 1 , further comprising instantiating the file system engine to perform actions, including providing reports, alerts, or messages that present information regarding percentage of data storage resource consumption by identified users of the file system.
7. The method of claim 1 , further comprising employing a global positioning systems transceiver to provide geolocation information that is employed to localize information presented to one or more users of the file system.
8. A system for managing consumption of data storage resources in a file system over a network, comprising:
one or more server computers that include:
a memory for storing instructions;
one or more processors, wherein the one or more processors execute the instructions that perform a method comprising:
instantiating a sampling engine to perform actions, including:
(i) determining a total amount of blocks of data in each of a plurality of cylinder groups in the file system;
(ii) determining a total amount of allocated blocks in the plurality of cylinder groups;
(iii) providing a confidence level based on selecting a defined amount of blocks in the plurality of cylinder groups to be sampled;
(iv) randomly selecting and sampling one block in a cylinder group;
(v) determining a file ID for a file associated with sampled block based on a reverse block map;
(vi) employing an inode tree to find an inode for the file, wherein the file's parent pointer is employed to identify the file's parent directory;
(vii) storing a file name for the file associated with the sampled block;
(viii) when the file ID is non-equivalent to a root node, determining the file's child file in its parent directory and associate the file ID with the parent directory, wherein the method loops returns to performing actions starting at step (iv) again;
(ix) when the file ID is equivalent to the root node and one or more of the selected blocks remain to be sampled, the method loops back to performing starting at step (iv); and
(x) when the file ID is equivalent to the root node and all of the selected blocks have been sampled, performing statistical analysis of the stored file names and sampled blocks.
9. The system of claim 8 , wherein the actions of the sampling engine further comprise employing the stored file names to identify sampled blocks.
10. The system of claim 8 , wherein the actions of the sampling engine further comprise identifying one or more users of the files associated with the sampled blocks.
11. The system of claim 8 , wherein the actions of the sampling engine further comprise identifying characteristics of the files associated with the sampled blocks, wherein the characteristics include one or more of: size of file, type of file, author, last user that accessed the file, last time a file was accessed, or other copies of the file.
12. The system of claim 8 , wherein the actions of the sampling engine further comprise employing statistical analysis of sampled blocks and file characteristics to identify data storage resource consumption by identified users of the file system.
13. The system of claim 8 , further comprising instantiating the file system engine to perform actions, including providing reports, alerts, or messages that present information regarding percentage of data storage resource consumption by identified users of the file system.
14. The system of claim 8 , further comprising employing a global positioning systems transceiver to provide geolocation information that is employed to localize information presented to one or more users of the file system.
15. A non-transitory computer readable storage media that includes instructions for managing consumption of data storage resources in a file system, wherein one or more processors execute instructions that perform the method comprising:
instantiating a sampling engine to perform actions, including:
(i) determining a total amount of blocks of data in each of a plurality of cylinder groups in the file system;
(ii) determining a total amount of allocated blocks in the plurality of cylinder groups;
(iii) providing a confidence level based on selecting a defined amount of blocks in the plurality of cylinder groups to be sampled;
(iv) randomly selecting and sampling one block in a cylinder group;
(v) determining a file ID for a file associated with sampled block based on a reverse block map;
(vi) employing an inode tree to find an inode for the file, wherein the file's parent pointer is employed to identify the file's parent directory;
(vii) storing a file name for the file associated with the sampled block;
(viii) when the file ID is non-equivalent to a root node, determining the file's child file in its parent directory and associate the file ID with the parent directory, wherein the method loops returns to performing actions starting at step (iv) again;
(ix) when the file ID is equivalent to the root node and one or more of the selected blocks remain to be sampled, the method loops back to performing starting at step (iv); and
(x) when the file ID is equivalent to the root node and all of the selected blocks have been sampled, performing statistical analysis of the stored file names and sampled blocks.
16. The media of claim 15 , wherein the actions of the sampling engine further comprise:
employing the stored file names to identify sampled blocks.
17. The media of claim 15 , wherein the actions of the sampling engine further comprise identifying one or more users of the files associated with the sampled blocks.
18. The media of claim 15 , wherein the actions of the sampling engine further comprise identifying characteristics of the files associated with the sampled blocks, wherein the characteristics include one or more of: size of file, type of file, author, last user that accessed the file, last time a file was accessed, or other copies of the file.
19. The media of claim 15 , wherein the actions of the sampling engine further comprise employing statistical analysis of sampled blocks and file characteristics to identify data storage resource consumption by identified users of the file system.
20. The media of claim 15 , further comprising instantiating the file system engine to perform actions, including providing reports, alerts, or messages that present information regarding percentage of data storage resource consumption by identified users of the file system.