De-duplication of access control lists
Described are methods, systems, and apparatus, including computer program products for de-duplicating access control lists (ACLs). A first ACL associated with a first computer file is received. A first checksum based at least in part on the first ACL is calculated. One or more directory entries based on the first checksum are retrieved from a de-duplication directory, wherein each directory entry of the one or more directory entries comprises a reference to an ACL and a name comprising the first checksum. A directory entry of the one or more directory entries is identified that references a second ACL that specifies the same permissions as the first ACL. A reference to the second ACL of the directory entry is added to the first computer file.
1. A storage system-implemented method for de-duplicating access control lists comprising:
a. receiving, at the storage system from a client, a first computer file including a first access control list (ACL) identifier identifying first ACL data associated with the first computer file, the first ACL data specifying permissions for the first computer file, the storage system including an ACL database and a de-duplication directory, the de-duplication directory having a plurality of directory entries, each directory entry including a reference that references ACL data stored in the ACL database, and each directory entry having a name that corresponds to a checksum calculated based at least in part on the referenced ACL data;
b. calculating, by the storage system, a first checksum based at least in part on the first ACL data identified by the first ACL identifier included in the first computer file;
c. determining, by the storage system, whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on (i) a respective directory entry of the de-duplication directory having a name that corresponds to the first checksum, (ii) the reference included in the respective directory entry referencing second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and (iii) one or more references included in one or more of the plurality of directory entries, respectively, referencing the second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and wherein the one or more of the plurality of directory entries comprise a first directory entry having a first name corresponding to the first checksum, and a second directory entry having a second name corresponding to the first checksum with duplicate indicia appended;
d. de-duplicating the ACL database by the storage system, the de-duplicating of the ACL database including:
having determined that the permissions specified by the first ACL data are not previously stored in the ACL database, storing the first ACL data in the ACL database; and
having determined that the permissions specified by the first ACL data are previously stored in the ACL database, updating the first ACL identifier included in the first computer file to identify the second ACL data stored in the ACL database, and avoiding storing the first ACL data in the ACL database; and
e. incrementing, by the storage system, a reference count associated with the second ACL data.
2. The method of claim 1 , further comprising storing, by the storage system, the first computer file.
3. The method of claim 1 , further comprising:
retrieving the second ACL data from the ACL database stored on the storage system.
4. The method of claim 1 , wherein the determining of whether or not the permissions specified by the first ACL data are previously stored in the ACL database is further based on the one or more of the plurality of directory entries of the de-duplication directory having one or more names, respectively, corresponding to the first checksum.
5. A storage system comprising:
a storage processor providing a file system, the storage processor configured to:
i. receive, from a client, a first computer file including a first access control list (ACL) identifier identifying first ACL data associated with the first computer file, the first ACL data specifying permissions for the first computer file, the storage system including an ACL database and a de-duplication directory, the de-duplication directory having a plurality of directory entries, each directory entry including a reference that references ACL data stored in the ACL database, and each directory entry having a name that corresponds to a checksum calculated based at least in part on the referenced ACL data;
ii. calculate a first checksum based at least in part on the first ACL data identified by the first ACL identifier included in the first computer file;
iii. determine whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on (i) a respective directory entry of the de-duplication directory having a name that corresponds to the first checksum, (ii) the reference included in the respective directory entry referencing second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and (iii) one or more references included in one or more of the plurality of directory entries, respectively, referencing the second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and wherein the one or more of the plurality of directory entries comprise a first directory entry having a first name corresponding to the first checksum, and a second directory entry having a second name corresponding to the first checksum with duplicate indicia appended;
iv. de-duplicate the ACL database including:
having determined that the permissions specified by the first ACL data are not previously stored in the ACL database, store the first ACL data in the ACL database; and
having determined that the permissions specified by the first ACL data are previously stored in the ACL database, update the first ACL identifier included in the first computer file to identify the second ACL data stored in the ACL database, and avoid storing the first ACL data in the ACL database; and
v. increment a reference count associated with the second ACL data.
6. The storage system of claim 5 , wherein the storage processor is further configured to store the first computer file.
7. The storage system of claim 5 , wherein the storage processor is further configured to retrieve the second ACL data from the ACL database stored on the storage system.
8. The storage system of claim 5 , wherein the storage processor is further configured to determine whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on the one or more of the plurality of directory entries of the de-duplication directory having one or more names, respectively, corresponding to the first checksum.
9. A storage-system implemented method for reducing duplication of access control lists comprising:
a. receiving, at the storage system from a client, a first computer file including a first access control list (ACL) identifier identifying first ACL data associated with the first computer file, the first ACL data specifying permissions for the first computer file, the storage system including an ACL database and a de-duplication directory, the de-duplication directory having a plurality of directory entries, each directory entry including a reference that references ACL data stored in the ACL database, and each directory entry having a name that corresponds to a checksum calculated based at least in part on the referenced ACL data;
b. calculating, by the storage system, a first checksum based at least in part on the first ACL data identified by the first ACL identifier included in the first computer file;
c. determining, by the storage system, whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on (i) a respective directory entry of the de-duplication directory having a name that corresponds to the first checksum, (ii) the reference included in the respective directory entry referencing second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and (iii) one or more references included in one or more of the plurality of directory entries, respectively, referencing the second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and wherein the one or more of the plurality of directory entries comprise a first directory entry having a first name corresponding to the first checksum, and a second directory entry having a second name corresponding to the first checksum with first duplicate indicia appended;
d. de-duplicating the ACL database by the storage system, the de-duplicating of the ACL database including:
having determined that the permissions specified by the first ACL data are not previously stored in the ACL database, storing, by the storage system, to the de-duplication directory, a third directory entry comprising a first reference that references the first ACL data and a third name comprising the first checksum, and storing, by the storage system, the first ACL data in the ACL database; and
having determined that the permissions specified by the first ACL data are previously stored in the ACL database, updating the first ACL identifier included in the first computer file to identify the second ACL data stored in the ACL database, and avoiding storing the first ACL data in the ACL database; and
e. incrementing, by the storage system, a reference count associated with the second ACL data.
10. The method of claim 9 , further comprising storing, by the storage system, the first computer file.
11. The method of claim 9 , wherein the determining of whether or not the permissions specified by the first ACL data are previously stored in the ACL database is further based on the one or more of the plurality of directory entries of the de-duplication directory having one or more names, respectively, corresponding to the first checksum.
12. The method of claim 11 , wherein the third name of the third directory entry corresponds to the first checksum with second duplicate indicia appended.
13. A storage system comprising:
a storage processor providing a file system, the storage processor configured to:
i. receive, from a client, a first computer file including a first access control list (ACL) identifier identifying first ACL data associated with the first computer file, the first ACL data specifying permissions for the first computer file, the storage system including an ACL database and a de-duplication directory, the de-duplication directory having a plurality of directory entries, each directory entry including a reference that references ACL data stored in the ACL database, and each directory entry having a name that corresponds to a checksum calculated based at least in part on the referenced ACL data;
ii. calculate a first checksum based at least in part on the first ACL data identified by the first ACL identifier included in the first computer file;
iii. determine whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on (i) a respective directory entry of the de-duplication directory having a name that corresponds to the first checksum, (ii) the reference included in the respective directory entry referencing second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and (iii) one or more references included in one or more of the plurality of directory entries, respectively, referencing the second ACL data stored in the ACL database that specify the same permissions specified by the first ACL data, and wherein the one or more of the plurality of directory entries comprise a first directory entry having a first name corresponding to the first checksum, and a second directory entry having a second name corresponding to the first checksum with first duplicate indicia appended;
iv. de-duplicate the ACL database including:
having determined that the permissions specified by the first ACL data are not previously stored in the ACL database, store, to the de-duplication directory, a third directory entry comprising a first reference that references the first ACL data and a third name comprising the first checksum, and store the first ACL data in the ACL database; and
having determined that the permissions specified by the first ACL data are previously stored in the ACL database, update the first ACL identifier included in the first computer file to identify the second ACL data stored in the ACL database, and avoid storing the first ACL data in the ACL database; and
v. increment a reference count associated with the second ACL data.
14. The storage system of claim 13 , wherein the storage processor is further configured to store the first computer file.
15. The storage system of claim 13 , wherein the storage processor is further configured to determine whether or not the permissions specified by the first ACL data are previously stored in the ACL database based on the one or more of the plurality of directory entries of the de-duplication directory having one or more names, respectively, corresponding to the first checksum.
16. The storage system of claim 15 , wherein the third name of the third directory entry corresponds to the first checksum with second duplicate indicia appended.