Representing and storing an optimized file system using a system of symlinks, hardlinks and file archives
View Patent ↗A data de-duplication system is used with network attached storage and serves to reduce data duplication and file storage costs. Techniques utilizing both symlinks and hardlinks ensure efficient deletion file/data cleanup and avoid data loss in the event of crashes.
1. A method comprising:
reading file listings from a directory for each file including a first file in the directory;
determining, using a processor, that the first file falls within a policy calling for the first file to be optimized;
optimizing data of the first file by using the processor, wherein the optimizing of data of the first file includes placing the optimized data for the first file into a suitcase file, wherein the suitcase file includes index information for accessing the first file;
creating a symlink and a hardlink to the symlink under a suitcase file directory, wherein the symlink is renamed to an original name of the first file, and wherein the symlink includes a data section that encodes a location of the suitcase file and an index number used to access the index information;
receiving a delete command for the first file in the directory, wherein the hardlink to the symlink is created in a laundry directory upon receiving the delete command, wherein the laundry directory is a directory that is read during system cleaning to determine whether the hardlink is present and to read a link count of the hard link, wherein a clean cycle ends when there are no hardlinks present in the laundry directory, and wherein the hardlink is removed upon determining that the link count is two, as a link count of two indicates that the symlink is deleted and the optimized data in the suitcase file should be deleted; and
returning a delete status of the first file to an application.
2. The method of claim 1 , wherein the symlink is deleted after the hardlink in the laundry directory is created.
3. The method of claim 1 , wherein the hardlink is skipped upon determining that the link count is three, as a count of three indicates that there is still a reference to the symlink and that the symlink and the optimized data in the suitcase file should not be deleted.
4. The method of claim 1 , where an error is logged upon determining that the link count is one.
5. A non-transitory computer readable storage medium storing instructions which, when executed, causes a computer to perform:
reading file listings from a directory for each file including a first file in the directory;
determining, using a processor, that the first file falls within a policy calling for the first file to be optimized;
optimizing data of the first file, by using the processor, wherein the optimizing of the data of the first file includes placing the optimized data for the first file into a suitcase file, wherein the suitcase file includes index information for accessing the optimized data associated with the first file;
creating an symlink and a hardlink to the symlink under a suitcase file directory, wherein the symlink is renamed to an original name of the first file, and wherein the symlink includes a data section that encodes a location of the suitcase file and an index number used to access the index information;
receiving a delete command for the first file in the directory, wherein the hardlink to the symlink is created in a laundry directory upon receiving the delete command, wherein the laundry directory is a directory that is read during system cleaning to determine whether the hardlink is present and to read a link count of the hard link, wherein a clean cycle ends when there are no hardlinks present in the laundry directory, and wherein the hardlink is removed upon determining that the link count is two, as a link count of two indicates that the symlink is deleted and the optimized data in the suitcase file should be deleted; and
returning a delete status of the first file to an application.
6. The non-transitory computer readable storage medium of claim 5 , wherein the symlink is deleted after the hardlink in the laundry directory is created.
7. The non-transitory computer readable storage medium of claim 5 , wherein the hardlink is skipped upon determining that the link count is three, as a count of three indicates that there is still a reference to the symlink and that the symlink and the optimized data in the suitcase file should not be deleted.
8. The non-transitory computer readable storage medium of claim 5 , where an error is logged upon determining that the link count is one.
9. A system comprising:
a storage interface operable to read file listings from a directory for each file including a first file in the directory;
a processor operable to:
determine that the first file falls within a policy calling for the first file to be optimized, optimizing data of the first file wherein the optimizing of data of the first file includes placing the optimized data for the first file into a suitcase file, wherein the suitcase file includes index information for accessing the first file;
create a symlink and a hardlink to the symlink under a suitcase file directory, wherein the symlink is renamed to an original name of the first file, and wherein the symlink includes a data section that encodes a location of the suitcase file and an index number used to access the index information; and
an input interface operable to receive a delete command for the first file in the directory and return a delete status of the first file to an application, wherein the hardlink to the symlink is created in a laundry directory upon receiving the delete command, wherein the laundry directory is read during system cleaning to determine whether the hardlink is present and to read a link count of the hard link, wherein a clean cycle ends if there are no hardlinks present in the laundry directory, wherein the hardlink is removed upon determining that the link count is two, as a link count of two indicates that the symlink is deleted and the optimized data in the suitcase file should be deleted.
10. The system of claim 9 , wherein the symlink is deleted after the hardlink in the laundry directory is created.
11. The system of claim 9 , wherein the hardlink is skipped upon determining that the link count is three, as a count of three indicates that there is still a reference to the symlink and that the symlink and the optimized data in the suitcase file should not be deleted.
12. The system of claim 9 , where an error is logged upon determining that the link count is one.