Performance- and cost-efficient archiving of small objects
A method includes receiving a plurality of files where each file represents an object. The method further combines selected files of the plurality of files into an aggregation file and stores the aggregation file on low-latency storage media, such as disk storage media. The method represents the aggregation file with a single inode that is stored on the low-latency storage media. The method migrates the aggregate file to high-latency storage media, such as tape storage media, while leaving the single inode representing the aggregation file on the low-latency storage media. By collecting files into larger aggregation files, the number of files archived is significantly reduced and the files archived are of larger size. The reduced number of files and increased size of the files improves archival and retrieval performance due to reduced metadata operation overhead. A corresponding system and computer program product are also disclosed.
1. A method for archiving small objects in a performance- and cost-efficient manner, the method comprising:
receiving a plurality of files, each file representing an object; combining selected files of the plurality of files into an aggregation file; storing the aggregation file on low-latency storage media;
representing the aggregation file with a single mode stored on the low-latency storage media; and
migrating the aggregation file to high-latency storage media while leaving the single mode representing the aggregation file on the low-latency storage media.
2. The method of claim 1 , wherein the selected files are associated with objects in a container.
3. The method of claim 1 , wherein the selected files are at least one of: files that were created at or near the same time; and files that are likely to be accessed at or near the same time.
4. The method of claim 1 , further comprising storing, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.
5. The method of claim 1 , further comprising recalling the aggregation file from the high-latency storage media.
6. The method of claim 5 , further comprising, upon recalling the aggregation file from the high-latency storage media, splitting the aggregation file into the selected files and storing the selected files on the low-latency storage media.
7. The method of claim 5 , further comprising, upon recalling the aggregation file from the high-latency storage media, replacing, on the low-latency storage media, the single mode with the individual modes associated with the selected files.
8. A computer program product for archiving small objects in a performance- and cost-efficient manner, the computer program product comprising a computer-readable medium having computer-usable program code embodied therein, the computer-usable program code configured to perform the following when executed by at least one processor:
receive a plurality of files, each file representing an object;
combine selected files of the plurality of files into an aggregation file; store the aggregation file on low-latency storage media;
represent the aggregation file with a single mode stored on the low-latency storage media; and
migrate the aggregation file to high-latency storage media while leaving the single mode representing the aggregation file on the low-latency storage media.
9. The computer program product of claim 8 , wherein the selected files are associated with objects in a container.
10. The computer program product of claim 8 , wherein the selected files are at least one of: files that were created at or near the same time; and files that are likely to be accessed at or near the same time.
11. The computer program product of claim 8 , wherein the computer-usable program code is further configured to store, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.
12. The computer program product of claim 8 , wherein the computer-usable program code is further configured to recall the aggregation file from the high-latency storage media.
13. The computer program product of claim 12 , wherein the computer-usable program code is further configured to, upon recalling the aggregation file from the high-latency storage media, split the aggregation file into the selected files and store the selected files on the low-latency storage media.
14. The computer program product of claim 12 , wherein the computer-usable program code is further configured to, upon recalling the aggregation file from the high-latency storage media, replace, on the low-latency storage media, the single mode with the individual modes associated with the selected files.
15. A system for archiving small objects in a performance- and cost-efficient manner, the system comprising:
at least one processor;
at least one memory device coupled to the at least one processor and storing instructions for execution on the at least one processor, the instructions causing the at least one processor to:
receive a plurality of files, each file representing an object;
combine selected files of the plurality of files into an aggregation file; store the aggregation file on low-latency storage media;
represent the aggregation file with a single mode stored on the low-latency storage media; and
migrate the aggregation file to high-latency storage media while leaving the single mode representing the aggregation file on the low-latency storage media.
16. The system of claim 15 , wherein the selected files are associated with objects in a container.
17. The system of claim 15 , wherein the instructions further cause the at least one processor to store, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.
18. The system of claim 15 , wherein the computer-usable program code is further configured to recall the aggregation file from the high-latency storage media.
19. The system of claim 18 , wherein the instructions further cause the at least one processor to, upon recalling the aggregation file from the high-latency storage media, split the aggregation file into the selected files and store the selected files on the low-latency storage media.
20. The system of claim 18 , wherein the instructions further cause the at least one processor to, upon recalling the aggregation file from the high-latency storage media, replace, on the low-latency storage media, the single mode with the individual modes associated with the selected files.