Object storage in stripe file systems
A computer-implemented method for storing backup data includes receiving, by a computing system, data associated with a file to be backed up; performing, by the computing system, de-duplication on the file to remove duplicated data and to generate unique backup data; receiving, by the computing system, a file handle based on opening a block-based storage file; storing, by the computing system, a metadata container in the block-based storage file stripe aligned; and storing, by the computing system, a data container in the block-based storage file stripe aligned, wherein the metadata container and the data container are associated with a set of objects, and wherein the data container includes the unique backup data.
1. A method comprising:
receiving, by a computing system, data associated with a file to be backed up;
performing, by the computing system, de-duplication on the file to remove duplicated data and to generate unique backup data;
causing, by the computing system, a block-based storage file to be opened based on the unique backup data, the block-based storage file being stripe aligned;
receiving, by the computing system, a file handle and an offset based on the opening of the stripe aligned block-based storage file;
storing, by the computing system, a metadata container using the file handle and the offset in the striped aligned block-based storage file; and
storing, by the computing system, a data container using the file handle and the offset, in the stripe aligned block-based storage file, the metadata container and the data container being associated with a set of objects, and the data container including the unique backup data.
2. The method of claim 1 , wherein the file to be backed up is associated with a file name.
3. The method of claim 2 , further comprising generating, by the computing system, an object identifier (ID) for the object based on the file name and the offset.
4. The method of claim 3 , wherein the object ID is included in the metadata container.
5. The method of claim 4 , wherein said storing of the metadata container and the data container in the block-based storage file is performed based on at least the offset and a length of the file to be backed up.
6. The method of claim 5 , wherein said storing of the metadata container and the data container in the block-based storage file is performed based on a stripe size of a block-based storage device.
7. The method of claim 6 , wherein said storing of the metadata container and the data container in the block-based storage file is performed based on using an existing storage layer interface associated with the block-based storage device.
8. The method of claim 7 , further comprising:
retrieving, by the computing system, the object from the block-based storage device based on the offset and the length and based on mapping the file name and the offset to the object ID.
9. A system comprising:
one or more processors; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
receive data associated with a file to be backed up;
perform de-duplication on the file to remove duplicated data, the performing of de-duplication causing generation of a unique backup data;
cause a block-based storage file to be opened based on the generation of the unique backup data, the block-based storage file being stripe aligned;
receive a file handle and an offset based on the opening of the stripe aligned block-based storage file;
store a metadata container using the file handle and the offset in the striped aligned block-based storage file; and
store a data container using the file handle and the offset, in the stripe aligned block-based storage file, the metadata file container and the data container being associated with a set of objects, and the data container including the unique backup data.
10. The system of claim 9 , wherein the file to be backed up is associated with a file name and.
11. The system of claim 10 , further comprising instructions to generate an object identifier (ID) for the object based on the file name and the offset.
12. The system of claim 11 , wherein the object ID is included in the metadata container.
13. The system of claim 12 , wherein said instructions to store the metadata container and the data container in the block-based storage file are performed based on at least the offset and a length of the file to be backed up.
14. The system of claim 13 , wherein said instructions to store the metadata container and the data container in the block-based storage file are performed based on a stripe size of a block-based storage device.
15. The system of claim 14 , wherein said instructions to store the metadata container and the data container in the block-based storage file are performed based on using an existing storage layer interface associated with the block-based storage device.
16. The system of claim 15 , further comprising instructions to:
retrieve the object from the block-based storage device based on the offset and the length and based on mapping the file name and the offset to the object ID.
17. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
receiving, by a computing system, data associated with a file to be backed up;
performing, by the computing system, de-duplication on the file to remove duplicated data, the performing of de-duplication causing generation of a unique backup data;
causing, by the computing system, a block-based storage file to be opened based on the generation of the unique backup data, the block-based storage file being stripe aligned;
receiving, by the computing system, a file handle and an offset based on the opening of the stripe aligned block-based storage file;
storing, by the computing system, a metadata container using the file handle and the offset in the striped aligned block-based storage file; and
storing, by the computing system, a data container using the file handle and the offset, in the stripe aligned block-based storage file, the metadata container and the data container being associated with a set of objects, and the data container including the unique backup data.
18. The computer program product of claim 17 , wherein the file to be backed up is associated with a file name.
19. The computer program product of claim 18 , further comprising instructions to generate an object identifier (ID) for the object based on the file name and the offset, wherein the object ID is included in the metadata file.