Method and computer system for managing blocks
A computer system (e.g. a file system) for managing blocks, comprises a memory unit including a memory and a processing unit including a processor and coupled to the memory unit. The memory unit is configured to store metadata of data including objects. The metadata points to blocks storing the objects in a storage unit including multiple different sized blocks. The processing unit is configured to, for at least one of the objects, determine different sized blocks for storing the object, from among free blocks in the multiple different sized blocks of the storage unit.
1. A computer system for improving efficiency of data storage and retrieval, the computer system comprising:
a storage unit that includes a plurality of storage blocks, wherein the plurality of storage blocks includes blocks different of sizes;
a memory comprising a cache, a free space bitmap, and a metadata management system;
wherein the cache stores metadata that is associated with data objects that are stored in the plurality of storage blocks, wherein data objects stored in the storage unit are managed based on the metadata and an amount of metadata stored in the memory varies according to a number of blocks in which a respective data object is stored; and
wherein a data allocation policy is retrieved from the metadata management system; and
a processor that is communicatively coupled to the memory and the storage unit, wherein the processor is configured to:
receive a file,
determine a content category of the file,
retrieve the data allocation policy for the file to obtain a header block size and a block size strategy for the content category determined, wherein the data allocation policy includes the header block size and the block size strategy,
store the file in one or more storage blocks of the plurality of storage blocks according to the data allocation policy determined, and
update the metadata for the file based on the one or more storage blocks where the file is stored.
2. The computer system according to claim 1 , wherein
the metadata comprises a pointer to the one or more storage blocks.
3. The computer system according to claim 1 , wherein the block size strategy includes at least one of:
a fixed block size strategy where a block size for allocation is a predetermined number,
a Min Fit strategy where the block size for allocation is chosen as a smallest block size that is equal to or greater than an unallocated remaining size of the file, and
a Max Fit strategy where the block size for allocation is chosen as a largest possible block which is smaller than the unallocated remaining size of the file.
4. The computer system according to claim 3 , wherein the block size strategy includes the Min Fit strategy.
5. The computer system according to claim 3 , wherein the block size strategy includes the Max Fit strategy.
6. The computer system according to claim 3 , wherein the block size strategy includes the fixed block size strategy.
7. A method for improving efficiency of data storage and retrieval, the method comprising:
providing a storage device that includes a plurality of storage blocks, wherein the plurality of storage blocks includes blocks different of sizes;
providing a memory comprising a cache, a free space bitmap, and a metadata management system;
wherein the cache stores metadata that is associated with data objects that are stored in the plurality of storage blocks, wherein data objects stored in the storage device are managed based on the metadata and an amount of metadata stored in the memory varies according to a number of blocks in which a respective data object is stored; and
wherein a data allocation policy is retrieved from the metadata management system; and
receiving, by a processor, a file which is a write target;
determining, by the processor, a content category of the file;
retrieving, by the processor, a data allocation policy for the file to obtain a header block size and a block size strategy for the content category determined, wherein the data allocation policy includes the header block size and the block size strategy,
storing, by the processor, the file in one or more storage blocks of the plurality of storage blocks in the storage device according to the data allocation policy; and
updating metadata for the file based on the one or more storage blocks where the file is stored, wherein data objects stored in the storage device are managed based on the metadata and an amount of metadata associated with a respective object varies according to a number of blocks in which a respective data object is stored.
8. A non-transitory computer readable medium for improving efficiency of data storage and retrieval storing instructions that when executed by a processor cause the processor to:
receive a file which is a write target;
determine a content category of the file;
retrieve a data allocation policy for the file to obtain a header block size and a block size strategy for the content category determined, wherein the data allocation policy includes the header block size and the block size strategy;
store the file in one or more storage blocks of a plurality of storage blocks in a storage device according to the data allocation policy;
access a memory comprising a cache, a free space bitmap, and a metadata management system;
wherein the cache stores metadata that is associated with data objects that are stored in the plurality of storage blocks, wherein the data objects stored in the storage device are managed based on the metadata and an amount of metadata stored in the memory varies according to a number of blocks in which a respective data object is stored; and
wherein a data allocation policy is retrieved from the metadata management system; and
and
update metadata for the file based on the one or more storage blocks where the file is stored, wherein data objects stored in the storage device are managed based on the metadata and an amount of metadata associated with a respective object varies according to a number of blocks in which a respective data object is stored.
9. The method of claim 7 ,
wherein the metadata includes a pointer that points to the one or more storage blocks that store the file.
10. The method of claim 7 , wherein the block size strategy includes at least one of:
a fixed block size strategy where a block size for allocation is a predetermined number,
a Min Fit strategy where the block size for allocation is chosen as a smallest block size that is equal to or greater than an unallocated remaining size of the file, and
a Max Fit strategy where the block size for allocation is chosen as a largest possible block which is smaller than the unallocated remaining size of the file.
11. The non-transitory computer readable medium of claim 8 ,
wherein the metadata includes a pointer that points to the one or more storage blocks that store the file.
12. The non-transitory computer readable medium of claim 8 , wherein the block size strategy includes at least one of:
a fixed block size strategy where a block size for allocation is a predetermined number,
a Min Fit strategy where the block size for allocation is chosen as a smallest block size that is equal to or greater than an unallocated remaining size of the file, and
a Max Fit strategy where the block size for allocation is chosen as a largest possible block which is smaller than the unallocated remaining size of the file.
13. The computer system according to claim 1 , wherein the category of the file is an audio file, a video file or a picture file.
14. The method of claim 7 , wherein the category of the file is an audio file, a video file or a picture file.
15. The non-transitory computer readable medium of claim 8 , wherein the category of the file is an audio file, a video file or a picture file.