Dispersed storage unit and methods with metadata separation for use in a dispersed storage system
View Patent ↗A data slice to be stored in a dispersed storage unit of the dispersed storage system is received. A plurality of data blocks are generated from the data slice and metadata are generated associated with each of the plurality of data blocks. A data file is generated based on the plurality of data blocks. A metadata file is generated based on the metadata associated with the plurality of data blocks. The metadata file and the data file are stored in at least one memory of the dispersed storage unit.
1. A method for use in a dispersed storage system, the method comprising:
receiving a data slice of a data object to be stored in a dispersed storage unit of the dispersed storage system, the data slice having an associated revision number;
generating a plurality of data blocks from the data slice and metadata associated with each of the plurality of data blocks, wherein the metadata includes revision data indicating the revision number associated with the data slice;
generating a data file based on the plurality of data blocks;
generating a metadata file based on the metadata associated with the plurality of data blocks; and
storing the metadata file and the data file in at least one memory of the dispersed storage unit.
2. The method of claim 1 wherein the metadata file is stored separately from the data file.
3. The method of claim 1 wherein the metadata file indicates the plurality of data blocks.
4. The method of claim 1 wherein the metadata associated with the plurality of data blocks includes a block number for each data block;
wherein the metadata file includes a range of block numbers corresponding to the block numbers for the plurality of data blocks.
5. The method of claim 1 wherein the metadata associated with the plurality of data blocks includes a checksum for each data block.
6. The method of claim 1 wherein the metadata associated with the plurality of data blocks includes data length for each data block.
7. The method of claim 1 wherein the plurality of data blocks are of equal length and wherein generating the plurality of data blocks includes:
padding one of the plurality of data blocks with padding data to fill the one of the plurality of data blocks.
8. The method of claim 1 wherein at least two of the plurality of data blocks are of unequal length.
9. The method of claim 1 wherein the metadata file includes a compilation of the metadata associated with the plurality of data blocks.
10. The method of claim 1 wherein the metadata file includes at least one data element associated with the data object.
11. A dispersed storage unit for use in a dispersed storage system, the dispersed storage unit comprising:
a storage unit control module that includes:
a data block generator that receives a data slice of a data object, the data slice having an associated revision number, and that generates a plurality of data blocks from the data slice and metadata associated with each of the plurality of data blocks, wherein the metadata includes revision data indicating the revision number associated with the data slice;
a data file generator that generates a data file based on the plurality of data blocks; and
a meta data file generator that generates a metadata file based on the metadata associated with the plurality of data blocks; and
at least one memory that stores the metadata file and the data file.
12. The dispersed storage unit of claim 11 wherein the at least one memory stores the metadata file separately from the data file.
13. The dispersed storage unit of claim 11 wherein the metadata file generator that generates the metadata file to indicate each of the plurality of data blocks.
14. The dispersed storage unit of claim 11 wherein the metadata file generator generates the metadata associated with the plurality of data blocks to include a block number for each data block.
15. The dispersed storage unit of claim 11 wherein the metadata file generator generates the metadata associated with the plurality of data blocks to include a checksum for each data block.
16. The dispersed storage unit of claim 11 wherein the metadata file generator generates the metadata associated with the plurality of data blocks to include a data length for each data block.
17. The dispersed storage unit of claim 11 wherein the plurality of data blocks are of equal length and wherein the data block generator pads one of the plurality of data blocks with padding data to fill the one of the plurality of data blocks.
18. The dispersed storage unit of claim 11 wherein at least two of the plurality of data blocks are of unequal length.
19. The dispersed storage unit of claim 11 wherein the metadata file includes a compilation of the metadata associated with the plurality of data blocks.
20. The dispersed storage unit of claim 11 wherein the metadata file includes at least one data element associated with the data object.