Concatenating data objects in a vast data storage network
A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.
1. A method for execution in a storage network (SN), the method comprises:
identifying a data object of a plurality of data objects for retrieval from the SN, wherein the plurality of data objects is combined to produce a concatenated data object and wherein the concatenated data object is encoded in accordance with a dispersed encoding function to produce a set of encoded data blocks;
identifying an encoded data block of the set of encoded data blocks corresponding to the data object based on a mapping of the plurality of data objects;
retrieving the encoded data block from a storage unit; and
decoding the encoded data block in accordance with the dispersed encoding function and the mapping to reproduce the data object.
2. The method of claim 1 , wherein the concatenated data object is encoded by:
generating a plurality of data blocks; and
dispersed error encoding the plurality of data blocks to produce the set of encoded data blocks.
3. The method of claim 2 , wherein the data object is mapped to the encoded data block of the plurality of data blocks.
4. The method of claim 1 , wherein the dispersed encoding function includes a Cauchy-Reed-Solomon encoding or a Reed-Solomon encoding.
5. The method of claim 1 , wherein the dispersed encoding function includes a forward error-correction encoding.
6. The method of claim 1 , wherein the mapping of the plurality of data objects includes a mapping of the plurality of data objects to a data structure corresponding to a plurality of data blocks, wherein the data object is mapped to one or more of the plurality of data blocks.
7. The method of claim 6 , wherein the data structure is a data matrix that includes the plurality of data blocks.
8. A computer readable memory comprises:
a memory section that stores operational instructions that, when executed by a computing device of a storage network (SN), causes the computing device to perform operations that include:
identifying a data object of a plurality of data objects for retrieval from the SN, wherein the plurality of data objects is combined to produce a concatenated data object and wherein the concatenated data object is encoded in accordance with a dispersed encoding function to produce a set of encoded data blocks;
identifying an encoded data block of the set of encoded data blocks corresponding to the data object based on a mapping of the plurality of data objects;
retrieving the encoded data block from a storage unit; and
decoding the encoded data block in accordance with the dispersed encoding function and the mapping to reproduce the data object.
9. The computer readable memory of claim 8 , wherein the concatenated data object is encoded by:
generating a plurality of data blocks; and
dispersed error encoding the plurality of data blocks to produce the set of encoded data blocks.
10. The computer readable memory of claim 9 , wherein the data object is mapped to the encoded data block of the plurality of data blocks.
11. The computer readable memory of claim 8 , wherein the dispersed encoding function includes a Cauchy-Reed-Solomon encoding or a Reed-Solomon encoding.
12. The computer readable memory of claim 8 , wherein the dispersed encoding function includes a forward error-correction encoding.
13. The computer readable memory of claim 8 , wherein the mapping of the plurality of data objects includes a mapping of the plurality of data objects to a data structure corresponding to a plurality of data blocks, wherein the data object is mapped to one or more of the plurality of data blocks.
14. The computer readable memory of claim 13 , wherein the data structure is a data matrix that includes the plurality of data blocks.
15. A device of a storage network (SN), the device comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to perform operations that include:
identifying a data object of a plurality of data objects for retrieval from the SN, wherein the plurality of data objects is combined to produce a concatenated data object and wherein the concatenated data object is encoded in accordance with a dispersed encoding function to produce a set of encoded data blocks;
identifying an encoded data block of the set of encoded data blocks corresponding to the data object based on a mapping of the plurality of data objects;
retrieving the encoded data block from a storage unit; and
decoding the encoded data block in accordance with the dispersed encoding function and the mapping to reproduce the data object.
16. The device of claim 15 , wherein the concatenated data object is encoded by:
generating a plurality of data blocks; and
dispersed error encoding the plurality of data blocks to produce the set of encoded data blocks.
17. The device of claim 16 , wherein the data object is mapped to the encoded data block of the plurality of data blocks.
18. The device of claim 15 , wherein the dispersed encoding function includes a forward error-correction encoding.
19. The device of claim 15 , wherein the dispersed encoding function includes a Cauchy-Reed-Solomon encoding or a Reed-Solomon encoding.
20. The device of claim 15 , wherein the mapping of the plurality of data objects includes a mapping of the plurality of data objects to a data structure corresponding to a plurality of data blocks, wherein the data object is mapped to one or more of the plurality of data blocks.