Accessing a large data object in a dispersed storage network
View Patent ↗A method begins by a dispersed storage (DS) processing module generating a data object identifier for data to be stored in a dispersed storage network (DSN) and partitioning the data into a plurality of data partitions based on a set of retrieval preferences and data boundary information. For a data partition, the method continues with the DS processing module dispersed storage error encoding the data partition to produce a plurality of sets of encoded data slices and generating a plurality of sets of DSN addresses for the plurality of sets of encoded data slices, wherein a DSN address of the plurality of sets of DSN addresses includes a representation of the data object identifier, a representation of one or more retrieval preferences of the set of retrieval preferences, a representation of a corresponding portion of the data boundary information, and dispersed storage addressing information.
1. A method for execution by a computing device, the method comprises:
obtaining a data object identifier regarding a data object to identify the data object;
obtaining a plurality of boundary indicators based on the data object and one or more retrieval preferences, wherein the one or more retrieval preferences are utilized to set partition boundaries associated with partitioning the data object into a plurality of data partitions, and wherein the plurality of boundary indicators indicate respective partition boundaries, in which respective data partitions are further segmented into a plurality of data segments and the data segments are dispersed storage error encoded into one or more sets of encoded data slices; and
generating sets of dispersed storage network (DSN) addresses corresponding to respective data partitions of the plurality of data partitions based on the plurality of boundary indicators and the data object identifier, such that a first DSN address set of the sets of DSN addresses has a first common identifier for a first data partition, a common indicator corresponding to retrieval preferences for the first data partition, and a first unique set of encoded data slice identifiers associated with encoded data slices of the first data partition and a second DSN address set of the sets of DSN addresses has a second common identifier for a second data partition, a common indicator corresponding to retrieval preferences for the second data partition, and a second unique set of encoded data slice identifiers associated with encoded data slices of the second data partition.
2. The method of claim 1 further comprises:
generating one or more write requests incorporating one or more sets of the DSN addresses.
3. The method of claim 1 further comprises:
generating one or more read requests incorporating one or more sets of the DSN addresses.
4. The method of claim 1 , wherein the obtaining the data object identifier comprises:
receiving a data name and a storage vault identifier of the data object;
generating the data object identifier to include the vault identifier and an object number, wherein the object number is a random number or is a result of a deterministic function performed on the data name; and
generating associating information, which associates the data name and the data object identifier.
5. The method of claim 1 further comprises:
the first common identifier for the first data partition including a representation of the first data partition, one or more retrieval preferences, and a corresponding boundary indicator for the first data partition;
the common indicator corresponding to the retrieval preferences of the first data partition including a vault identifier; and
the first unique set of encoded data slice identifiers including one or more of a slice index and a segment identifier.
6. The method of claim 1 , wherein the one or more retrieval preferences comprise one or more of: a keyword, a data type, a previous retrieval preference, a data transition indicator, a pattern, a data value, and a data size.
7. A dispersed storage processing module comprises:
directory accessor of a computing device configured to obtain a data object identifier regarding a data object;
boundary generator of the computing device configured to obtain a plurality of boundary indicators based on the data object and one or more retrieval preferences, wherein the one or more retrieval preferences are utilized to set partition boundaries associated with partitioning the data object into a plurality of data partitions, and wherein the plurality of boundary indicators indicate respective partition boundaries, in which respective data partitions are further segmented into a plurality of data segments and the data segments are dispersed storage error encoded into one or more sets of encoded data slices; and
one or more address generators of the computing device configured to generate sets of dispersed storage network (DSN) addresses corresponding to respective data partitions of the plurality of data partitions based on the plurality of boundary indicators and the data object identifier, such that a first DSN address set of the sets of DSN addresses has a first common identifier for a first data partition, a common indicator corresponding to retrieval preferences for the first data partition, and a first unique set of encoded data slice identifiers associated with encoded data slices of the first data partition and a second DSN address set of the sets of DSN addresses has a second common identifier for a second data partition, a common indicator corresponding to retrieval preferences for the second data partition, and a second unique set of encoded data slice identifiers associated with encoded data slices of the second data partition.
8. The dispersed storage processing module of claim 7 further comprises:
write request generator of the computing device configured to generate one or more write requests incorporating one or more sets of the DSN addresses.
9. The dispersed storage processing module of claim 7 further comprises:
read request generator of the computing device configured to generate one or more read requests incorporating one or more sets of the DSN addresses.
10. The dispersed storage processing module of claim 7 , wherein the directory accessor causes the computing device to:
receive a data name and a storage vault identifier of the data object;
generate the data object identifier to include the vault identifier and an object number, wherein the object number is a random number or is a result of a deterministic function performed on the data name; and
generate associating information, which associates the data name and the data object identifier.
11. The dispersed storage processing module of claim 7 further comprises:
the first common identifier for the first data partition including a representation of the first data partition, one or more retrieval preferences, and a corresponding boundary indicator for the first data partition;
the common indicator corresponding to the retrieval preferences of the first data partition including a vault identifier; and
the first unique set of encoded data slice identifiers including one or more of a slice index and a segment identifier.
12. The dispersed storage processing module of claim 7 , wherein the one or more retrieval preferences comprise one or more of: a keyword, a data type, a previous retrieval preference, a data transition indicator, a pattern, a data value, and a data size.