MANAGING UNAVAILABLE STORAGE IN A DISPERSED STORAGE NETWORK
A method begins by a processing module of a dispersed storage network (DSN) identifying a data object of a group of data objects for storage in the DSN and determining micro slice encoding parameters for the encoding the data object. The method continues by identifying a set of distributed storage (DS) units for storing encoded micro slices (EMSs) and generating a set of meta-slices from the set of EMSs, followed by determining whether DS units are available to store the meta-slices, determining a mapping scheme for storing the set of EMSs and mapping each meta-slice associated with a DS unit available to store the EMSs. The method continues by transmitting each meta-slice associated with a DS unit available to store the EMSs and when certain DS units of are not available to store the EMSs, transmitting each meta-slice associated with a DS unit not available to the DS units available to store the EMSs.
1 . A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:
identifying a data object of a group of data objects for storage in the DSN;
determining micro slice encoding parameters for the encoding the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment is encoded into a plurality of encoded micro slices in accordance with micro slice encoding parameters, wherein a micro slice decode threshold number of encoded micro slices of the plurality of encoded micro slices is needed to recover the data segment;
dispersed storage error encoding the data segment into a set of encoded micro slices in accordance with micro slice encoding parameters;
identifying a set of distributed storage (DS) units for storing the set of encoded micro slices;
generating a set of meta-slices from the set of encoded micro slices, wherein each meta-slice is generated according to dispersal parameters;
determining whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices;
determining a mapping scheme for storing the set of encoded micro slices;
mapping each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices in accordance with the mapping scheme;
transmitting each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available to store the encoded micro slices; and
in response to determining that one or more DS units of the set of DS units is not available to store the encoded micro slices, transmitting each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices.
2 . The method of claim 1 , wherein the identifying a data object of a group of data objects for storage in the DSN comprises:
receiving a read request for the data segment.
3 . The method of claim 1 , wherein the determining whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices comprises at least one of:
interpreting an error message, initiating a query, interpreting a query response, initiating a test and interpreting a test result.
4 . The method of claim 1 wherein the determining a mapping scheme for storing the set of encoded micro slices is based on at least one of:
a predetermination, a system registry entry, a network performance level indicator and a storage unit performance level indicator.
5 . The method of claim 1 , wherein the transmitting each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available includes generating and sending write slice requests to each of the storage units, wherein each write slice request includes a group of encoded micro slices in accordance with the micro slice mapping.
6 . The method of claim 1 , wherein the transmitting each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices includes issuing one or more write slice requests to the one or more available storage units.
7 . The method of claim 1 , wherein the transmitting each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices includes mapping the meta-slice of the DS unit not available to the one or more DS units of the set of DS units that are available to store the encoded micro slices.
8 . The method of claim 1 further comprising:
after transmitting each meta-slice associated with a DS unit that is unavailable to store the encoded micro slices of the set of encoded micro slices to one of the DS units that is available to store the encoded micro slices of the set of encoded micro slices, determining whether a DS unit that was previously unavailable to store the encoded micro slices of the set of encoded micro slices has become available; and
based on a determination that the DS unit that is unavailable to store the encoded micro slices of the set of encoded micro slices has become available, facilitating migration of one or more meta-slices from the one of the DS units that is available to store the encoded micro slices of the set of encoded micro slices to the DS unit that was previously unavailable to store the encoded micro slices of the set of encoded micro slices and that has become available.
9 . A computer readable memory device comprises:
at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
identify a data object of a group of data objects for storage in the DSN;
determine micro slice encoding parameters for the encoding the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment is encoded into a plurality of encoded micro slices in accordance with micro slice encoding parameters, wherein a micro slice decode threshold number of encoded micro slices of the plurality of encoded micro slices is needed to recover the data segment;
dispersed storage error encode the data segment into a set of encoded micro slices in accordance with micro slice encoding parameters;
identify a set of distributed storage (DS) units for storing the set of encoded micro slices;
generate a set of meta-slices from the set of encoded micro slices, wherein each meta-slice is generated according to dispersal parameters;
determine whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices;
determine a mapping scheme for storing the set of encoded micro slices;
map each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices in accordance with the mapping scheme;
transmit each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available to store the encoded micro slices; and
in response to a determination that one or more DS units of the set of DS units is not available to store the encoded micro slices, transmit each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices.
10 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
identify a data object of a group of data objects for storage in the DSN based on receipt of a write request for the data object.
11 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
determine whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices based on at least one of an error message interpretation, a query, a query response initiation interpretation, a test initiation and a test result interpretation.
12 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
determine a mapping scheme for storing the set of encoded micro slices is based on at least one of a predetermination, a system registry entry, a network performance level indicator and a storage unit performance level indicator.
13 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
transmit each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available by generating and sending write slice requests to each of the storage units, wherein each write slice request includes a group of encoded micro slices in accordance with the micro slice mapping.
14 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
transmit each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available by generating and sending write slice requests to each of the storage units, wherein each write slice request includes a group of encoded micro slices in accordance with the micro slice mapping.
15 . The computer readable memory device of claim 9 , wherein the at least one memory section stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to:
transmit each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices by issuing one or more write slice requests to the one or more DS units of the set of DS units that are available to store the encoded micro slices.
16 . The computer readable memory device of claim 9 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to:
transmit each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices by mapping the meta-slice of the DS unit not available to one of the other available storage units.
17 . The computer readable memory device of claim 9 further comprises:
the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to:
after transmitting each meta-slice associated with a DS unit that is unavailable to store the encoded micro slices of the set of encoded micro slices to one of the DS units that is available to store the encoded micro slices of the set of encoded micro slices, determining whether a DS unit that was previously unavailable to store the encoded micro slices of the set of encoded micro slices has become available; and
based on a determination that the DS unit that is unavailable to store the encoded micro slices of the set of encoded micro slices has become available, facilitating migration of one or more meta-slices from the one of the DS units that is available to store the encoded micro slices of the set of encoded micro slices to the DS unit that was previously unavailable to store the encoded micro slices of the set of encoded micro slices and that has become available.
18 . A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:
an interface;
a local memory; and
a processing module operably coupled to the interface and the local memory, wherein the processing module functions to:
identify a data object of a group of data objects for storage in the DSN;
determine micro slice encoding parameters for the encoding the data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment is encoded into a plurality of encoded micro slices in accordance with micro slice encoding parameters, wherein a micro slice decode threshold number of encoded micro slices of the plurality of encoded micro slices is needed to recover the data segment;
dispersed storage error encode the data segment into a set of encoded micro slices in accordance with micro slice encoding parameters;
identify a set of distributed storage (DS) units for storing the set of encoded micro slices;
generate a set of meta-slices from the set of encoded micro slices, wherein each meta-slice is generated according to dispersal parameters;
determine whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices;
determine a mapping scheme for storing the set of encoded micro slices;
map each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices in accordance with the mapping scheme;
transmit each meta-slice associated with a DS unit available to store the encoded micro slices of the set of encoded micro slices to the one or more DS units of the set of DS units that are available to store the encoded micro slices; and
in response to a determination that one or more DS units of the set of DS units is not available to store the encoded micro slices, transmit each meta-slice associated with a DS unit not available to store the encoded micro slices of the set of encoded micro slices to one of the DS units available to store the encoded micro slices of the set of encoded micro slices.
19 . The computing device of claim 18 , wherein the processing module functions to determine whether one or more DS units of the set of DS units is available to store each meta-slice of the set of meta-slices based on at least one of an error message interpretation, a query, a query response initiation interpretation, a test initiation and a test result interpretation.
20 . The computing device of claim 18 , wherein the processing module functions to determine a mapping scheme for storing the set of encoded micro slices is based on at least one of a predetermination, a system registry entry, a network performance level indicator and a storage unit performance level indicator.