Modifying and utilizing a file structure in a dispersed storage network
A method for execution by a storage unit of a dispersed storage network includes receiving a slice access request that includes a slice name. A first function is performed on the slice name to produce a bucket file identifier. A second function is performed on the slice name to produce a bucket identifier. A bucket file of a plurality of bucket files is accessed, where the bucket file is identified by utilizing the bucket file identifier. A bucket of a plurality of buckets within the bucket file is accessed, where the bucket is identified by utilizing the bucket identifier. It is determined to modify a number of buckets in the bucket file based on bucket utilization. An attribute of the second function is modified in response, where the attribute is associated with the number of buckets in the bucket file.
1. A method for execution by a storage unit that includes a processor, the method comprises:
receiving a slice access request that includes a slice name;
performing a first function on the slice name to produce a bucket file identifier;
performing a second function on the slice name to produce a bucket identifier;
accessing a bucket file of a plurality of bucket files, wherein the bucket file is identified by utilizing the bucket file identifier;
accessing a bucket of a plurality of buckets within the bucket file, wherein the bucket is identified in the plurality of buckets within the bucket file by utilizing the bucket identifier;
determining to modify a number of buckets in the bucket file based on bucket utilization; and
modifying an attribute of the second function in response to determining to modify a number of buckets in the bucket file, wherein the attribute is associated with the number of buckets in the bucket file.
2. The method of claim 1 , wherein producing the bucket identifier includes:
obtaining a unique attribute associated with the second function based on the bucket file identifier by performing a lookup; and
applying a second one-way deterministic function to the at least a portion of the slice name utilizing the unique attribute to produce the bucket identifier.
3. The method of claim 2 , wherein the unique attribute is a modulo value, and wherein the second one-way deterministic function includes performing a modulo operation utilizing the modulo value.
4. The method of claim 3 , wherein modifying the attribute of the second function includes:
increasing the modulo value when a bucket utilization level of the bucket file is greater than a maximum bucket utilization threshold level;
decreasing the modulo value when the bucket utilization level of the bucket file is less than a minimum bucket utilization threshold level; and
associating the modified modulo value with the bucket file identifier of the bucket file in local memory.
5. The method of claim 1 , wherein the slice name is associated with one encoded data slice of a set of encoded data slices, and wherein a data segment was dispersed storage error encoded to produce the set of encoded data slices for storage in a set of storage units that includes the storage unit.
6. The method of claim 1 , wherein producing the bucket file identifier includes performing a first one-way deterministic function on at least a portion of the slice name to produce the bucket file identifier.
7. The method of claim 1 , wherein the bucket file includes an encoded data slice associated with the slice name, and wherein the bucket file further includes a number of entries indicator.
8. A processing system of a storage unit comprises:
at least one processor;
a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to:
receive a slice access request that includes a slice name;
perform a first function on the slice name to produce a bucket file identifier;
perform a second function on the slice name to produce a bucket identifier;
access a bucket file of a plurality of bucket files, wherein the bucket file is identified by utilizing the bucket file identifier;
access a bucket of a plurality of buckets within the bucket file, wherein the bucket is identified in the plurality of buckets within the bucket file by utilizing the bucket identifier;
determine to modify a number of buckets in the bucket file based on bucket utilization; and
modify an attribute of the second function in response to determining to modify a number of buckets in the bucket file, wherein the attribute is associated with the number of buckets in the bucket file.
9. The processing system of claim 8 , wherein producing the bucket identifier includes:
obtaining a unique attribute associated with the second function based on the bucket file identifier by performing a lookup; and
applying a second one-way deterministic function to the at least a portion of the slice name utilizing the unique attribute to produce the bucket identifier.
10. The processing system of claim 9 , wherein the unique attribute is a modulo value, and wherein the second one-way deterministic function includes performing a modulo operation utilizing the modulo value.
11. The processing system of claim 10 , wherein modifying the attribute of the second function includes:
increasing the modulo value when a bucket utilization level of the bucket file is greater than a maximum bucket utilization threshold level;
decreasing the modulo value when the bucket utilization level of the bucket file is less than a minimum bucket utilization threshold level; and
associating the modified modulo value with the bucket file identifier of the bucket file in local memory.
12. The processing system of claim 8 , wherein the slice name is associated with one encoded data slice of a set of encoded data slices, and wherein a data segment was dispersed storage error encoded to produce the set of encoded data slices for storage in a set of storage units that includes the storage unit.
13. The processing system of claim 8 , wherein producing the bucket file identifier includes performing a first one-way deterministic function on at least a portion of the slice name to produce the bucket file identifier.
14. The processing system of claim 8 , wherein the bucket file includes an encoded data slice associated with the slice name, and wherein the bucket file further includes a number of entries indicator.
15. A non-transitory computer readable storage medium comprises:
at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to:
receive a slice access request that includes a slice name;
perform a first function on the slice name to produce a bucket file identifier;
perform a second function on the slice name to produce a bucket identifier;
access a bucket file of a plurality of bucket files, wherein the bucket file is identified by utilizing the bucket file identifier;
access a bucket of a plurality of buckets within the bucket file, wherein the bucket is identified in the plurality of buckets within the bucket file by utilizing the bucket identifier;
determine to modify a number of buckets in the bucket file based on bucket utilization; and
modify an attribute of the second function in response to determining to modify a number of buckets in the bucket file, wherein the attribute is associated with the number of buckets in the bucket file.
16. The non-transitory computer readable storage medium of claim 15 , wherein producing the bucket identifier includes:
obtaining a unique attribute associated with the second function based on the bucket file identifier by performing a lookup; and
applying a second one-way deterministic function to the at least a portion of the slice name utilizing the unique attribute to produce the bucket identifier.
17. The non-transitory computer readable storage medium of claim 16 , wherein the unique attribute is a modulo value, and wherein the second one-way deterministic function includes performing a modulo operation utilizing the modulo value.
18. The non-transitory computer readable storage medium of claim 17 , wherein modifying the attribute of the second function includes:
increasing the modulo value when a bucket utilization level of the bucket file is greater than a maximum bucket utilization threshold level;
decreasing the modulo value when the bucket utilization level of the bucket file is less than a minimum bucket utilization threshold level; and
associating the modified modulo value with the bucket file identifier of the bucket file in local memory.
19. The non-transitory computer readable storage medium of claim 15 , wherein the slice name is associated with one encoded data slice of a set of encoded data slices, and wherein a data segment was dispersed storage error encoded to produce the set of encoded data slices for storage in a set of storage units.
20. The non-transitory computer readable storage medium of claim 15 , wherein producing the bucket file identifier includes performing a first one-way deterministic function on at least a portion of the slice name to produce the bucket file identifier.