Utilizing metadata storage trees in a vast storage network
A method includes receiving data for storage and encoding the data to produce a plurality of data slices. Metadata is determined for a data slice of the plurality of data slices. The metadata is stored in a metadata storage tree. The metadata storage tree is stored via a first plurality of memory devices of a first memory type. The data slice is stored in a slice storage location in a second plurality of memory devices of a second memory type. The slice storage location is indicated by the metadata. The first memory type has a higher performance level than the second memory type based on a utilization approach.
1. A method comprises:
receiving data for storage;
error encoding the data to produce a plurality of data slices;
determining metadata for a data slice of the plurality of data slices;
storing the metadata in a metadata storage tree, wherein the metadata storage tree is stored via a first plurality of memory devices of a first memory type;
storing the data slice in a slice storage location, indicated by the metadata, within a second plurality of memory devices of a second memory type, wherein the first memory type has a higher performance level than the second memory type based on a utilization approach;
determining to access the data slice based on a request;
based on determining to access the data slice:
accessing the metadata for the data slice in the metadata storage tree to determine the slice storage location for the data slice; and
accessing the data slice in the slice storage location based on determining the slice storage location for the data slice via accessing the metadata storage tree.
2. The method of claim 1 , wherein the utilization approach indicates metadata be stored in higher performing memory than data slices.
3. The method of claim 1 , wherein the first plurality of memory devices corresponds to a plurality of solid-state memory devices.
4. The method of claim 1 , further comprising determining at least one first memory device from the first plurality of memory devices to store the metadata based on the utilization approach.
5. The method of claim 4 , further comprising determining at least one second memory device from the second plurality of memory devices to store the data slice based on the utilization approach.
6. The method of claim 1 , further comprising:
determining utilization parameters for the utilization approach by identifying the first plurality of memory devices for storing the metadata storage tree and identifying the second plurality of memory devices for storing data slices.
7. The method of claim 6 , further comprising:
generating an initial metadata storage tree structure in at least one of the first plurality of memory devices.
8. The method of claim 1 , wherein the metadata stored in the metadata storage tree includes an address indicating the slice storage location of the data slice in the second plurality of memory devices.
9. The method of claim 1 , wherein the first plurality of memory devices store a plurality of metadata storage trees that includes the metadata storage tree.
10. The method of claim 1 , wherein the metadata further indicates at least one of: a data object name, or a revision number.
11. The method of claim 1 , further comprising:
receiving access information from a plurality of storage units that include at least one of:
at least one of the first plurality of memory devices, or at least one of the second plurality of memory devices;
generating memory utilization data based on the access information; and
configuring the plurality of storage units based on the memory utilization data.
12. A processing system of a storage system comprises:
at least one processor;
a memory that stores operational instructions that, when executed by the at least one processor, cause the processing system to:
receive data for storage;
encode the data via erasure coding to produce a plurality of data slices;
determine metadata for a data slice of the plurality of data slices;
store the metadata in a metadata storage tree, wherein the metadata storage tree is stored via a first plurality of memory devices of a first memory type;
store the data slice in a slice storage location, indicated by the metadata, within a second plurality of memory devices of a second memory type, wherein the first memory type has a higher performance level than the second memory type based on a utilization approach;
determine to access the data slice based on a request;
based on determining to access the data slice:
access the metadata for the data slice in the metadata storage tree to determine the slice storage location for the data slice; and
access the data slice in the slice storage location based on determining the slice storage location for the data slice via accessing the metadata storage tree.
13. The processing system of claim 12 , wherein the utilization approach indicates metadata be stored higher performing memory than data slices.
14. The processing system of claim 12 , wherein the first plurality of memory devices corresponds to a plurality of solid-state memory devices.
15. The processing system of claim 12 , wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:
determine at least one first memory device from the first plurality of memory devices to store the metadata and determine at least one second memory device from the second plurality of memory devices to store the data slice based on the utilization approach.
16. The processing system of claim 12 , wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:
determine utilization parameters for the utilization approach by identifying the first plurality of memory devices for storing the metadata storage tree and identifying the second plurality of memory devices for storing data slices.
17. The processing system of claim 12 , wherein the metadata stored in the metadata storage tree includes an address indicating the slice storage location of the data slice in the second plurality of memory devices.
18. The processing system of claim 12 , wherein the first plurality of memory devices store a plurality of metadata storage trees that includes the metadata storage tree.
19. The processing system of claim 12 , wherein the metadata further indicates at least one of: a data object name, or a revision number.
20. A non-transitory computer readable storage medium comprises:
at least one memory section that stores operational instructions that, when executed by a processing system that includes a processor and a memory, causes the processing system to:
receive data for storage;
encode the data via erasure coding to produce a plurality of data slices;
determine metadata for a data slice of the plurality of data slices;
store the metadata in a metadata storage tree, wherein the metadata storage tree is stored via a first plurality of memory devices of a first memory type;
store the data slice in a slice storage location, indicated by the metadata, within a second plurality of memory devices of a second memory type, wherein the first memory type has a higher performance level than the second memory type based on a utilization approach;
determine to access the data slice based on a request;
based on determining to access the data slice:
access the metadata for the data slice in the metadata storage tree to determine the slice storage location for the data slice; and
access the data slice in the slice storage location based on determining the slice storage location for the data slice via accessing the metadata storage tree.