Modifying data storage in response to detection of a memory system imbalance
View Patent ↗A method begins by a processing module determining an imbalance between inode memory utilization and data storage memory utilization. When the imbalance compares unfavorably to an imbalance threshold, the method continues with the processing module determining whether the inode memory utilization is out of balance with respect to the data storage memory utilization or whether the data storage memory utilization is out of balance with respect to the inode memory utilization. When the inode memory utilization is out of balance with respect to the data storage memory utilization, the method continues with the processing module transferring a set of data objects from a data object section to a data block section and transferring object mapping information of the set of data objects into block mapping information for the set of data objects.
1. A method for execution by a dispersed storage (DS) unit, the method comprises:
determining an imbalance between inode memory utilization of an inode memory and data storage memory utilization of a data storage memory, wherein the data storage memory stores data objects in a data object section and stores data blocks in a data block section, and wherein the inode memory stores object mapping information that associates object directory information of the data objects with object storage location information of the data objects and stores block mapping information that associates block directory information of the data objects with block storage location information of the data blocks;
when the imbalance between the inode memory utilization and the data storage memory utilization compares unfavorably to an imbalance threshold, determining whether the inode memory utilization is out of balance with respect to the data storage memory utilization or whether the data storage memory utilization is out of balance with respect to the inode memory utilization; and
when the inode memory utilization is out of balance with respect to the data storage memory utilization:
transferring a set of data objects from the data object section to the data block section; and
transferring object mapping information of the set of data objects into block mapping information for the set of data objects.
2. The method of claim 1 , wherein the transferring the set of data objects comprises:
identifying one or more of the data objects that are associated with a common pillar number as the set of data objects;
determining an address offset value between data objects of the set of data objects based on the common pillar and size of the data objects of the set of data objects;
aggregating the set of data objects into an aggregated data block in accordance with the address offset value; and
storing the aggregated data block as the set of data objects in the data block section.
3. The method of claim 2 , wherein the transferring the object mapping information of the set of data objects comprises:
converting data object addresses of the set of data objects into a data block address based on the address offset value; and
replacing the data object addresses of the set of data objects with the data block address.
4. The method of claim 2 , wherein the determining the address offset value comprises:
obtaining different sizes for data objects of the set of data objects;
determining a maximum size from the difference sizes; and
generating the address offset value based on the maximum size.
5. The method of claim 1 further comprises:
when the data storage memory utilization is out of balance with respect to the inode memory utilization:
transferring a data block of the data blocks from the data block section to the data object section; and
transferring block mapping information of the data block into object mapping information for the data block.
6. The method of claim 5 , wherein the transferring the data block comprises:
dividing the data block into a set of transfer data objects;
generating data object addresses for the set of transfer data objects based on a data block address of the data block and a size of transfer data objects of the set of transfer data objects; and
storing the set of transfer data objects in the data object section.
7. The method of claim 6 , wherein the transferring the block mapping information of the set of data objects comprises:
converting a data block address of the data block into a set of data object addresses based on the address offset value and the data block address; and
replacing the data block address of the data block with the set of data object addresses.
8. The method of claim 1 , wherein a data object of the data objects comprises:
an encoded data slice, wherein a data segment is encoded into a plurality of encoded data slices in accordance with a dispersed storage error coding function, and wherein the encoded data slice is one of the plurality of encoded data slices.
9. A dispersed storage (DS) module comprises:
a first module operable to determine an imbalance between inode memory utilization of an inode memory and data storage memory utilization of a data storage memory, wherein the data storage memory stores data objects in a data object section and stores data blocks in a data block section, and wherein the inode memory stores object mapping information that associates object directory information of the data objects with object storage location information of the data objects and stores block mapping information that associates block directory information of the data objects with block storage location information of the data blocks;
when the imbalance between the inode memory utilization and the data storage memory utilization compares unfavorably to an imbalance threshold, a second module operable to determine whether the inode memory utilization is out of balance with respect to the data storage memory utilization or whether the data storage memory utilization is out of balance with respect to the inode memory utilization; and
when the inode memory utilization is out of balance with respect to the data storage memory utilization:
a third module operable to transfer a set of data objects from the data object section to the data block section; and
a fourth module operable to transfer object mapping information of the set of data objects into block mapping information for the set of data objects.
10. The DS module of claim 9 , wherein the third module is further operable to transfer the set of data objects by:
identifying one or more of the data objects that are associated with a common pillar number as the set of data objects;
determining an address offset value between data objects of the set of data objects based on the common pillar number and size of the data objects of the set of data objects;
aggregating the set of data objects into an aggregated data block in accordance with the address offset value; and
storing the aggregated data block as the set of data objects in the data block section.
11. The DS module of claim 10 , wherein the fourth module is further operable to transfer the object mapping information of the set of data objects by:
converting data object addresses of the set of data objects into a data block address based on the address offset value; and
replacing the data object addresses of the set of data objects with the data block address.
12. The DS module of claim 10 , wherein the third module is further operable to determine the address offset value by:
obtaining different sizes for data objects of the set of data objects;
determining a maximum size from the difference sizes; and
generating the address offset value based on the maximum size.
13. The DS module of claim 9 further comprises:
when the data storage memory utilization is out of balance with respect to the inode memory utilization:
a fifth module operable to transfer a data block of the data blocks from the data block section to the data object section; and
a sixth module operable to transfer block mapping information of the data block into object mapping information for the data block.
14. The DS module of claim 13 , wherein the fifth module is further operable to transfer the data block by:
dividing the data block into a set of transfer data objects;
generating data object addresses for the set of transfer data objects based on a data block address of the data block and a size of transfer data objects of the set of transfer data objects; and
storing the set of transfer data objects in the data object section.
15. The DS module of claim 14 , wherein the sixth module is further operable to transfer the block mapping information of the set of data objects by:
converting a data block address of the data block into a set of data object addresses based on the address offset value and the data block address; and
replacing the data block address of the data block with the set of data object addresses.
16. The DS module of claim 9 , wherein a data object of the data objects comprises:
an encoded data slice, wherein a data segment is encoded into a plurality of encoded data slices in accordance with a dispersed storage error coding function, and wherein the encoded data slice is one of the plurality of encoded data slices.