Methods and systems for efficient erasure-coded storage systems
Methods and the like according to the disclosure can include determining an information type of digital information, writing the digital information using a first process (in response to a determination that the information type of the digital information is a first information type), and, in response to a determination that the information type of the digital information is a second information type, determining erasure-coded parity information for the digital information and writing the digital information using a second process (where the digital information is to be written to erasure-coded storage). In the former case, the first process that includes writing the digital information to storage. The second process includes writing the digital information and the erasure-coded parity information to erasure-coded storage, without writing the digital information to a log.
1. A computer-implemented method, implemented in a computer system, comprising:
determining an information type of digital information, wherein
the digital information is to be written to a storage system;
in response to a determination that the information type is a first information type,
writing the digital information using a first process, wherein
the first information type is metadata, and
the first process comprises
writing the digital information to a log, and
writing the digital information to a storage unit of the storage system; and
in response to a determination that the information type is a second information type,
determining erasure-coded parity information for the digital information, wherein
the second information type is data, and
the digital information is to be written to an erasure-coded storage unit of the storage system, and
writing the digital information using a second process, wherein
the second process comprises
writing the digital information and the erasure-coded parity information to the erasure-coded storage unit, without writing the digital information to the log.
2. The method of claim 1 , wherein
the storage unit comprises journaled storage,
the erasure-coded storage unit comprises non-journaled erasure-coded storage,
the first process writes the digital information to the journaled storage, and
the second process writes the digital information to the non-journaled erasure-coded storage.
3. The method of claim 2 , wherein
the journaled storage is a journaled erasure-coded storage volume, and
the non-journaled erasure-coded storage is a non-journaled erasure-coded storage volume.
4. The method of claim 2 , wherein
the non-journaled erasure-coded storage comprises a non-journaled erasure-coded storage volume.
5. The method of claim 4 , wherein
the journaled storage is journaled erasure-coded storage,
the first process writes the digital information to the journaled erasure-coded storage, and
the second process writes the digital information to the non-journaled erasure-coded storage.
6. The method of claim 5 , further comprising:
determining other erasure-coded parity information for other digital information; and
writing the other digital information and the other erasure-coded parity information to the journaled storage.
7. The method of claim 2 , wherein
the writing the digital information to the erasure-coded storage unit comprises the writing the digital information to the log,
the digital information is a file of a container, and
the file is one of a data file or a metadata file.
8. The method of claim 7 , wherein
the journaled storage comprises a journaled storage volume,
the non-journaled erasure-coded storage comprises a non-journaled erasure-coded storage volume,
the storage unit and the erasure-coded storage unit are spanned by a multi-volume file system,
a file system attribute of the journaled storage volume is set to “metadataok”, and
a file system attribute of the non-journaled erasure-coded storage volume is set to “dataonly”.
9. The method of claim 1 , wherein
the storage unit is of a storage type, and
the storage type is erasure-coded storage.
10. The method of claim 4 , wherein the writing the digital information and the erasure-coded parity information to the erasure-coded storage unit, without writing the digital information to the log, comprises:
writing the data to the non-journaled erasure-coded storage volume of the erasure-coded storage unit.
11. The method of claim 10 , further comprising:
dividing the data into a plurality of extents, wherein
a size of each extent of the plurality of extents is such that the writing the data to the non-journaled erasure-coded storage volume facilitates the each extent of the plurality of extents being written starting on an erasure-coded stripe boundary of the non-journaled erasure-coded storage volume.
12. The method of claim 1 , wherein
the storage unit is of a storage type, and
the storage type is at least one of
an n-way mirrored storage type,
a redundant array of independent disks storage type, or
a nested redundant array of inexpensive disks storage type.
13. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:
determining an information type of digital information, wherein
the digital information is to be written to a storage system;
in response to a determination that the information type is a first information type,
writing the digital information using a first process, wherein
the first information type is metadata, and
the first process comprises
writing the digital information to a log, and
writing the digital information to a storage unit of the storage system; and
in response to a determination that the information type is a second information type,
determining erasure-coded parity information for the digital information, wherein
the second information type is data, and
the digital information is to be written to an erasure-coded storage unit of the storage system, and
writing the digital information using a second process, wherein
the second process comprises
writing the digital information and the erasure-coded parity information to the erasure-coded storage unit, without writing the digital information to the log.
14. The non-transitory computer-readable storage medium of claim 13 , wherein
the storage unit comprises journaled storage,
the erasure-coded storage unit comprises non-journaled erasure-coded storage,
the first process writes the digital information to the journaled storage, and
the second process writes the digital information to the non-journaled erasure-coded storage.
15. The non-transitory computer-readable storage medium of claim 14 , wherein
the digital information is one of a data file or a metadata file, and
the storage unit and the erasure-coded storage unit are spanned by a multi-volume file system.
16. The non-transitory computer-readable storage medium of claim 13 , wherein
the storage unit is of a storage type, and
the storage type is erasure-coded storage.
17. The non-transitory computer-readable storage medium of claim 14 , wherein
the non-journaled erasure-coded storage comprises a non-journaled erasure-coded storage volume,
and
the writing the digital information and the erasure-coded parity information to the erasure-coded storage unit, without writing the digital information to the log, comprises
writing the data to the non-journaled erasure-coded storage volume of the erasure-coded storage unit.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
dividing the data into a plurality of extents, wherein
the non-journaled erasure-coded storage comprises a non-journaled erasure-coded storage volume, and
a size of each extent of the plurality of extents is such that the writing the data to the non-journaled erasure-coded storage volume facilitates the each extent of the plurality of extents being written starting on an erasure-coded stripe boundary of the non-journaled erasure-coded storage volume.
19. The non-transitory computer-readable storage medium of claim 14 , wherein
the first process writes the digital information to the journaled storage, and
the second process writes the digital information to the non-journaled erasure-coded storage.
20. A computing system comprising:
one or more processors, wherein
the computing system is configured to be communicatively coupled to a storage system; and
a computer-readable storage medium coupled to the one or more processors, comprising program instructions, which, when executed by the one or more processors, perform a method comprising
determining an information type of digital information, wherein
the digital information is to be written to the storage system,
in response to a determination that the information type is a first information type,
writing the digital information using a first process, wherein
the first information type is metadata, and
the first process comprises
writing the digital information to a log, and
writing the digital information to a storage unit of the storage system, and
in response to a determination that the information type is a second information type,
determining erasure-coded parity information for the digital information, wherein
the second information type is data, and
the digital information is to be written to an erasure-coded storage unit of the storage system, and
writing the digital information using a second process, wherein
the second process comprises
writing the digital information and the erasure-coded parity information to the erasure-coded storage unit, without writing the digital information to the log.