Data caching system and method
A system, computer program product, and computer-implemented method for defining a data stripe that spans one or more of N data storage devices and one or more of M erasure code devices. The one or more N data storage devices and the one or more M erasure code devices are included within a data storage system. A data chunk to be written to the data storage system is received. At least a portion of the data chunk is written to the portion of the data stripe that spans the one or more N data storage devices. Each of the N data storage devices and the M erasure code devices is included within a unique server computer.
1. A data caching system comprising:
a data storage system including:
N data storage devices, and
M erasure code devices;
at least one processor;
at least one memory architecture coupled with the at least one processor;
a first software module executed on the at least one processor and the at least one memory architecture, wherein the first software module is configured to perform operations including defining a data stripe that spans one or more of the N data storage devices and one or more of the M erasure code devices, including defining a plurality of data stripes that each span one or more of the N data storage devices and one or more of the M erasure code devices;
a second software module executed on the at least one processor and the at least one memory architecture, wherein the second software module is configured to perform operations including receiving a data chunk to be written to the data storage system, wherein the data chunk received is larger in size than the capacity of the data stripe; and
a third software module executed on the at least one processor and the at least one memory architecture, wherein the third software module is configured to perform operations including writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices, including writing at least a portion of the data chunk to the portion of each of the plurality of data stripes that spans the one or more N data storage devices;
wherein each of the N data storage devices and the M erasure code devices is included within a unique server computer.
2. The data caching system of claim 1 wherein the data stripe spans all of the N data storage devices, and wherein writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices includes:
writing at least a portion of the data chunk to the portion of the data stripe that spans all of the N data storage devices.
3. The data caching system of claim 1 wherein the data stripe spans one of the N data storage devices, and wherein writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices includes:
writing at least a portion of the data chunk to the portion of the data stripe that spans one of the N data storage devices.
4. The data caching system of claim 1 further comprising:
a fourth software module executed on the at least one processor and the at least one memory architecture, wherein the fourth software module is configured to perform operations including:
generating erasure code data based, at least in part, upon the data chunk to be written to the data storage system, and
writing at least a portion of the erasure code data to the portion of the data stripe that spans the one or more M erasure code devices.
5. The data caching system of claim 4 wherein the erasure code data includes parity data.
6. The data caching system of claim 1 further comprising a fifth software module executed on the at least one processor and the at least one memory architecture, wherein the fifth software module is configured to perform operations including:
temporarily storing at least a portion of the data chunk in a high availability fashion prior to writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices.
7. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
defining a data stripe that spans one or more of N data storage devices and one or more of M erasure code devices, including defining a plurality of data stripes that each span one or more of the N data storage devices and one or more of the M erasure code devices, wherein the one or more N data storage devices and the one or more M erasure code devices are included within a data storage system;
receiving a data chunk to be written to the data storage system, wherein the data chunk received is larger in size than the capacity of the data stripe; and
writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices, including writing at least a portion of the data chunk to the portion of each of the plurality of data stripes that spans the one or more N data storage devices;
wherein each of the N data storage devices and the M erasure code devices is included within a unique server computer.
8. The computer program product of claim 7 wherein the data stripe spans all of the N data storage devices, and wherein the instructions for writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices include instructions for:
writing at least a portion of the data chunk to the portion of the data stripe that spans all of the N data storage devices.
9. The computer program product of claim 7 wherein the data stripe spans one of the N data storage devices, and wherein the instructions for writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices include instructions for:
writing at least a portion of the data chunk to the portion of the data stripe that spans one of the N data storage devices.
10. The computer program product of claim 7 further comprising instructions for:
generating erasure code data based, at least in part, upon the data chunk to be written to the data storage system, and
writing at least a portion of the erasure code data to the portion of the data stripe that spans the one or more M erasure code devices.
11. The computer program product of claim 10 wherein the erasure code data includes parity data.
12. The computer program product of claim 7 further comprising instructions for:
temporarily storing at least a portion of the data chunk in a high availability fashion prior to writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices.
13. A computer-implemented method comprising:
defining a data stripe that spans one or more of N data storage devices and one or more of M erasure code devices, including defining a plurality of data stripes that each span one or more of the N data storage devices and one or more of the M erasure code devices, wherein the one or more N data storage devices and the one or more M erasure code devices are included within a data storage system;
receiving a data chunk to be written to the data storage system, wherein the data chunk received is larger in size than the capacity of the data stripe; and
writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices, including writing at least a portion of the data chunk to the portion of each of the plurality of data stripes that spans the one or more N data storage devices;
wherein each of the N data storage devices and the M erasure code devices is included within a unique server computer.
14. The computer-implemented method of claim 13 wherein the data stripe spans all of the N data storage devices, and wherein writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices includes:
writing at least a portion of the data chunk to the portion of the data stripe that spans all of the N data storage devices.
15. The computer-implemented method of claim 13 wherein the data stripe spans one of the N data storage devices, and wherein writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices includes:
writing at least a portion of the data chunk to the portion of the data stripe that spans one of the N data storage devices.
16. The computer-implemented method of claim 13 further comprising:
generating erasure code data based, at least in part, upon the data chunk to be written to the data storage system, and
writing at least a portion of the erasure code data to the portion of the data stripe that spans the one or more M erasure code devices.
17. The computer-implemented method of claim 16 wherein the erasure code data includes parity data.
18. The computer-implemented method of claim 13 further comprising: temporarily storing at least a portion of the data chunk in a high availability fashion prior to writing at least a portion of the data chunk to the portion of the data stripe that spans the one or more N data storage devices.