Cloud storage write cache management system and method
A method, computer program product, and computer system for monitoring health of at least one storage device of a cache in a clustered system. A recovery journal may be maintained, wherein the recovery journal may identify whether one or more chunks of data stored in the cache have been dumped from the at least one storage device to persistent storage in the clustered system. A state of the at least one storage device may be determined based upon, at least in part, the health of the at least one storage device. A recovery action may be performed on the one or more chunks of data stored in the at least one storage device based upon, at least in part, the state of the at least one storage device.
1. A computer-implemented method comprising:
monitoring health of at least one storage device of a cache in a clustered system;
maintaining a recovery journal, wherein the recovery journal identifies whether one or more chunks of data stored in the cache have been dumped from the at least one storage device to persistent storage in the clustered system;
determining a state of the at least one storage device based upon, at least in part, the health of the at least one storage device, wherein the state of the at least one storage device is determined to be a second state if the at least one storage device is incapable of completing I/O requests; and
performing a recovery action on the one or more chunks of data stored in the at least one storage device based upon, at least in part, the state of the at least one storage device, wherein the recovery action includes:
invalidating at least a portion of the one or more chunks of data stored in the at least one storage device if the state of the at least one storage device is the second state; and
recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
2. The computer-implemented method of claim 1 wherein the state of the at least one storage device is determined to be a first state if at least one I/O request directed to the at least one storage device fails.
3. The computer-implemented method of claim 2 wherein the recovery action includes attempting to dump each of the one or more chunks of data stored in the at least one storage device to the persistent storage if the state of the at least one storage device is the first state, wherein each chunk of data successfully dumped is removed from the recovery journal.
4. The computer-implemented method of claim 3 wherein the recovery action further includes recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
5. The computer-implemented method of claim 1 wherein the recovery action includes restoring the recovery journal from a file system on the at least one storage device if the recovery journal is one of corrupted and lost.
6. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, cause at least a portion of the one or more processor to perform operations comprising:
monitoring health of at least one storage device of a cache in a clustered system;
maintaining a recovery journal, wherein the recovery journal identifies whether one or more chunks of data stored in the cache have been dumped from the at least one storage device to persistent storage in the clustered system;
determining a state of the at least one storage device based upon, at least in part, the health of the at least one storage device, wherein the state of the at least one storage device is determined to be a first state if at least one I/O request directed to the at least one storage device fails; and
performing a recovery action on the one or more chunks of data stored in the at least one storage device based upon, at least in part, the state of the at least one storage device, wherein the recovery action includes attempting to dump each of the one or more chunks of data stored in the at least one storage device to the persistent storage if the state of the at least one storage device is the first state, wherein each chunk of data successfully dumped is removed from the recovery journal.
7. The computer program product of claim 6 wherein the recovery action further includes recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
8. The computer program product of claim 6 wherein the state of the at least one storage device is determined to be a second state if the at least one storage device is incapable of completing I/O requests.
9. The computer program product of claim 8 wherein the recovery action further includes:
invalidating at least a portion of the one or more chunks of data stored in the at least one storage device if the state of the at least one storage device is the second state; and
recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
10. The computer program product of claim 6 wherein the recovery action includes restoring the recovery journal from a file system on the at least one storage device if the recovery journal is one of corrupted and lost.
11. A computing system including one or more processors and one or more memories configured to perform operations comprising:
monitoring health of at least one storage device of a cache in a clustered system;
maintaining a recovery journal, wherein the recovery journal identifies whether one or more chunks of data stored in the cache have been dumped from the at least one storage device to persistent storage in the clustered system;
determining a state of the at least one storage device based upon, at least in part, the health of the at least one storage device, wherein the state of the at least one storage device is determined to be a second state if the at least one storage device is incapable of completing I/O requests; and
performing a recovery action on the one or more chunks of data stored in the at least one storage device based upon, at least in part, the state of the at least one storage device, wherein the recovery action includes:
invalidating at least a portion of the one or more chunks of data stored in the at least one storage device if the state of the at least one storage device is the second state; and
recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
12. The computing system of claim 11 wherein the state of the at least one storage device is determined to be a first state if at least one I/O request directed to the at least one storage device fails.
13. The computing system of claim 12 wherein the recovery action includes attempting to dump each of the one or more chunks of data stored in the at least one storage device to the persistent storage if the state of the at least one storage device is the first state, wherein each chunk of data successfully dumped is removed from the recovery journal.
14. The computing system of claim 13 wherein the recovery action further includes recovering at least a portion of the one or more chunks of data that remain in the recovery journal.
15. The computing system of claim 11 wherein the recovery action includes restoring the recovery journal from a file system on the at least one storage device if the recovery journal is one of corrupted and lost.