Storage device as buffer for unreliable storage
A system and method for managing storage of a digital stream including writing data of the stream to a network storage device, while monitoring availability of the network storage device, switching to writing additional data of the stream to a standby storage device while continuing monitoring the availability of the network storage device, if the network storage device becomes unavailable, switching back to writing additional data of the stream to the network storage device while continuing monitoring the availability of the network storage device, when the network storage device becomes available, and writing to the network storage device the additional data that was written to the standby storage device while the network storage device was unavailable, concurrently with the writing of the additional data of the stream to the network storage device.
1. A method for managing storage of a digital stream, the method comprising:
writing data of the stream to a main storage device, while monitoring availability of the main storage device;
switching to writing additional data of the stream to a standby storage device while continuing monitoring the availability of the main storage device, if the main storage device becomes unavailable;
switching back to writing additional data of the stream to the main storage device while continuing monitoring the availability of the main storage device, when the main storage device becomes available; and
writing to the main storage device the additional data that was written to the standby storage device while the main storage device was unavailable, concurrently with the writing of the additional data of the stream to the main storage device,
wherein monitoring availability of the main storage device comprises:
performing test read and write operations into various locations on the main storage device;
measuring response time of the test operations; and
determining that the main storage device is unavailable if the response time is above a predetermined threshold.
2. The method of claim 1 , wherein the main storage device is a network storage device and the standby storage device is a direct access storage device.
3. The method of claim 2 , wherein monitoring availability of the network storage device comprises performing network tests.
4. The method of claim 1 , comprising:
determining whether the main storage device is available or unavailable based on at least one of: a type of failure of the main storage device, amount of failures of the main storage device that occurred in a time interval and on an average duration of each failure of the main storage device.
5. The method of claim 1 , comprising:
monitoring of available throughput of the main storage device,
wherein the writing to the main storage device the additional data that was written to the standby storage device is performed considering available throughput of the main storage device.
6. The method of claim 1 , wherein the data is saved in files, and the method comprises:
maintaining a list of the files, wherein the list comprises for each file an indication of whether it is stored on the main storage device or on the standby storage device and location of each file on either the main storage device or the standby storage device.
7. A method for managing storage of a digital stream, the method comprising:
writing data of the stream to a main storage device, while monitoring availability of the main storage device;
switching to writing additional data of the stream to a standby storage device while continuing monitoring the availability of the main storage device, if the main storage device becomes unavailable;
switching back to writing additional data of the stream to the main storage device while continuing monitoring the availability of the main storage device, when the main storage device becomes available; and
writing to the main storage device the additional data that was written to the standby storage device while the main storage device was unavailable, concurrently with the writing of the additional data of the stream to the main storage device,
wherein the data is saved in files, and the method comprises:
maintaining a list of the files, wherein the list comprises for each file an indication of whether it is stored on the main storage device or on the standby storage device and location of each file on either the main storage device or the standby storage device; and
maintaining a linked list, wherein the linked list comprises order of files containing consecutive data of a stream, and indication for each file on whether it is stored on the main storage device or on the standby storage device,
wherein the writing to the main storage device the additional data that was written to the standby storage device comprises:
scanning the linked list;
writing files that are stored in the standby storage device to the main storage device; and
updating the linked list.
8. A recording server for managing storage of a digital stream, the recording server comprising:
a processor configured to:
write data of the stream to a main storage device, while monitoring availability of the main storage device;
switch to writing additional data of the stream to a standby storage device while continuing monitoring the availability of the main storage device, if the main storage device becomes unavailable;
switch back to writing additional data of the stream to the main storage device while continuing monitoring the availability of the main storage device, when the main storage device becomes available; and
write to the main storage device the additional data that was written to the standby storage device while the main storage device was unavailable, concurrently with the writing of the additional data of the stream to the main storage device,
wherein the processor is configured to monitor availability of the network storage device by:
performing test read and write operations into various locations on the main storage device;
measuring response time of the test operations; and
determining that the main storage device is unavailable if the response time is above a predetermined threshold.
9. The recording server of claim 8 , wherein the main storage device is a network storage device and the standby storage device is a direct access storage device.
10. The recording server of claim 9 , wherein the processor is configured to monitor availability of the network storage device by performing network tests.
11. The recording server of claim 8 , wherein the processor is configured to:
determine whether the main storage device is available or unavailable based on at least one of: a type of failure of the main storage device, amount of failures of the main storage device that occurred in a time interval and on an average duration of each failure of the main storage device.
12. The recording server of claim 8 , wherein the processor is configured to:
monitor available throughput of the main storage device; and
write to the main storage device the additional data that was written to the standby storage device considering available throughput of the main storage device.
13. A non-transitory computer-readable storage medium, having stored thereon instructions, that when executed on a processor, cause the processor to:
write data of the stream to a main storage device, while monitoring availability of the main storage device;
switch to writing additional data of the stream to a standby storage device while continuing monitoring the availability of the main storage device, if the main storage device becomes unavailable;
switch back to writing additional data of the stream to the main storage device while continuing monitoring the availability of the main storage device, when the main storage device becomes available; and
write to the main storage device the additional data that was written to the standby storage device while the main storage device was unavailable, concurrently with the writing of the additional data of the stream to the main storage device,
wherein the instructions when executed cause the processor to monitor availability of the network storage device by:
performing test read and write operations into various locations on the main storage device;
measuring response time of the test operations; and
determining that the main storage device is unavailable if the response time is above a predetermined threshold.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the main storage device is a network storage device and the standby storage device is a direct access storage device.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the instructions when executed cause the processor to monitor availability of the network storage device by performing network tests.
16. The non-transitory computer-readable storage medium of claim 13 , wherein the instructions when executed cause the processor to:
determine whether the main storage device is available or unavailable based on at least one of: a type of failure of the main storage device, amount of failures of the main storage device that occurred in a time interval and on an average duration of each failure of the main storage device.
17. The non-transitory computer-readable storage medium of claim 13 , wherein the instructions when executed cause the processor to:
monitor available throughput of the main storage device; and
write to the main storage device the additional data that was written to the standby storage device considering available throughput of the main storage device.