Archival data storage system and method

1. An archival storage apparatus for use in a hierarchical storage infrastructure having a primary memory, the archival storage apparatus comprising:

an archival storage unit configured to store archival data, the archival storage unit including at least one spindle of disks configured to magnetically store archival data;

an interconnect configured to couple the archival storage system to the primary storage system; and

a control unit configured to process requests over the interconnect to either archive data stored in the primary memory in the archival storage unit or retrieve data from the archival storage unit and provide the retrieved data to the primary storare unit,

the control unit further configured to extend the lifetime of the at least one spindle of disks by selectively controlling the at least one spindle of disks using an algorithm intended to extend the longevity of the at least one spindle of disks, the algorithm relying on the following parameters:

the cost of the archival storage unit;

the cost per watt to operate the archival storage unit;

the spin lifetime of the at least one spindle of disks;

the total number of start-and-stops of the at least one spindle of disks;

the time taken to spin in the at least one spindle of disks;

the number of watts consumed for read or write operation to the archival storage unit; and

the number of watts consumed for a spin up of the at least one spindle of disks.

2. The apparatus of claim 1 , further comprising a power controller coupled to the control unit and the archival storage unit, the control unit further configured to generate commands to the power controller to selectively power on and off the archival storage unit.

3. The apparatus of claim 2 , wherein the control unit issues an on/off command signal and an ID signal corrsponding to the archival storage unit to the power controller to selectively power on and off the archival storage unit through the power controller.

4. The apparatus of claim 1 , wherein the archival storage unit further includes a controller configured as a slave in a master-slave arrangement with respect to the control unit.

5. The apparatus of claim 1 , wherein the archival storage unit is configured to perform one or more of the following tasks in response to commands from the control unit: shut down or power up the archival storage unit; enter running or standby modes; spin up or spin down the at least one spindle of disks; and periodically perform data integrity checks of the archival data stored on the archival storage unit.

6. The apparatus of claim 1 , wherein the least one spindle of disks is arranged in a RAID configuration.

7. The apparatus of claim 1 , wherein the at least one spindle of disks are configured as individual disks in either a logical or physical arrangement.

8. The apparatus of claim 1 , wherein the control unit maintains a directory of the data archived on the archival storage unit, the directory including a map of the archived data stored in data blocks on the archival storage unit.

9. The apparatus of claim 8 , wherein the directory is updated whenever data is archived or retrieved from the archival storage unit.

10. The apparatus of claim 1 , wherein the control unit is configured to issue commands to the archival storage unit, the commands including at least one of the following: commands to shut down or power up the archival storage unit; commands to cause the archival storage unit to enter a running mode or a standby mode; commands to cause the archival storage unit to spin up or spin down the at least one spindle of disks; and commands to instruct the archival storage unit to perform a data integrity check of the archival data stored on the archival storage unit.

11. The apparatus of claim 1 , further comprising a plurality of the archival storage units, each configured to store archival data on one or more spindles of disks respectively, the control unit further configured to generate commands to the plurality of archival storage units to cause the plurality of archival storage units to selectively shut down or power up, enter run mode or standby mode, cause the spindles to spin up or down, and to perform data integrity checks.

12. The apparatus of claim 1 , further comprising a plurality of the archival storage units, each configured to store archival data on one or more spindles of disks, the control unit further configured to update a directory map of the archival data stored on each of the plurality of archival storage units each time data is archived or retrieved from one of the plurality of archival storage units.

13. The apparatus of claim 1 , wherein the control unit is further configured to process archival and retrieval requests from the primary archival storage system of an information infrastructure, the information infrastructure further including a server cluster coupled to the primary storage unit and further configured to be coupled to one or more clients through a network.

14. The apparatus of claim 1 , wherein the control unit manages access to a plurality of the archival storage units by:

receiving a request;

identifying the appropriate archival storage unit among the plurality of archival storage units to handle the request;

determining if the appropriate archival storage unit is on; and either:

sending the request to the appropriate archival storage unit if the appropriate archival storage unit is on; or

waking up or powering on the appropriate archival storage unit if the appropriate archival storage unit is in sleep mode or is powered off before sending the request to the appropriate archival storage unit;

processing the request at the appropriate archival storage unit; and

repeating the above for each subsequent request received by the control unit.

15. The apparatus of claim 14 , wherein either:

if the appropriate archival storage unit is on, comparing a current value of SU.H which is substantially equal to the duration of time where the life cost and power cost of an idle spinning disk approximately equals the life cost and power cost of a disk spin up and spin down cycle of the appropriate archival storage unit to a predetermined value of Hmax and incrementing the value of SU.H if SU.H is less than Hmax; or

if the appropriate archival storage unit is off or is in sleep mode, comparing the current value of SU.H to a second predetermined value of Hmin and decrementing SU.H if SU.H is greater than Hmin.

16. The apparatus of claim 1 , wherein the control unit manages access to a plurality of the archival storage units by:

defining a threshold H for the archival storage unit equal to the duration of time where the life cost and power cost of an idle spinning disk approximately equals the life cost and power cost of a disk spin up and spin down cycle;

initializing a parameter L which is the remaining spin lifetime of the archival storage unit;

initializing a parameter N which equals the remaining number of starts and stops of the archival storage unit;

noting the last time-stamp T of the previous request;

receiving a new request;

identifflng the appropriate archival storage unit to handle the new request determining if the appropriate archival storage unit is on; and either

sending the request to the appropriate archival storage unit if the appropriate archival storage unit is on or is in wake up mode; or

turning on or waking up the appropriate archival storage unit if the appropriate archival storage unit is in sleep mode or is powered off before sending the request to the appropriate archival storage unit;

processing the request on the appropriate archival storage unit;

updating the parameters of L and T; and

waiting for the next request.

17. The apparatus of claim 16 , wherein the value of H is determined by:

H

=

(

C

SU

N

+

C

W

×

W

Up

×

T

Up

)

/

(

C

SU

L

+

C

W

×

W

RW

)

(

1

)

where:

C SU : the cost of the archival storage unit

C W : the cost per watt

L: the spin lifetime

N: the total number of start-and-stops

T Up : the time taken to spinup

W RW : the number of watts consumed for read or write operations, and

W Up : the number of watts consumed for a spin up.

18. The apparatus of claim 1 wherein the control unit manages the power consumption and disk lifetime of a plurality of the archival storage units by:

sequencing through each of the plurality of archival storage units to determine which of the plurality of archival storage units are running, and for each archival storage unit identified as running:

computing an SU.H value for the running archival storage unit, the SU.H value being defined as substantially equal to the duration of time where the life cost and power cost of an idle spinning disk approximately equals the life cost and power cost of a disk spin up and spin down cycle of the archival storage unit; and

comparing the SU.H value for the running archival storage unit to the elapsed time since the previous request to that running archival storage unit, and either

powering off or placing the running archival storage unit in standby mode if the elapsed time exceeds the SH.U value; or

returning to sequencing through each of the plurality of archival storage units if the elapsed time is less than the SH.U value.

19. The apparatus of claim 18 , further generating a warning if the spinning lifetime or the remaining number of start-and-stops of the archival storage unit powered off or placed in standby mode falls below a predetermined threshold.

20. The apparatus of claim 1 , wherein the control unit manages the data integrity of a plurality of archival storage units by:

identifying a data block stored at a first location on one of the plurality of archival storage units;

calculating an error correction code for the identified block and comparing it to a previous error correction code calculated when the data was first stored on the identified data block at the first location; and either:

re-writing the data block to the first location if the calculated error correction code is the same as the previous error correction code; or

determining if the error is correctable if the calculated error correction code is different than the previous error correction code; and

migrating the data stored on the identified data block to a second location on one of the plurality of archival storage units if the error is correctable.

21. The apparatus of claim 19 , further comprising logging the error if it is determined that the error is not correctable.

22. The apparatus of claim 19 , further comprising repeating the above for all of the data blocks stored on the plurality of archival storage units.

23. The apparatus of claim 19 wherein the data integrity of the plurality of archival storage units is checked with a periodicity P where:

P

=

1

p

⁢

max

⁢

{

S

BW

,

L

N

}

and S is the size of the archival storage unit and BW is the bandwidth of checking data integrity, N is the remaining number of starts-and-stops, L is the remaining spin lifetime and p is a desired percentage of the total lifetime and number of start and stops of the disks of the archival storage units.

24. The apparatus of claim 1 , further comprising a data integrity checker configured to check the integrity of the data stored on the disk.

25. The apparatus of claim 24 , wherein the data integrity checker checks the integrity of the data stored on the disk every period P where P is defined by

P

=

1

p

⁢

max

⁢

{

S

BW

,

L

N

}

,

where S is the size of the archival storage unit and BW is the bandwidth of the data integrity checker, N is the remaining number of starts-and-stops, L is the remaining spin lifetime, and p is a desired percentage of the total spin lifetime of the disk.

26. The apparatus of claim 24 , wherein the data integrity checker checks the integrity of the data with a periodicity of P where P is determined based on one or more of the following: the remaining number of start-and-stops of the disk or the remaining spin lifetime of the disk.

27. The apparatus of claim 24 , wherein the data integrity checker performs a data integrity check on the disk by:

identifying a data block stored at a first location on the disk;

calculating an error correction code for the identified data block and comparing it to a previous error correction code calculated when the data block was first stored at the first location; and either:

re-writing the data block to the first location if the calculated error correction code is the same as the previous error correction code; or

determining if the error is correctable if the calculated error correction code is different than the previous error correction code; and

migrating the data block a second location if the error is correctable; and

repeating the aforementioned for each data block stored on the disk.

28. The apparatus of claim 27 , wherein the second location is located on the disk.

29. The apparatus of claim 27 , wherein the second location is located on another disk.

30. The apparatus of claim 24 , wherein the data integrity checker is further configured to periodically check the integrity of the data stored on a plurality of the disks.

31. The apparatus of claim 1 further comprising a managing unit configured to manage the operation of the disk to extend the lifetime of the disk, the managing unit being configured to estimate the remaining spin lifetime of the disk and to determine when to spin up or spin down the disk to extend its lifetime based on the estimated remaining spin lifetime of the disk.

32. The apparatus of claim 31 , wherein the managing unit is further configured to estimate the remaining start and stop limit of the disk and determine when to spin up or spin down the disk to extend the lifetime of the disk based on the estimated remaining spin lifetime and the estimated remaining start and stop limit of the disk.

33. The apparatus of claim 32 , wherein the managing unit determines when to spin up or spin down the disk further by:

computing a threshold value for the disk, the threshold value being defined as substantially equal to the duration of time where the life cost of the disk spinning idle approximately equals the life cost of a spin up and spin down cycle of the disk;

comparing the threshold value for the disk when it is running to the elapsed time since the previous request to the disk, and

powering off or placing the disk into a standby mode if the elapsed time exceed the threshold value.

34. The apparatus of claim 33 , wherein the computing the threshold value further includes the consideration of the power cost of having the disk run idle versus the power cost of having the disk spin up and spin down.

35. The apparatus of claim 34 , wherein the threshold value is adjustable.

36. The apparatus of claim 33 , wherein the managing unit is further configured to generate a warning if either the remaining spin lifetime falls below a first minimum value or remaining start and stop limit falls below a second minimum value.

37. A method comprising:

providing an archival data storage system having an archival storage disk in a memory hierarchy having a primary storage, the archival data storage system configured to periodically archive data stored in the primary memory and to periodically provide archived data to the primary storage unit;

providing a data integrity checker within the archival storage system to check the integrity of the data stored on the archival storage disk; and

configuring the data integrity checker to check the integrity of the data stored on the archival storage disk; and

providing a control to control the archival storage unit, the control unit further configured to extend the lifetime of the archival storage disk by selectively controlling the archival storage disk using an algorithm intended to extend the longevity of the archival storage disk, the algorithm relying on the following parameters:

the cost of the archival storage unit;

the cost per watt to operate the archival storage unit;

the spin lifetime of the archival storage disk;

the total number of start-and-stops of the archival storage disk;

the time taken to spin up the archival storage disk;

the number of watts consumed for a read or write operation to the archival storage unit; and

the number of watts consumed for a spin up of the archival storage disk.

38. The method of claim 37 , wherein the data integrity checker checks the integrity of the data every period P where period P is defined by

P

=

1

p

⁢

max

⁢

{

S

BW

,

L

N

}

,

where S is the size of the archival data system, and BW is the bandwidth of the data integrity checker, N is the remaining number of starts-and-stops, L is the remaining spin lifetime, and p is a desired percentage of the total spin lifetime of the of the archival storage disk.

39. The method of claim 37 , wherein the period P is determined based on one or more of the following: the remaining number of start-and-stops of the archival storage disk or the remaining spin lifetime of the archival storage disk.

40. The method of claim 37 , wherein the data integrity checker performs a data integrity check on the archival storage disk by:

identifying a data block stored at a first location on the archival storage disk;

calculating an error correction code for the identified data block and comparing it to a previous error correction code calculated when the data block was first stored at the first location; and either:

re-writing the data block to the first location if the calculated error correction code is the same as the previous error correction code; or

determining if the error is correctable if the calculated error correction code is different than the previous error correction code; and

migrating the data block a second location if the error is correctable; and

repeating the aforementioned for each data block stored on the storage disk.

41. The method of claim 40 , wherein the second location is located on the archival storage disk.

42. The method of claim 40 , wherein the second location is located on another archival storage disk.

43. The method of claim 37 , further comprising:

providing a plurality of the archival storage disks in the archival storage system; and

configuring the data integrity checker to successively check the integrity of the data stored on the plurality of the archival storage disks.

44. A method comprising:

providing an archival data storage system having an archival storage disk in a memory hierarchy having a primary storage, the archival data storage system configured to archive data stored in the primary memory and to provide archived data to the primary storage unit;

providing a control unit the control unit further configured to control the archival data storage system, the control unit configured to extend the lifetime of the archival storage disk by selectively controlling the archival storage disk using an algorithm intended to extend the longevity if the archival storage disk, the algorithm relying on the following parameters:

the cost of the archival data storage system;

the cost per watt to operate the archival data storage system;

the spin lifetime of the archival storage disk;

the total number of start-and-stops of the archival storage disk;

the time taken to spin up the archival storage disk;

the number of watts consumed for read or write operation to the archival data storage system; and

the number of watts consumed for a spin up of the archival storage disk;

estimating the remaining spin lifetime of the archival storage disk in the archival data storage system using the algorithm; and

determining when to spin up or spin down the archival storage disk to extend the lifetime of the archival storage disk using the algorithm.

45. The method of claim 44 wherein the algorithm is further configured for:

estimating the remaining start and stop limit of the archival storage disk; and

determining when to spin up or spin down the archival storage disk to extend the lifetime of the archival storage disk based on the estimated remaining spin lifetime and the estimated remaining start and stop limit of the archival storage disk.

46. The method of claim 45 , wherein determining when to spin up or spin down the archival storage disk further comprises:

computing a threshold value for the archival storage disk, the threshold value being defined as substantially equal to the duration of time where the life cost of the archival storage disk spinning idle approximately equals the life cost of a spin up and spin down cycle of the archival storage disk;

comparing the threshold value for the archival storage disk when it is running to the elapsed time since the previous request to the archival storage disk; and

powering off or placing the archival storage disk into a standby mode if the elapsed time exceed the threshold value.

47. The method of claim 46 , wherein the computing the threshold value further includes the consideration of the power cost of having the archival storage disk run idle versus the power cost of having the archival storage disk spin up and spin down.

48. The method of claim 46 , wherein the threshold value is adjustable.

49. The method of claim 45 , wherein the algorithm is further configured for generating a warning if either the remaining spin lifetime of the archival storage disk falls below a first minimum value or the remaining start and stop limit of the archival storage disk falls below a second minimum value.

50. The apparatus of claim 1 , wherein the interconnect comprises one of the following types of network connections: fiber channel, Ethernet, or SCSI.