File server storage arrangement
The invention provides a storage system that is highly available even in the face of component failures in the storage system, and a method for operating that storage system. A first and a second file server each includes a file server request log for storing incoming file server requests. Both the first and second file servers have access to a common set of mass storage elements. Each incoming file server request is copied to both the first and second file servers; the first file server processes the file server request while the second file server maintains a copy in its file server request log. Each file server operates using a file system that maintains consistent state after each file server request. On failover, the second file server can perform those file server requests in its file server request log since the most recent consistent state. There is no single point of failure that prevents access to any individual mass storage element.
1. A computer-readable memory storing information including instructions, the instructions executable by at least one processor to operate a file system, the instructions, when executed by the at least one processor, causing the at least one processor to perform steps comprising:
responding to an incoming file server request by copying a descriptor of a file system change to both a first file server and a second file server;
processing said file system change at said first file server while maintaining said descriptor copy at said second file server; and
performing, at said second file server, a file system change in response to a copied descriptor and a service interruption by said first file server.
2. A memory as in claim 1 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of associating said first file server and said second file server with a mass storage element.
3. A memory as in claim 2 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of delaying output by at least one said file server to said mass storage element without delaying a response to file system changes.
4. A memory as in claim 1 , wherein said first file server is disposed for processing said file system changes atomically, whereby said second file server can on failover process exactly those file system changes not already processed by said first file server.
5. A memory as in claim 2 , wherein said mass storage element includes a file storage system and each said file server is disposed for leaving said file storage system in an internally consistent state after processing each said file system change.
6. A memory as in claim 2 , wherein said steps of performing a file system change in response to a copied descriptor are also operative in response to a service interruption by said second file server.
7. A memory as in claim 1 , wherein said first file server is disposed to respond identically to service interruptions for itself and for said second file server.
8. A memory as in claim 1 , wherein at least one said file server responds to said file system change before committing a result of said file system change to mass storage.
9. A memory as in claim 1 , wherein
each said file server includes a file server change memory;
each one of said file servers is coupled to at least a portion of said file server change memory included in the other one of said file servers using local memory access; and
each one of said file servers is coupled to at least a portion of said file server change memory included in the other one of said file servers using remote memory access.
10. A memory as in claim 9 , wherein said file server change memory includes a disk block.
11. A memory as in claim 9 , wherein said file server change memory includes a file server request.
12. A memory as in claim 9 , wherein said file server change memory is disposed to delay output to a mass storage element without delaying a response to file server requests.
13. A memory as in claim 1 , wherein said step of processing includes steps of processing said file system change at both a primary mass storage element and a mirror mass storage element.
14. A computer-readable memory storing information including instructions, the instructions executable by at least one processor to operate a file system, the instructions, when executed by the at least one processor, causing the at least one processor to perform steps comprising:
receiving a file server request at one of a plurality of file servers and in response thereto copying a descriptor of a file system change into a server change memory;
processing said file system change for both a first set of mass storage devices coupled to a first one of said file servers and for at least one mass storage device in a second set of mass storage devices coupled to a second one of said file servers.
15. A memory as in claim 14 , wherein said descriptor includes a file server request.
16. A memory as in claim 14 , wherein said server change memory includes a disk block.
17. A memory as in claim 14 , wherein said server change memory includes a file server request.
18. A memory as in claim 14 , wherein said server change memory includes a first portion disposed at said first file server and a second portion disposed at said second file server.
19. A memory as in claim 14 , wherein said server change memory includes a first portion disposed at said first file server and a second portion disposed at said second file server; and wherein said steps of copying include steps of copying said descriptor into both said first portion and said second portion.
20. A memory as in claim 14 , wherein said server change memory includes a first portion disposed at said first file server and a second portion disposed at said second file server; and said steps of copying include steps of copying said descriptor into both said first portion and said second portion by either of said first file server or said second file server.
21. A memory as in claim 14 , wherein said server change memory is disposed to delay output to said at least one mass storage device without delaying a response to file server requests.
22. A memory as in claim 14 , wherein
said steps of receiving include receiving a file server request at either said first file server or said second file server, and said steps of copying said descriptor include copying by either said first file server or said second file server; and wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform step of
processing said file system change for both said second set of mass storage devices and for at least one mass storage device in said first set.
23. A computer-readable memory storing information including instructions, the instructions executable by at least one processor to operate a file system, the instructions, when executed by the at least one processor, causing the at least one processor to perform steps comprising:
receiving a file server request at one of a plurality of file servers and in response thereto copying a descriptor of a file system change into a file server change memory; and
responding to a service interruption by performing a file system change in response to a descriptor in said file server change memory.
24. A memory as in claim 23 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of associating said plurality of file servers with at least one mass storage element and at least one file server change memory.
25. A memory as in claim 24 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of leaving a file storage system on said mass storage element in an internally consistent state after processing each said file system change.
26. A memory as in claim 24 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of:
leaving a file storage system on said mass storage element in an internally consistent state after processing file system changes;
associating said internally consistent state with a set of completed file system changes; and
identifying said set of completed file system changes by at least one said file server.
27. A memory as in claim 23 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of delaying output to a mass storage element without delaying a response to file server requests.
28. A memory as in claim 23 , the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of performing said received file server request at both a primary mass storage element and a mirror mass storage element.
29. A memory as in claim 23 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of:
processing said file system changes atomically at a first said file server; and
on failover processing exactly those file system changes not already processed by said first file server.
30. A memory as in claim 23 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of responding identically at a first said file server to service interruptions for itself and for a second said file server.
31. A memory as in claim 23 , wherein said file server change memory includes a disk block.
32. A memory as in claim 23 , wherein said file server change memory includes a file server request.
33. A memory as in claim 23 , wherein said file server change memory is disposed to delay output to a mass storage element without delaying a response to file server requests.
34. A memory as in claim 23 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform steps of responding to a file system change before committing a result of said file system change to mass storage at one of said file servers.