Secure storage management system and method
View Patent ↗A storage management system and method for managing access between a plurality of processes and a common store. In one embodiment, each individual process comprises data processing means, a cache for the temporary storage of data generated by the data processing means, and a control unit for managing the transferral of data between the cache and a common store. The control unit comprises a manager for monitoring the availability of storage locations in the store to receive and store data and for allocating data to available storage locations, an interface for transferring the allocated data to the available storage locations, and a locking arrangement for locking the store during data transfer in order to ensure exclusive access and thereby preserve data integrity.
1. A storage management system, comprising:
a shared store, accessible by each of a plurality of parties, that stores a first version of management data indicating the storage allocations of the shared store and a second version of management data indicating the storage allocations of the shared store, wherein the first version of management data is more up-to-date than the second version of management data;
a plurality of parties, each party having access to said shared store and each party comprising:
a data processor;
a cache that stores data generated by the data processor and the second version of management data indicating storage allocations of the shared store; and
a control unit that manages a first data transfer between the cache and the shared store, wherein said control unit operates independently without interfering with the access of another party to said shared store and wherein the control unit separates the data into data blocks for transfer to the shared store, the control unit being embedded in each party of said plurality of parties in a shared access network and comprising:
a manager that identifies one or more available storage locations to receive and store the data in the shared store and allocates the data to the one or more available storage locations to render allocated data, wherein allocating to the one or more available storage locations comprises determining the size of the data blocks and allocating the data blocks to a selected address and offset in the shared store;
an interface that manages a second data transfer of the allocated data to the one or more available storage locations; and
a locking mechanism that generates a lock and locks the shared store during the first data transfer and the second data transfer to ensure that each party obtains exclusive access by preventing access from other parties of the plurality of the parties and that data integrity is preserved.
2. The system according to claim 1 , wherein the manager comprises a fault tolerant mechanism that recovers uncorrupted data when data errors arise.
3. The system according to claim 1 , wherein the control unit further comprises a data verifier that detects the corruption of data.
4. The system according to claim 3 , wherein the manager is arranged to co-operate with the data verifier to cancel corrupted data.
5. The system according to claim 1 , wherein the control unit employs version control and in which the manager includes a data version manager to ensure that the data stored in the shared store includes a current data set and a previous data set.
6. The system according to claim 5 , wherein the control unit comprises a data verifier that checks for data corruption and in which the manager is responsive to the data verifier to cancel corrupted data and to revert to the previous version.
7. The system according to claim 1 , wherein the control unit further comprises a data encoder for encrypting data.
8. The system according to claim 1 , wherein the control unit comprises at least one map representing storage locations within the shared store.
9. The system according to claim 8 , wherein the at least one map includes a block map representing storage blocks in the shared store and an address map representing addresses for the storage blocks which have already been allocated.
10. The system according to claim 8 further comprising at least one list representing storage locations in the shared store which are currently available.
11. The system according to claim 10 , wherein the at least one list comprises a free locked list representing blocks in the shared store which have previously been allocated and are no longer needed but which are locked and cannot be overwritten.
12. The system according to claim 10 , wherein the at least one list comprises a free offset list representing blocks in the shared store which have not yet been allocated, and a free address list representing addresses for blocks in the shared store which have previously been allocated and are now free for re-use.
13. The system according to claim 1 , wherein the control unit employs a garbage collection policy.
14. A shared access network comprising:
a plurality of parties,
a shared store, accessible by said plurality of parties, that stores a first version of management data indicating the storage allocations of the shared store and a second version of management data indicating the storage allocations of the shared store, wherein the first version of management data is more up-to-date than the second version of management data, and
each of said parties comprising a storage management system, the storage management system comprising:
a data processor;
a cache that stores data generated by the data processor and the second version of management data indicating storage allocations of the shared store: and
a control unit that manages a first data transfer between the cache and a shared store, wherein said control unit operates independently without interfering with the access of another party to said shared store and wherein the control unit separates the data into data blocks for transfer to the shared store, the control unit being embedded in each party of said plurality of parties in the shared access network and comprising:
a manager that identifies one or more available storage locations to receive and store data in the shared store and allocates the data to one or more available storage locations to render allocated data, wherein allocating to the one or more available storage locations comprises determining the size of the data blocks and allocating the data blocks to a selected address and offset in the shared store;
an interface that manages a second data transfer of the allocated data to the one or more available storage locations; and
a locking mechanism that generates a lock and locks the shared store during the first data transfer and the second data transfer in order to ensure that each party obtains exclusive access by preventing access from other parties of the plurality of the parties and that data integrity is preserved.
15. The network according to claim 14 , wherein the manager comprises a fault tolerant mechanism that recovers uncorrupted data when data errors arise.
16. The network according to claim 14 , wherein the control unit further comprises a data verifier that detects the corruption of data.
17. The network according to claim 16 , wherein the manager is arranged to co-operate with the data verifier to cancel corrupted data.
18. The network according to claim 14 , wherein the control unit employs version control for the data storage and in which the manager includes a data version manager to ensure that data stored in the shared store includes a current data set and a previous data set.
19. The network according to claim 18 , wherein the control unit comprises a data verifier that checks for data corruption and in which the manager is responsive to the data verifier to cancel corrupted data and to revert to the previous version.
20. The network according to claim 14 , wherein the control unit further comprises a data encoder for encrypting data.
21. The network according to claim 14 , wherein the control unit comprises at least one map representing storage locations within the shared store.
22. The network according to claim 21 , wherein the at least one map includes a block map representing storage blocks in the shared store and an address map representing addresses for the storage blocks which have already been allocated.
23. The network according to claim 21 further comprising at least one list representing storage locations in the shared store which are currently available.
24. The network according to claim 23 , wherein the at least one list comprises a free locked list representing blocks in the shared store which have previously been allocated and are no longer needed but which are currently locked and cannot be overwritten.
25. The network according to claim 23 , wherein the at least one list comprises a free offset list representing blocks in the shared store which have not yet been allocated, and a free address list representing addresses for blocks in the shared store which have been previously allocated and are now free for re-use.
26. The network according to claim 14 , wherein the control unit employs a garbage collection policy.
27. A method of storage management comprising:
generating data with a data processor in each party of a plurality of parties in a shared access network, each party of a plurality of parties having an embedded control unit, wherein said control unit operates independently without interfering with the access of another party to a shared store;
storing in a shared store, accessible by each of said parties, a first set of management data indicating the storage allocations of the shared store and a second set of management data indicating the storage allocations of the shared store, wherein the first version of management data is more up-to-date than the second version of management data;
said control unit of each party configured for:
storing in a cache the data generated by the data processor and the second version of management data indicating the current storage allocations of the shared store;
managing a first data transfer between the cache and the shared store;
monitoring the availability of storage locations in the shared store to receive and store the data;
allocating the data to one or more available storage locations to render allocated data, said allocating comprising separating the data into data blocks for transfer to the shared store, determining the size of the data blocks, and allocating the data blocks to a selected address and offset in the shared store;
managing a second data transfer of the allocated data and to one or more available storage locations; and
locking the shared store with a lock generated by the control unit of the party during the first data transfer and the second data transfer in order to ensure that each party obtains exclusive access by preventing access from other parties of the plurality of the parties and that data integrity is preserved.
28. The method according to claim 27 further comprising recovering uncorrupted data when data errors arise.
29. The method according to claim 27 , further comprising detecting the corruption of the data.
30. The method according to claim 29 further comprising cancelling corrupted data.
31. The method according to claim 29 further comprising employing version control for the data storage and storing in the shared store a current set of data and a previous set of data.
32. The method according to claim 31 further comprising checking for data corruption and in the event of data corruption cancelling the corrupted data and reverting to the previous set of data.
33. The method according to claim 27 further comprising encrypting data to be stored.
34. The method according to claim 27 further comprising employing a garbage collection policy for reducing the size of the shared store.