Method and apparatus for efficient scalable storage management
View Patent ↗A hybrid centralized and distributed processing system includes a switching device that connects a storage processor to one or more servers through a host channel processor. The switching device also connects the storage processor to one or more storage devices such as disk drive arrays, and to a metadata cache and a block data cache memory. The storage processor processes access request from one or more servers in the form of a logical volume or logical block address and accesses the metadata cache to determine the physical data address. The storage processor monitors the performance of the storage system and performs automatic tuning by reallocating the logical volume, load balancing, hot spot removal, and dynamic expansion of storage volume. The storage processor also provides fault-tolerant access and provides parallel high performance data paths for fail over. The storage processor also provides faster access by providing parallel data paths for, making local copies and providing remote data copies, and by selecting data from a storage device that retrieves the data the earliest.
1. A storage controller for controlling a storage system for exchanging data between at least one host device and at least one storage device, the storage-controller comprising:
a metadata cache memory operable to store virtualization tables to determine the memory address of stored information based on a virtual address of the stored information;
a block cache memory controlled by said storage processor and operable to store a predetermined set of information for rapid access to a host device through said switching device;
a switching device coupled to a host device, a storage device, the metadata cache and the block cache memory;
a first storage processor coupled to the switching device for controlling a transfer of data from the metadata cache and the block cache memory via the switching device; and
a second storage processor coupled to the switching device for controlling a transfer of data from the metadata cache and the block cache memory via the switching device, wherein the second storage processor is controlling a transfer of data from the metadata cache via the switching device at the same time as the first storage processor is controlling a transfer of data from the block cache memory via the switching device.
2. The storage controller of claim 1 , wherein said host device comprises a server.
3. The storage controller of claim 1 , wherein said at least one storage device includes at least one disk drive array.
4. The storage controller of claim 1 , wherein said switching device is connected to said at least one host device through at least one host channel processor.
5. The storage controller of claim 4 , wherein said at least one host channel processor is implemented through firmware.
6. The storage controller of claim 1 , wherein said switching device is connected to said at least one storage device through at least one storage device channel processor.
7. The storage controller of claim 6 , wherein said at least storage device channel processor is implemented through firmware.
8. The storage controller of claim 1 , wherein said switching device is comprised of an active switching system.
9. The storage controller of claim 8 , wherein said active switching system is a Hyper Transport data switch that is connected to said at least one host device and said at least one storage device through a Hyper Transport bus.
10. The storage controller of claim 1 , wherein said switching device is connected to said at least one storage device through a Fibre Channel processor.
11. The storage controller of claim 1 , wherein said storage processor collects information on disk access, usage, and failure and enables storage resource management (SRM) and storage area management (SAM) functions.
12. The storage controller of claim 11 , wherein the SRM and SAM functions are not specified by a policy external to the storage controller.
13. The storage controller of claim 11 , wherein the SRM and SAM functions are specified by a policy external to the storage controller.
14. The method of claim 11 , wherein the SRM and SAM functions are specified by a policy external to a storage controller.
15. The storage controller of claim 1 , wherein the storage processor provides parallel high performance data paths for at least one of: fail over, making local copies and providing remote data copies.
16. The storage controller of claim 1 , wherein the storage processor selects the data read from a storage device among a plurality of storage devices that retrieves the data first, thereby minimizing the time to first byte.
17. The storage controller of claim 1 , wherein, as volume or block data is spread across multiple storage devices, the storage processor per forms automated performance tuning, including at least one of: load balancing, and hot spot removal.
18. The storage controller of claim 1 , wherein the storage processor uses metadata to enforce secure access to storage at a block or a LUN level so that access rights may be enforced.
19. The storage controller of claim 1 , wherein the storage processor uses metadata processing and channel processing for performing an automated fail over in case of storage device failure.
20. The storage controller of claim 1 , wherein the storage processor uses metadata processing and storage device channel processing for performing at least one of: automated storage volume, block allocation, and dynamic expansion of storage volume for storage applications.
21. The storage controller of claim 1 , wherein the metadata cache memory and block cache memory are independently scalable in size and performance to match system requirements.
22. The storage controller of claim 1 , wherein the metadata cache and block cache may be combined in a single unified memory or distributed over multiple coherent or non coherent memories.
23. The storage controller of claim 1 , further comprising
redundant storage processors connected to said storage processor through one or more switching devices.
24. The storage controller of claim 1 , wherein said switching device is connected to said at least one storage device through a serial protocol, including at least one of Hyper Transport, Fibre Channel and InfiniBand protocols.
25. The storage controller of claim 1 , further comprising:
a host channel processor;
a third storage processor coupled to the switching device for receiving a command from the channel processor via the switching device at the same time as the first and second storage processors are controlling a transfer of data via the switching device.
26. The storage controller of claim 25 , wherein the command includes a status command.
27. The storage controller of claim 25 , wherein the command includes a control command.
28. The storage controller of claim 1 , further comprising:
a host channel processor coupled between a host and the switching device;
a fourth storage processor coupled to the switching device for receiving and processing a non-I/O request from the host channel processor via the switching device at the same time as the first and second storage processors are controlling a transfer of data via the switching device.
29. A method for accessing a storage device, the method comprising:
receiving a request for a read operation, wherein the request includes a reference to a virtual storage space;
accessing a metadata cache memory with a first processor via a switch to obtain one or more logical locations corresponding to the virtual storage space;
determining that at least a part of the one or more logical locations is mapped to a block cache memory; and
transferring data from the at least a part of the one or more logical locations from the block cache memory to the first processor via the switch concurrently with transferring data from the metadata cache to a second processor via the switch.
30. The method of claim 29 , wherein the data transferred from the metadata cache is in association with a read request.
31. The method of claim 29 , wherein the data transferred from the metadata cache is in association with a write request.
32. The method of claim 29 , further comprising:
performing storage resource management (SRM) and storage area management (SAM) functions.
33. The method of claim 32 , wherein the SRM and SAM functions are not specified by a policy external to the storage controller.
34. A method for accessing a storage device, the method comprising:
transferring data from a metadata cache memory to a first processor via a switch to obtain one or more logical locations corresponding to a virtual storage space;
determining that at least a part of the one or more logical locations is mapped to a block cache memory; and
transferring data from the at least a part of the one or more logical locations from the block cache memory to the first processor via the switch concurrently with transferring data via the switch from the metadata cache memory to a second processor for a subsequent data operation.
35. The method of claim 34 , wherein the subsequent data operation includes a write operation.
36. The method of claim 35 , wherein the subsequent data operation includes a read operation.
37. The method of claim 35 , further comprising:
performing storage resource management (SRM) and storage area management (SAM) functions.
38. The method of claim 37 , wherein the SRM and SAM functions are not specified by a policy external to the storage controller.
39. The method of claim 37 , wherein the SRM and SAM functions are specified by a policy external to a storage controller.