Method and apparatus for implementing high-performance, scaleable data processing and storage systems
View Patent ↗A data system architecture is described that allows multiple processing and storage resources to be connected to multiple clients so as 1) to distribute the clients' workload efficiently across the available resources; and 2) to enable scaleable expansion, both in terms of the number of clients and in the number of resources. The major features of the architecture are separate, modular, client and resource elements that can be added independently, a high-performance cross-bar data switch interconnecting these various elements, separate serial communication paths for controlling the cross-bar switch settings, separate communication paths for passing control information among the various elements and a resource utilization methodology that enables clients to distribute processing or storage tasks across all available resources, thereby eliminating “hot spots” resulting from uneven utilization of those resources.
1. Apparatus for providing high-performance, scaleable data storage services to a client from a plurality of storage devices, comprising:
an access interface module which receives from the client data storage requests, each including a data object to be stored and a data object identifier that identifies that data object and, in response to each storage request and based on a workload and on relative demands placed on subsets of the plurality of storage devices instead of a physical location in the plurality of storage devices, dynamically selects a subset of the plurality of storage devices in which the data is stored so that data corresponding to the same data object identifier can be transferred to different physical storage device locations from request to request in order to dynamically distribute the workload across the plurality of storage devices; and
a switch fabric for temporarily connecting the access interface module to the selected subset of the plurality of storage devices so that the data can be transferred to the selected subset of storage devices.
2. The apparatus of claim 1 wherein the switch fabric comprises a control switch fabric for transferring control information and a separate data switch fabric for transferring data.
3. The apparatus of claim 2 wherein the control switch fabric is optimized for transferring control information and the data switch fabric is optimized for transferring data.
4. The apparatus of claim 2 wherein the request for storage includes control information and data and wherein the access interface module separates the control information and the data and transfers the data to the selected subset of storage devices over the data switch fabric.
5. The apparatus of claim 2 wherein the data switch fabric comprises a non-blocking crossbar switch for data transfer and the control switch fabric comprises an Ethernet switch for control information transfer.
6. The apparatus of claim 1 further comprising a resource module connected to the plurality of storage devices for generating preallocation information that preallocates storage from the plurality of storage devices in order to evenly distribute a workload across the plurality of storage devices.
7. The apparatus of claim 6 wherein the switch fabric connects the access interface module to the resource module so that the resource module can transfer the preallocation information to the access interface module.
8. The apparatus of claim 7 wherein the access interface module selects a subset of the plurality of storage devices based on the preallocation information.
9. The apparatus of claim 1 wherein the access interface module comprises a data memory which temporarily stores information transferred between the access interface module and the selected subset of the plurality of storage devices.
10. The apparatus of claim 1 further comprising a plurality of access interface modules each access interface module receiving storage requests from a plurality of clients.
11. A method for providing high-performance, scaleable data storage services to a client from a plurality of storage devices, the method comprising
(a) providing an access interface module which receives from the client data storage requests, each including a data object to be stored and a data object identifier that identifies that data object;
(b) using the access interface module in response to each data storage request and based on a workload and on relative demands placed on subsets of the plurality of storage devices instead of a physical location in the plurality of storage devices, dynamically selects a subset of the plurality of storage devices in which the data is stored so that data corresponding to the same data object identifier can be transferred to different physical storage device locations from request to request in order to dynamically distribute the workload across the plurality of storage devices; and
(c) using a switch fabric to temporarily connect the access interface module to the selected subset of the plurality of storage devices so that the data can be transferred to the selected subset of storage devices.
12. The method of claim 11 wherein step (c) comprises:
(c1) using a control switch fabric for transferring control information; and
(c2) using a separate data switch fabric for transferring data.
13. The method of claim 12 wherein step (c1) comprises optimizing the control switch fabric for transferring control information and step (c2) comprises optimizing the data switch fabric for transferring data.
14. The method of claim 12 wherein the request for storage includes control information and data and wherein step (b) comprises separating the control information and the data and step (c) comprises transferring the data to the selected subset of storage devices over the data switch fabric.
15. The method of claim 12 wherein step (c1) comprises using a non-blocking crossbar switch for data transfer and step (c2) comprises using an Ethernet switch for control information transfer.
16. The method of claim 11 further comprising:
(d) providing a resource module connected to the plurality of storage devices; and
(e) using the resource module to generate preallocation information that preallocates storage from the plurality of storage devices in order to evenly distribute a workload across the plurality of storage devices.
17. The method of claim 16 wherein step (c) comprises connecting the access interface module to the resource module so that the resource module can transfer the preallocation information to the access interface module.
18. The method of claim 17 wherein step (b) comprises selecting a subset of the plurality of storage devices based on the preallocation information.
19. The method of claim 11 wherein step (b) comprises temporarily storing information transferred between the access interface module and the selected subset of the plurality of storage devices.
20. The method of claim 11 wherein step (a) further comprises providing a plurality of access interface modules each access interface module receiving storage requests from a plurality of clients.