Scalable data access system and methods of eliminating controller bottlenecks
A data access system has host computers having front-end controllers nFE_SAN connected via a bus or network interconnect to back-end storage controllers nBE_SAN, and physical disk drives connected via network interconnect to the nBE_SANs to provide a distributed, high performance, policy based or dynamically reconfigurable, centrally managed, data storage acceleration system. The hardware and software architectural solutions eliminate BE_SAN controller bottlenecks and improve performance and scalability. In an embodiment, the nBE_SAN (BE_SAN) firmware recognize controller overload conditions, informs Distributed Resource Manager (DRM), and, based on the DRM provided optimal topology information, delegates part of its workload to additional controllers. The nFE_SAN firmware and additional hardware using functionally independent and redundant CPUs and memory that mitigate single points of failure and accelerates write performance. The nFE_SAN and FE_SAN controllers facilitate Converged I/O Interface by simultaneously supporting storage I/O and network traffic.
1. A method of writing write data to at least one storage device comprising:
generating a write request in a first process of a host computer;
passing the write request to FE_SAN driver in the host computer;
passing the write request from the FE_SAN driver to a front-end storage controller (nFE_SAN);
copying the write data to a cache memory of the FE_SAN driver;
copying the write data to a cache memory of the nFE_SAN;
generating a write lock request and transmitting the write lock request from the nFE_SAN over a network interconnect selected from a first and a second storage area interconnect to a back-end storage controller (nBE_SAN);
returning a write lock grant from the nBE_SAN to the nFE_SAN;
upon completing copying the write data to the cache memory of the FE_SAN driver and the cache memory of the nFE_SAN and receiving the write lock grant from the nBE_SAN, the FE_SAN driver providing a write complete signal to the first process of the host computer processor;
copying the write data over a network interconnect selected from the first and second storage area interconnect to the nBE_SAN;
writing, by the BE_SAN, the write data to the at least one storage device;
the write complete signal being provided to the first process of the computer processor before the copying of the write data over the network interconnect from the nFE_SAN to the nBE_SAN completes.
2. The method of claim 1 wherein, upon failure of the cache memory of the FE_SAN driver, the write data is copied from the cache memory of the nFE_SAN to the nBE_SAN.
3. A method of writing write data to at least one storage device comprising:
generating a write request in a host process of a computer;
passing the write request to a front-end storage controller (nFE_SAN);
copying the write data to a cache memory of an FE_SAN driver in the computer;
copying the write data to a cache memory of the nFE_SAN;
generating a write lock request and transmitting the write lock request from the nFE_SAN over a network interconnect selected from a first and a second storage area interconnect to a selected back-end storage controller (nBE_SAN), the nBE_SAN selected from a first and a second nBE_SAN of a plurality of nBE_SANs according to logical block addresses (LBAs) identification associated with the write request, where a first of the nBE_SANs is assigned to write data associated with a first and second LBAs, and a second nBE_SAN is assigned to write data associated with third LBAs, the first, second, and third LBAs being different and the first nBE_SAN being different from the second nBE_SAN;
returning a write lock grant from the selected nBE_SAN to the nFE_SAN;
upon completing copying the write data to the cache memory of the FE_SAN driver and the cache memory of the nFE_SAN and receiving the write lock grant from the selected nBE_SAN, the nFE_SAN providing a write complete signal to the computer processor without waiting for copying the write data to the nBE_SAN to complete;
copying the write data over a network interconnect selected from the first and second storage area interconnect to the selected nBE_SAN; and
writing, by the selected BE_SAN, the write data to the at least one storage device.
4. The method of claim 3 wherein, upon detecting overload of the first nBE_SAN by frequent accesses to the second LBAs, copying data associated with the second LBAs from the first nBE_SAN to the second nBE_SAN and directing further references to the second LBAs to the second nBE_SAN.