Scalable data storage architecture and methods of eliminating I/O traffic bottlenecks
View Patent ↗A Storage Area Network (SAN) system has host computers, front-end SAN controllers (FE_SAN) connected via a bus or network interconnect to back-end SAN controllers (BE_SAN), and physical disk drives connected via network interconnect to the BE_SANs to provide distributed high performance centrally managed storage. Described are hardware and software architectural solutions designed to eliminate I/O traffic bottlenecks, improve scalability, and reduce the overall cost of SAN systems. In an embodiment, the BE_SAN has firmware to recognize when, in order to support a multidisc volume, such as a RAID volume, it is configured to support, it requires access to a physical disk attached to a second BE_SAN; when such a reference is recognized it passes assess commands to the second BE_SAN. Buffer memory of each FE_SAN is mapped into application memory space to increase access speed, where multiple hosts share an LBA the BE_SAN tracks writes and invalidates the unwritten buffers.
1. A data processing system having a storage area network (SAN), the system comprising:
at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first CPU configured to execute first FE_SAN virtualization firmware, and the second CPU configured to execute second FE_SAN virtualization firmware;
at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices;
a network interconnecting the FE_SANs with the BE_SANs; and
BE_SAN virtualization firmware executing on the BE_SANs;
wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and
wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory;
wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN;
wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer.
2. The system of claim 1 wherein the FE_SAN and BE_SAN firmware is configured for writeback caching without mirroring.
3. A data processing system having a storage area network (SAN), the system comprising:
at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first FE_SAN controller and first CPU configured to execute first FE_SAN virtualization firmware, and the second FE_SAN controller and second CPU configured to execute second FE_SAN virtualization firmware;
at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices;
a network interconnecting the FE_SANs with the BE_SANs; and
BE_SAN virtualization firmware executing on the BE_SANs;
wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and
wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory;
wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN;
wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's physical memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer; and
wherein the FE_SAN and BE_SAN firmware is configured for writeback caching with mirroring.
4. A data processing system having a storage area network (SAN), the system comprising:
at least a first and a second host processor central units (CPUs), the first CPU coupled to a first FE_SAN (Front End SAN) controller and the second CPU coupled to a second FE_SAN controller, the first FE_SAN controller and first CPU configured to execute first FE_SAN virtualization firmware, and the second FE_SAN controller and second CPU configured to execute second FE_SAN virtualization firmware;
at least a first and a second Back End SAN controller (BE_SAN), each BE_SAN being interconnected via at least one back-end network to a plurality of storage devices;
a network interconnecting the FE_SANs with the BE_SANs; and
BE_SAN virtualization firmware executing on the BE_SANs;
wherein the FE_SAN virtualization firmware and the BE_SAN virtualization firmware are together configured to request data from a particular BE_SAN of the at least a first and a second BE_SANs when data is required by a particular requesting host CPU of the host CPUs; and
wherein an operating system (OS) executing on a first CPU maps a portion of physical memory as a first buffer of a first logical block address (LBA) into an application's address space allowing the application to directly access that memory;
wherein a copy-on-write method is used to automatically copy data from the first buffer to at least one corresponding buffer allocated on the first BE_SAN;
wherein, if an operating system executing a host maps a portion of the second FE_SAN controller's physical memory as a second buffer of the first LBA into an application's address space, and if data of the first buffer is modified by a host coupled to the first FE_SAN, the first BE_SAN controller notifies the second FE_SAN to invalidate data of the second buffer; and
wherein the FE_SAN and BE_SAN firmware is configured for writeback caching with and without mirroring.
5. The system of claim 4 wherein the FE_SAN and BE_SAN firmware is configured for writeback caching without mirroring.