Storage processor architecture for high throughput applications providing efficient user data channel loading
A storage processor particularly suited to RAID systems provides high throughput for applications such as streaming video data. An embodiment is configured as an ASIC with a high degree of parallelism in its interconnections. The communications architecture provides saturation of user data pathways with low complexity and low latency by employing multiple memory channels under software control, an efficient parity calculation mechanism and other features.
1. A storage processor, comprising:
at least one switch connecting a front end interface connectable to host and a back end interface connectable to a data store, said front and back ends each having a channel connecting it to said at least one switch, each said front end and back end channels having respective bandwidths;
at least two memory interfaces each with a respective channel permitting simultaneous read and writes to memories connected thereto;
a data translator connected to transfer data to and from said memories through each of said two memory interfaces;
said at least one switch being configured to provide for transfer of write data from said front end to said back end by storing said write data in a selected subset of said memories, generating translated data responsively to said write data stored in said selected subset, storing store data responsive to said translated data and said write data in said selected subset or another subset of said memories;
said at least one switch being further configured such that a rate of transfer of said write data through said front end channel is substantially equal to said bandwidth of said front end channel;
wherein said at least one switch, when generating translated data, is configured to perform an error-checking process on the write data stored in said selected subset of said memories to generate error-checking data and, when storing store data, is configured to store said store data responsive to said error-checking data and said write data in said selected subset or another subset of said memories;
wherein data is transferred through the switch by writing to selected ones of said memories and reading from said selected ones of said memories in a single hop between said front end and said back end such that a load on said memories is balanced.
2. A storage processor as in claim 1 , wherein said data translator includes a parity calculator.
3. A storage processor as in claim 1 , wherein said selected one of said memories is selected by software configured to share bandwidths of said at least two memory interfaces, whereby said bandwidths of said at least two memory interfaces may be less than would be required for a single memory interface.
4. A storage processor as in claim 3 , further comprising a control processor connected to said at least one switch to control said data translator.
5. A storage processor as in claim 4 , wherein said at least one switch is configured to transfer requests from said front end to said control processor through said at least one switch.
6. A storage processor as in claim 3 , wherein all of said at least two memories are connected to read and write to and from both of said front and back end interfaces.
7. A storage processor as in claim 1 further comprising a control processor connected to said at least one switch to control said data translator.
8. A storage processor as in claim 1 wherein said at least one switch is configured to transfer requests from said front end to said control processor through said at least one switch.
9. A storage processor as in claim 1 , wherein all of said at least two memories are connected to read and write to and from both of said front and back end interfaces.
10. A storage processor for a redundant array of independent disks, comprising:
an ASIC including a communications fabric connecting a front end connectable to a communications device generating a requesting process to a back end connectable to an array of disks, each of said front and back ends being connected to said communications fabric by a port having a respective bandwidth;
said communications fabric including multiple memories, a parity calculator, a control processor, and a DMA engine connected by said communications fabric for control by said control processor;
said memories each having a bandwidth less than a bandwidth of said front end port bandwidth multiplied by a number of reads and writes required to transfer data to and from said front end port;
said communications fabric providing for transfer of control data from said requesting process to said control processor;
said communications fabric providing for serial transfer of data from said front end to said back end interfaces such that data from said requesting process is written with parity data responsive thereto to said redundant array at a rate that is substantially equal to said front end port bandwidth;
wherein data is transferred through said communications fabric by writing to selected ones of said multiple memories and reading from said selected ones of said multiple memories in a single hop between said front end and said back end; and
wherein said control processor controls which of said multiple memories is included in said selected ones of said multiple memories such that a load on said multiple memories is balanced.
11. A storage processor as in claim 10 , wherein said parity calculator includes an XOR FIFO.
12. A storage processor as in claim 10 , wherein all of said multiple memories are connected to read and write to and from both of said front and back end interfaces.
13. A storage processor as in claim 10 , wherein all of said multiple memories are connected to read and write to and from both of said front and back end interfaces.
14. The storage processor of claim 1 , wherein the at least one switch and the data translator form part of an ASIC, and wherein the data translator comprises a DMA engine and an XOR FIFO that link the front end interface and the back end interface with a memory.
15. The storage processor of claim 11 , wherein the communications fabric of the ASIC comprises a crossbar switch coupled to the DMA engine and the XOR FIFO to link the front end interface and the back end interface with one of the multiple memories.
16. The storage processor of claim 11 , wherein the DMA engine and the XOR FIFO are integrated as part of the ASIC.
17. The storage processor of claim 1 , wherein the memories connected to the at least to memory interfaces comprise non-cache memories.
18. The storage processor of claim 10 , wherein the multiple memories of the communications fabric comprise non-cache memories.