Data storage architecture and system for high performance computing incorporating a distributed hash table and using a hash on metadata of data items to obtain storage locations
Data storage systems and methods for storing data are described herein. The storage system may be integrated with or coupled with a compute cluster or super computer having multiple computing nodes. A plurality of nonvolatile memory units may be included with computing nodes, coupled with computing nodes or coupled with input/output nodes. The input/output nodes may be included with the compute cluster or super computer, or coupled thereto. The nonvolatile memory units store data items provided by the computing nodes, and the input/output nodes maintain where the data items are stored in the nonvolatile memory units via a hash table distributed among the input/output nodes. The use of a distributed hash table allows for quick access to data items stored in the nonvolatile memory units even as the computing nodes are writing large amounts of data to the storage system quickly in bursts.
1. A compute cluster comprising:
a plurality of computing nodes each coupled with a high speed interconnect and with a local interconnect, wherein the high speed interconnect conforms to one of the INFINIBAND, OMNI-PATH, CASCADE or GEMINI architectures, and wherein the local interconnect conforms to one of a serial attached storage (SAS) standard, a peripheral component interconnect express (PCI Express) standard, or a DRAM-like protocol;
a plurality of nonvolatile memory units coupled with the local interconnect and accessible to the computing nodes via the local interconnect, wherein the nonvolatile memory units comprise at least one selected from the group including solid state drives (SSDs), flash memory, NAND-based flash memory, phase change memory, and spin torque memory;
a plurality of input/output nodes coupled with the high speed interconnect, each input/output node including a portion of a distributed hash table to maintain a data item location of data items provided by the computing nodes, the distributed hash table indexed according to a hash on metadata for the data items, wherein each input/output node is a server computer.
2. The compute cluster of claim 1 wherein a number of computing nodes exceeds a number of nonvolatile memory units.
3. The compute cluster of claim 1 wherein a number of nonvolatile memory units exceeds a number of input/output nodes.
4. The compute cluster of claim 1 wherein each computing node has multiple cores.
5. The compute cluster of claim 1 wherein the metadata includes a data item identifier and an offset for the data item.
6. The compute cluster of claim 1 wherein each entry in the distributed hash table includes either an address in one of the nonvolatile memory units where the data item is stored or an indication the data item is stored in a primary storage.
7. The compute cluster of claim 1 further comprising:
a primary storage including a plurality of storage servers, wherein the input/output nodes are coupled with the primary storage via a network.
8. The compute cluster of claim 1 wherein the high speed interconnect comprises an optical fiber technology.
9. The compute cluster of claim 1 wherein the compute cluster is a super computer.
10. A compute cluster comprising:
a plurality of computing nodes each coupled with a high speed interconnect and with a local interconnect, wherein the high speed interconnect conforms to one of the INFINIBAND, OMNI-PATH, CASCADE or GEMINI architectures;
a plurality of nonvolatile memory units coupled with the local interconnect to serve as a burst buffer to receive large amounts of data at high speed from the computing nodes;
a plurality of input/output nodes coupled with the high speed interconnect, wherein each input/output node is a server computer;
a distributed hash table stored in portions among the input/output nodes to maintain a data item location of data items provided by the computing nodes, the distributed hash table indexed according to a hash on metadata for the data items, wherein each entry in the distributed hash table includes either an address in one of the nonvolatile memory units where the data item is stored or an indication the data item is stored in a primary storage.
11. The compute cluster of claim 10 wherein the high speed interconnect includes an optical fiber technology.
12. The compute cluster of claim 10 wherein the local interconnect conforms to one of a serial attached storage (SAS) standard, a peripheral component interconnect express (PCI Express) standard, or a DRAM-like protocol.
13. The compute cluster of claim 10 wherein the nonvolatile memory units comprise at least one selected from the group including solid state drives aka silicon storage devices (SSDs), flash memory, NAND-based flash memory, phase change memory, and spin torque memory.
14. The compute cluster of claim 10 wherein a number of computing nodes exceeds a number of nonvolatile memory units.
15. The compute cluster of claim 10 wherein a number of nonvolatile memory units exceeds a number of input/output nodes.
16. The compute cluster of claim 10 wherein each computing node has multiple cores.
17. The compute cluster of claim 10 wherein the metadata includes a data item identifier and an offset for the data item.
18. The compute cluster of claim 10 further comprising:
a primary storage including a plurality of storage servers, wherein the input/output nodes are coupled with the primary storage via a network.
19. The compute cluster of claim 10 wherein the compute cluster is a super computer.