Architecture and method for a burst buffer using flash technology

1. A parallel supercomputing cluster system comprising:

hardware compute nodes interconnected in a mesh of data links for executing a Message Passing Interface (MPI) job stored in memory and for using MPI data transfer between the compute nodes over the mesh of data links; and

hardware solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and

magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage;

wherein each solid-state storage node includes a hardware data processor coupled to the respective group of compute nodes for receiving the checkpoint data from the respective group of compute nodes and coupled to the magnetic disk storage for transmitting the checkpoint data to the magnetic disk storage, solid-state storage coupled to the data processor for buffering the checkpoint data, and non-transitory computer readable storage medium storing computer instructions that, when executed by the data processor, perform the steps of:

(a) presenting a file system interface to the MPI job, and multiple MPI processes of the MPI job writing the checkpoint data to a shared file in the solid-state storage in a strided fashion in a first data layout; and

(b) asynchronously migrating the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writing the checkpoint data to the magnetic disk storage in a sequential fashion in a second data layout.

2. The parallel supercomputing cluster as claimed in claim 1 , wherein the solid-state storage includes flash memory in solid-state drives.

3. The parallel supercomputer cluster as claimed in claim 1 , wherein the second data layout includes a separate checkpoint file for each of the multiple MPI processes of the MPI job.

4. The parallel supercomputing cluster as claimed in claim 1 , wherein the computer instructions include a first file system server layer for presenting the file system interface to the MPI job and defining the first data layout, a writeback scheduler for scheduling the asynchronous migration of the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage, a second file system layer for conversion between the first data layout and the second data layout, and a third file system layer for defining the second data layout.

5. The parallel supercomputer cluster as claimed in claim 4 , wherein the second file system layer provides a parallel log-structured file system.

6. The parallel supercomputer cluster as claimed in claim 4 , wherein the third file system layer provides a global file system.

7. The parallel supercomputer cluster as claimed in claim 1 , wherein the computer instructions, when executed by the data processor, respond to MPI invocations from the MPI job in the respective group of compute nodes to locate and specify data placement in the first data layout in the solid-state storage.

8. The parallel supercomputer cluster as claimed in claim 7 , wherein the MPI invocations include an invocation collective to all the compute nodes and the solid-state storage nodes to take file view information and read and distribute associated file data to the other nodes, so that there is a local copy in the solid-state storage of each solid-state storage node of what each MPI process will use on a subsequent MPI operation.

9. A parallel supercomputing cluster system comprising:

hardware compute nodes interconnected in a mesh of data links for executing a Message Passing Interface (MPI) job stored in memory and for using MPI data transfer between the compute nodes over the mesh of data links; and

hardware solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and

magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage;

wherein each solid-state storage node includes a data processor coupled to the respective group of compute nodes for receiving the checkpoint data from the respective group of compute nodes and coupled to the magnetic disk storage for transmitting the checkpoint data to the magnetic disk storage, solid-state storage coupled to the data processor for buffering the checkpoint data, and non-transitory computer readable storage medium storing computer instructions that, when executed by the data processor, perform the steps of:

(a) presenting a file system interface to the MPI job, and multiple MPI processes of the MPI job writing the checkpoint data to a shared file in the solid-state storage in a strided fashion in a first data layout, the checkpoint data being migrated from each compute node in the respective group of compute nodes to said each solid-state storage node by using remote direct memory access (RDMA); and

(b) asynchronously migrating the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writing the checkpoint data to the magnetic disk storage in a sequential fashion in a second data layout.

10. The parallel supercomputing cluster as claimed in claim 9 , wherein the solid-state storage includes flash memory in solid-state drives.

11. The parallel supercomputer cluster as claimed in claim 9 , wherein the second data layout includes a separate checkpoint file for each of the multiple MPI processes of the MPI job.

12. The parallel supercomputer cluster as claimed in claim 9 , wherein the computer instructions, when executed by the data processor, respond to MPI invocations from the MPI job in the respective group of compute nodes to locate and specify data placement in the first data layout in the solid-state storage, and the MPI invocations include an invocation collective to all the compute nodes and the solid-state storage nodes to take file view information and read and distribute associated file data to the other nodes, so that there is a local copy in the solid-state storage of each solid-state storage node of what each MPI process will need use on a subsequent MPI operation.

13. The parallel supercomputing cluster as claimed in claim 9 , which further includes the Parallel Log-Structured File System (PLFS) writing metadata and logs to a Lustre File System to write the metadata and logs to backing storage, and the Parallel Log-Structured File System (PLFS) writing data to a Network File System (NFS) to write the data to backing storage.

14. A method of operating a parallel supercomputing cluster including hardware compute nodes interconnected in a mesh of data links for executing a Message Passing Interface (MPI) job stored in memory and for using MPI data transfer between the compute nodes over the mesh of data links, hardware solid-state storage nodes each linked to a respective group of the compute nodes for receiving checkpoint data from the respective compute nodes, and magnetic disk storage linked to each of the solid-state storage nodes for asynchronous migration of the checkpoint data from the solid-state storage nodes to the magnetic disk storage, wherein each solid-state storage node includes a hardware data processor coupled to the respective group of compute nodes for receiving the checkpoint data from the respective group of compute nodes and coupled to the magnetic disk storage for transmitting the checkpoint data to the magnetic disk storage, and solid-state storage coupled to the data processor for buffering the checkpoint data, and non-transitory computer readable storage medium storing computer instructions, said method comprising the data processor executing the computer instructions to perform the steps of:

(a) presenting a file system interface to the MPI job, and multiple MPI processes of the MPI job writing the checkpoint data to a shared file in the solid-state storage in a strided fashion in a first data layout; and

(b) asynchronously migrating the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage and writing the checkpoint data to the magnetic disk storage in a sequential fashion in a second data layout.

15. The method as claimed in claim 14 , wherein the computer instructions include a first file system server layer presenting the file system interface to the MPI job and defining the first data layout, a writeback scheduler scheduling the asynchronous migration of the checkpoint data from the shared file in the solid-state storage to the magnetic disk storage, a second file system layer converting between the first data layout and the second data layout, and a third file system layer defining the second data layout.

16. The method as claimed in claim 15 , which further includes the second file system layer providing a parallel log-structured file system.

17. The method as claimed in claim 15 , which further includes the third file system layer providing a global file system.

18. The method as claimed in claim 15 , which further includes the data processor executing the computer instructions to respond to MPI invocations from the MPI job in the respective group of compute nodes to locate and specify data placement in the first data layout in the solid-state storage.

19. The method as claimed in claim 18 , wherein the MPI invocations include an invocation collective to all the compute nodes and the solid-state storage nodes to take file view information and read and distribute associated file data to the other nodes, so that there is a local copy in the solid-state storage of each solid-state storage node of what each MPI process will use on a subsequent MPI operation.

20. The method as claimed in claim 14 , which further includes migrating the checkpoint data from each compute node in the respective group of compute nodes to said each solid-state storage node by using remote direct memory access (RDMA).