Supercomputer system, method of data transmission in such supercomputer system and associated computer program product
A supercomputer system having at least one server, a plurality of compute clients and a distributed file system. The server is configured to distribute computation tickets and each compute client is configured to request a computation ticket. The distributed file system is configured to stock input data for all tasks and output data produced by the compute clients. The supercomputer system is characterized in that it further includes at least one I/O client configured to exchange data with each compute client and with the distributed file system, to distribute input data to at least some of the compute clients or to collect output data produced by at least some of the compute clients.
1. A supercomputer system comprising at least one server, a plurality of compute clients and a distributed file system;
the server being configured to distribute computation tickets to the compute clients;
each compute client being configured to request a computation ticket from the server and upon receipt of a computation ticket from the server, to execute a task defined by this ticket, each task comprising analyzing input data and producing output data;
the distributed file system being configured to stock input data for all tasks and output data produced by the compute clients;
wherein the supercomputer system comprises at least one I/O client configured to exchange data with each compute client and with the distributed file system, to distribute input data to at least some of the compute clients or to collect output data produced by at least some of the compute clients,
wherein each compute client is configured to request corresponding input data from the I/O client and if the I/O client is unable to provide the requested input data, to read the corresponding input data from the distributed file system.
2. The supercomputer system according to claim 1 , wherein each compute client is configured to request an identifier of the I/O client from the server.
3. The supercomputer system according to claim 1 , wherein an identifier of the I/O client is transmitted by the server after the execution of the corresponding task by the corresponding compute client.
4. The supercomputer system according to claim 1 , wherein the I/O client is configured to read periodically input data usable by the compute clients.
5. The supercomputer system according to claim 1 , wherein the I/O client is configured to implement a pre-processing of the read input data from the distributed file system.
6. The supercomputer system according to claim 1 , wherein each compute client is configured to send all produced output data to the I/O client.
7. The supercomputer system according to claim 1 , wherein the I/O client is configured to implement a post-processing of the collected output data.
8. The supercomputer system according to claim 7 , wherein the I/O client is configured to stock in the distributed file system results of the post-processing.
9. The supercomputer system according to claim 1 , wherein the I/O client is configured to write periodically in the distributed file system the collected output data.
10. The supercomputer system according to claim 1 , wherein the server, the I/O client, the distributed file system and each compute client are implemented by one or several physical compute nodes.
11. The supercomputer system according to claim 1 , wherein the I/O client is configured to read periodically input data usable by the compute clients and determined by the server, from the distributed file system.
12. The supercomputer system according to claim 1 , wherein the server is configured to designate a new I/O client, from one of the compute clients, if the current I/O client is not available.
13. A method of data transmission in a supercomputer system comprising at least one server, a plurality of compute clients, an I/O client and a distributed file system;
the method being implemented by at least one compute client and comprising the following steps:
requesting a computation ticket from the server;
receiving a computation ticket from the server; and
executing a task defined by this ticket, each task comprising analyzing input data and producing output data, input data being stocked in the distributed file system and output data being intended to be stocked or processing in or by the distributed file system;
wherein the method further comprises a step of collecting input data from the I/O client or a step of sending output data to the I/O client,
wherein each compute client is configured to request corresponding input data from the I/O client and if the I/O client is unable to provide the requested input data, to read the corresponding input data from the distributed file system.
14. A non-transitory computer program product comprising software instructions which, when executed by a processor, implement a method of data transmission in a supercomputer system comprising at least one server, a plurality of compute clients, an I/O client and a distributed file system;
the method being implemented by at least one compute client and comprising the following steps:
requesting a computation ticket from the server;
receiving a computation ticket from the server; and
executing a task defined by this ticket, each task comprising analyzing input data and producing output data, input data being stocked in the distributed file system and output data being intended to be stocked or processing in or by the distributed file system;
wherein the method further comprises a step of collecting input data from the I/O client or a step of sending output data to the I/O client,
wherein each compute client is configured to request corresponding input data from the I/O client and if the I/O client is unable to provide the requested input data, to read the corresponding input data from the distributed file system.