Method of storing data, storage system, and storage apparatus

1. A method of storing data using a first storage apparatus, a second storage apparatus, and a third storage apparatus coupled with each other through a network, the method comprising:

receiving, by the first storage apparatus, a write request of first data having a destination of the second storage apparatus and second data having a destination of the third storage apparatus from a terminal apparatus coupled to the first storage apparatus via the network;

writing the first data and the second data into the first storage apparatus in response to the write request;

generating first replication data by replicating the first data;

generating second replication data by replicating the second data;

transmitting a packet including the first replication data and the second replication data to the second storage apparatus, when an amount of data stored in a transmission buffer of the first storage apparatus is not equal to or more than a first threshold value which is determined based on a total capacity of the transmission buffer and when a communication load of the network is equal to or more than a second threshold value;

writing, by the second storage apparatus, the first replication data and the second replication data into the second storage apparatus when the packet is received;

generating third replication data by replicating the second replication data; and

transmitting the third replication data from the second storage apparatus to the third storage apparatus; and

transmitting, by the first storage apparatus, a packet including only the first replication data among the first replication data and the second replication data, when the amount of data is equal to or higher than the first threshold value.

2. The method according to claim 1 , wherein

the transmitting of the third replication data includes transmitting the third replication data when a communication load of the network is equal to or less than the second threshold value.

3. The method according to claim 1 , further comprising:

transmitting, by the second storage apparatus, a first completion notification to the first storage apparatus when the writing of the first replication data and the second replication data is completed; and

transmitting, by the first storage apparatus, a second completion notification to a terminal apparatus that issued the write request, when the first completion notification is received.

4. The method according to claim 3 , further comprising:

recording, by the first storage apparatus, a first corresponding relationship between the first replication data and a destination address of the first replication data, and a second corresponding relationship between the second replication data and a destination address of the second replication data, in a replication list stored in the first storage apparatus, when the first storage apparatus transmits the first replication data and the second replication data to the second storage apparatus; and

deleting the first corresponding relationship and the second corresponding relationship from the replication list, when the first completion notification is received from the second storage apparatus.

5. The method according to claim 1 , further comprising:

writing, by the third storage apparatus, the third replication data into the third storage apparatus when the third replication data is received;

transmitting a third completion notification to the second storage apparatus when the writing of the third replication data is completed; and

deleting, by the second storage apparatus, the second replication data , when the third completion notification is received.

6. The method according to claim 1 , wherein

destinations of the first data and the second data are determined based on a first data ID attached to the first data, and a second data ID attached to the second data, respectively, and the second storage apparatus is configured to recognize the second data as data to be addressed to the third storage apparatus based on the data ID of the second data.

7. The method according to claim 1 , wherein

the communication load of the network is determined based on a number of packets transmitted on the network per unit time.

8. The method according to claim 1 ,

wherein the communication load of the network is determined based on time occupied for transmission and reception of a Ping command between the second storage apparatus and the third storage apparatus.

9. The method according to claim 1 , further comprising

transmitting the packet including only the first replication data among the first replication data and the second replication data, when the amount of data is not equal to or more than the first threshold and when the communication load of the network is not equal to or more than the second threshold value.

10. The method according to claim 9 , further comprising:

storing the first replication data and the second replication data, when it is determined that the another replication data is not stored in the transmission buffer.

11. The method according to claim 1 , further comprising:

determining whether another replication data is stored in the transmission buffer, and

determining whether a total amount of the another replication data is equal to or more than a third threshold value, when it is determined that the another replication data is stored in the transmission buffer,

transmitting the another replication data, when it is determined that the total amount of the another replication data is equal to or more than the third threshold value, and

storing the first replication data and the second replication data after the transmission buffer becomes empty by the transmitting.

12. A storage system comprising:

a first storage apparatus configured to:

receive a write request of first data having a destination of a second storage apparatus and second data having a destination of a third storage apparatus from a terminal apparatus coupled to the first storage apparatus via the network;

write the first data and the second data into the first storage apparatus in response to the write request;

generate first replication data by replicating the first data;

generate second replication data by replicating the second data; and

transmit a packet including the first replication data and the second replication data to the second storage apparatus, when an amount of data stored in a transmission buffer of the first storage apparatus is not equal to or more than a first threshold which is determined based on a total capacity of the transmission buffer and when a communication load of the network is equal to or more than a second threshold, and

the second storage apparatus configured to:

write the first replication data and the second replication data into the second storage apparatus when the packet is received;

generate third replication data by replicating the second replication data; and

transmit the third replication data received from the second storage apparatus to the third storage apparatus, and

the first storage apparatus configured to

transmit a packet including only the first replication data among the first replication data and the second replication data, when the amount of data is equal to or higher than the first threshold value.

13. The storage system according to claim 12 , wherein

the second storage apparatus is configured to transmit a first completion notification to the first storage apparatus when the writing of the first replication data and the second replication data is completed, and

the first storage apparatus is configured to transmit a second completion notification to a terminal apparatus that issued the write request, when the first completion notification is received.

14. The storage system according to claim 12 , wherein

the third storage apparatus is configured to:

write the third replication data into the third storage apparatus when the third replication data is received, and

transmit a third completion notification to the second storage apparatus when the writing of the third replication data is completed, and

the second storage apparatus is configured to

delete the second replication data , when the third completion notification is received.

15. The storage system according to claim 12 ,

wherein destinations of the first data and the second data are determined based on a first data ID attached to the first data, and a second data ID attached to the second data, respectively, and the second storage apparatus is configured to recognize the second data as data to be addressed to the third storage apparatus based on the data ID of the second data.

16. The storage system according to claim 12 ,

wherein the communication load of the network is determined based on a number of packets transmitted on the network per unit time.

17. The storage system according to claim 12 ,

wherein the communication load of the network is determined based on time occupied for transmission and reception of a Ping command between the second storage apparatus and the third storage apparatus.

18. A storage apparatus coupled with a first storage apparatus and another storage apparatus, the storage apparatus comprising:

a memory; and

at least one processor coupled to the memory and configured to:

receive, from the first storage apparatus, a packet including a first replication data and a second replication data respectively generated by the first storage apparatus, the first replication data being generated by replicating first data having a destination of the storage apparatus, the second replication data being generated by replicating second data having a destination of the another storage apparatus, the packet being transmitted by the first storage apparatus when an amount of data stored in a transmission buffer of the first storage apparatus is not equal to or more than a first threshold value which is determined based on a total capacity of the transmission buffer and when a communication load of the network is equal to or more than a second threshold value,

write the first replication data and the second replication data into the memory,

generate third replication data by replicating the second replication data, and

transmit the third replication data to the another storage apparatus,

wherein the first storage apparatus is configured to

transmit a packet including only the first replication data among the first replication data and the second replication data, when the amount of data is equal to or higher than the first threshold value.

19. The storage apparatus according to claim 18 , wherein

the processor is configured to transmit the third replication data to the another storage apparatus when a communication load of the network is equal to or less than a second threshold value.

20. The storage apparatus according to claim 18 , wherein

the processor is configured to delete the second replication data , when a completion notification, indicating that writing of the third replication data into the another storage apparatus is completed, is received from the another storage apparatus.