Communication method and related components
Disclosed are a communication method and related components. In this solution, two network stacks are firstly established, and all connection relationships between a client and each server node and between processes of any two server nodes are then divided, so as to obtain two kinds of connection relationships, wherein the two kinds of connection relationships are respectively constructed on the basis of the two network stacks, and the number of one of the two kinds of connection relationships is related to the number of QPs.
1. A communication method applied to a distributed storage system, wherein the distributed storage system comprises a client and a plurality of server nodes, and the method comprises:
establishing a first network stack and a second network stack; and
dividing all connection relationships between the client and each server node and between processes of any two server nodes to obtain a first connection relationship and a second connection relationship, so that communications between the client and each server node and between processes of any two server nodes are based on a corresponding connection relationship;
wherein the first connection relationship is established based on the first network stack, and the second connection relationship is established based on the second network stack; and
a number of first connection relationships or a number of second connection relationships is related to a number of quantization parameters (QPs).
2. The method according to claim 1 , the dividing all connection relationships between the client and each server node and between processes of any two server nodes to obtain the first connection relationship and the second connection relationship, so that communications between the client and each server node and between processes of any two server nodes are based on a corresponding connection relationship, comprises:
dividing a connection relationship between the client and each server node into the first connection relationship, so that communication between the client and each server node is based on the first connection relationship; and
dividing a connection relationship between processes of any two server nodes into the second connection relationship, so that communication between processes of any two server nodes is based on the second connection relationship.
3. The method according to claim 2 , wherein the number of first connection relationships established based on the first network stack is related to the number of QPs, and the number of second connection relationships established based on the second network stack is not related to the number of QPs.
4. The method according to claim 3 , wherein the first network stack is an RDMA network stack and the second network stack is a Posix network stack.
5. The method according to claim 2 , wherein the distributed storage system further comprises a monitoring terminal for monitoring an operating state of each server node, and the method further comprises:
dividing a connection relationship between the monitoring terminal and each server node into the first connection relationship, so that the monitoring terminal monitors each server node based on the first connection relationship.
6. The method according to claim 5 , wherein the monitoring terminal is a display.
7. The method according to claim 2 , wherein the first network stack is an RDMA network stack, and
after the dividing a connection relationship between the client and each server node into the first connection relationship, so that the communication between the client and each server node is based on the first connection relationship, the method further comprises:
determining an amount of data transmitted between the client and the server nodes;
determining whether the amount of data is within a preset range;
controlling the client to transmit data to each server node through a bilateral primitive when the amount of data is within the preset range; and
controlling the client to transmit data to all server nodes through a unilateral primitive when the amount of data is not within the preset range.
8. The method according to claim 7 , wherein the server nodes comprise a main server node and a plurality of sub-server nodes, and
the controlling the client to transmit data to each server node through the bilateral primitive, comprises:
controlling the client to transmit data to the main server node through the bilateral primitive, and then controlling the main server node to transmit data to the sub-server nodes.
9. The method according to claim 7 , wherein before the controlling the client to transmit data to all server nodes through the unilateral primitive, the method further comprises:
controlling the client to send its own memory address and secret key to each server node through the bilateral primitive; and
controlling each server node to pull data from the memory address corresponding to the client based on the memory address and the secret key for data transmission between the client and each server node.
10. The method according to claim 7 , after the controlling each server node to pull data from the memory address corresponding to the client based on the memory address and the secret key, the method further comprises:
storing the pulled data in a memory address of each server node corresponding to the data.
11. The method according to claim 7 , wherein the preset range is less than 64 kb.
12. The method according to claim 7 , wherein the controlling the client to transmit data to all server nodes through the unilateral primitive comprises:
controlling all server nodes to pull data from the client.
13. The method according to claim 2 , wherein the first network stack is an RDMA network stack, and
after the dividing the connection relationship between the client and each server node into the first connection relationship, so that communication between the client and each server node is based on the first connection relationship, the method further comprises:
determining a type of data transmitted by the client to the server nodes;
determining whether the type of data is a preset type;
controlling the client to transmit the preset type of data to the server nodes through a bilateral primitive when the type of data is the preset type; and
controlling the client to transmit the preset type of data to all server nodes through a unilateral primitive when the type of data is not of the preset type.
14. The method according to claim 2 , wherein the number of first connection relationships established based on the first network stack is not related to the number of QPs, and the number of second connection relationships established based on the second network stack is related to the number of QPs.
15. The method according to claim 1 , wherein the method further comprises:
initializing the first network stack and the second network stack at startup of the distributed storage system.
16. The method according to claim 15 , wherein the method further comprises:
obtaining an accessed network segment detected by a first thread group corresponding to the first network stack and the second network stack at a phase of establishing a connection relationship; and
determining a target network stack from the first network stack and the second network stack based on the network segment.
17. A communication apparatus, comprising:
a memory for storing a computer program; and
a processor for implementing steps of the method according to claim 1 when the computer program is stored.
18. A distributed storage system, comprising a client, a plurality of server nodes and the communication apparatus according to claim 17 .
19. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores a computer program which implements steps of the method according to claim 1 when executed by a processor.
20. A communication system applied to a distributed storage system, wherein the distributed storage system comprises a client and a plurality of server nodes, and the communication system comprises:
an establishment unit for establishing a first network stack and a second network stack; and
a division unit for dividing all connection relationships between the client and each server node and between processes of any two server nodes to obtain a first connection relationship and a second connection relationship, so that communications between the client and each server node and between processes of any two server nodes are based on a corresponding connection relationship;
wherein the first connection relationship is established based on the first network stack, and the second connection relationship is established based on the second network stack; and
a number of first connection relationships or a number of second connection relationships is related to a number of quantization parameters (QPs).