Communication between host systems using a socket connection and shared memories
View Patent ↗Mechanisms for communication between host systems using a socket connection and shared memories are provided. With such socket-based communication, a work queue in the host systems may be used to listen for incoming socket initialization requests. A first host system that wishes to establish a socket communication connection with a second host system may generate a socket initialization request work queue element in its work queue and may inform the second host system that the socket initialization request work queue element is available for processing. The second host system may then accept or deny the request. If the second host system accepts the request, it returns the second half of the socket's parameters for use by the first host system in performing socket based communications between the first and second host systems.
1. A method, in a data processing system, for communicating between a first host system and a second host system, comprising:
initializing a memory address space in a shared memory of the first host system to comprise an address range assigned to an endpoint associated with the second host system;
generating a work queue structure in the address range;
posting, by the first host system, a socket initialization request to the work queue structure;
retrieving, by the second host system, the socket initialization request from the work queue structure via the endpoint by performing a direct memory access (DMA) operation to retrieve the socket initialization request from the work queue structure, wherein the socket initialization request contains a first portion of parameters for establishing the socket communication connection;
performing a DMA operation from the endpoint to a root complex of the first host system to transfer a second portion of parameters for establishing the socket communication connection responsive to the second host system accepting the socket communication request;
establishing a socket communication connection between the first host system and the second host system based on the socket initialization request, the first portion of parameters, and the second portion of parameters; and
processing send and receive communications between the first host system and the second host system using the established socket communication connection.
2. The method of claim 1 , wherein initializing the memory address space in the shared memory of the first host system comprises processing vital product data (VPD) information for the endpoint, wherein the VPD information includes a field indicating whether the endpoint supports socket connections, and wherein the socket initialization request is posted to the work queue structure only if the VPD information indicates that the endpoint supports socket connections.
3. The method of claim 1 , further comprising:
performing a write operation to a doorbell structure in the shared memory in response to posting the socket initialization request; and
polling, by the second host system, the doorbell structure to determine if a socket initialization request is available for processing by the second host system.
4. The method of claim 1 , further comprising:
parsing, by the second host system, the socket initialization request;
determining, by the second host system, whether to accept or deny the socket initialization request; and
sending, by the second host system, a response to the first host system based on results of determining whether to accept or deny the socket initialization request.
5. The method of claim 4 , wherein the socket initialization request contains a base address, limit, and starting offset into the work queue structure to be used for the first portion of parameters for establishing the socket communication connection.
6. The method of claim 1 , wherein the second portion of parameters for establishing the socket communication connection comprises a base address, limit, and starting offset into a work queue structure provided in a shared memory associated with the second host system.
7. The method of claim 1 , wherein the send and receive operations comprise at least one of push operations or pull operations in accordance with an established transaction protocol.
8. The method of claim 7 , wherein the push operations and pull operations are direct memory access (DMA) operations.
9. The method of claim 1 , wherein initializing the memory address space in a shared memory of the first host system comprises:
traversing links in a communication fabric of the data processing system to gather information about endpoints and root complexes present in the communication fabric;
generating at least one virtual hierarchy identifying at least one endpoint and at least one root complex that are physically or logically associated with each other; and
initializing the memory address space based on the at least one virtual hierarchy such that each endpoint associated with a root complex of the first host system has a corresponding address range in the memory address space.
10. The method of claim 1 , wherein the data processing system is a blade server and the first host system and second host system are blades in the blade server.
11. The method of claim 1 , wherein the data processing system comprises a peripheral component interconnect express (PCIe) fabric to which the first host system and second host system are coupled, and wherein the endpoint is a PCIe adapter.
12. The method of claim 11 , wherein the PCIe fabric is a multiple root aware PCIe fabric comprising one or more multiple root aware (MRA) switches.
13. A computer program product comprising a non-transitory computer recordable medium having a computer readable program recorded thereon, wherein the computer readable program, when executed in a data processing system, causes the data processing system to:
initialize a memory address space in a shared memory of the first host system to comprise an address range assigned to an endpoint associated with the second host system;
generate a work queue structure in the address range;
post, by the first host system, a socket initialization request to the work queue structure;
retrieve, by the second host system, the socket initialization request from the work queue structure via the endpoint by performing a direct memory access (DMA) operation to retrieve the socket initialization request from the work queue structure, wherein the socket initialization request contains a first portion of parameters for establishing the socket communication connection;
perform a DMA operation from the endpoint to a root complex of the first host system to transfer a second portion of parameters for establishing the socket communication connection responsive to the second host system accepting the socket communication request;
establish a socket communication connection between the first host system and the second host system based on the socket initialization request, the first portion of parameters, and the second portion of parameters; and
process send and receive communications between the first host system and the second host system using the established socket communication connection.
14. The computer program product of claim 13 , wherein the computer readable program causes the data processing system to initialize the memory address space in the shared memory of the first host system by processing vital product data (VPD) information for the endpoint, wherein the VPD information includes a field indicating whether the endpoint supports socket connections, and wherein the socket initialization request is posted to the work queue structure only if the VPD information indicates that the endpoint supports socket connections.
15. The computer program product of claim 13 , wherein the computer readable program further causes the data processing system to:
perform a write operation to a doorbell structure in the shared memory in response to posting the socket initialization request; and
poll, by the second host system, the doorbell structure to determine if a socket initialization, request is available for processing by the second host system.
16. The computer program product of claim 13 , wherein the computer readable program further causes the data processing system to:
parse, by the second host system, the socket initialization request;
determine, by the second host system, whether to accept or deny the socket initialization request; and
send, by the second host system, a response to the first host system based on results of determining whether to accept or deny the socket initialization request.
17. The computer program product of claim 13 , wherein the socket initialization request contains a base address, limit, and starting offset into the work queue structure to be used for the first portion of parameters for establishing the socket communication connection.
18. The computer program product of claim 13 , wherein the second portion of parameters for establishing the socket communication connection comprises a base address, limit, and starting offset into a work queue structure provided in a shared memory associated with the second host system.
19. The computer program product of claim 13 , wherein the send and receive operations comprise at least one of push operations or pull operations in accordance with an established transaction protocol.
20. The computer program product of claim 19 , wherein the push operations and pull operations are direct memory access (DMA) operations.
21. The computer program product of claim 13 , wherein the computer readable program causes the data processing system to initialize the memory address space in a shared memory of the first host system by:
traversing links in a communication fabric of the data processing system to gather information about endpoints and root complexes present in the communication fabric;
generating at least one virtual hierarchy identifying at least one endpoint and at least one root complex that are physically or logically associated with each other; and
initializing the memory address space based on the at least one virtual hierarchy such that each endpoint associated with a root complex of the first host system has a corresponding address range in the memory address space.
22. The computer program product of claim 13 , wherein the data processing system is a blade server and the first host system and second host system are blades in the blade server.
23. The computer program product of claim 13 , wherein the data processing system comprises a peripheral component interconnect express (PCIe) fabric to which the first host system and second host system are coupled, and wherein the endpoint is a PCIe adapter.
24. The computer program product of claim 23 , wherein the PCIe fabric is a multiple root aware PCIe fabric comprising one or more multiple root aware (MRA) switches.
25. A data processing system, comprising:
a first host system;
a second host system; and
a communication fabric coupled to the first host system and the second host system, wherein:
a memory address space in a shared memory of the first host system is initialized to comprise an address range assigned to an endpoint associated with the second host system;
a work queue structure is generated in the address range;
a socket initialization request is posted, by the first host system, to the work queue structure;
the socket initialization request is retrieved, by the second host system, from the work queue structure via the endpoint by performing a direct memory access (DMA) operation to retrieve the socket initialization request from the work queue structure, wherein the socket initialization request contains a first portion of parameters for establishing the socket communication connection;
a DMA operation is performed from the endpoint to a root complex of the first host system to transfer a second portion of parameters for establishing the socket communication connection responsive to the second host system accepting the socket communication request;
a socket communication connection is established between the first host system and the second host system based on the socket initialization request, the first portion of parameters, and the second portion of parameters; and
send and receive communications between the first host system and the second host system are processed using the established socket communication connection.
26. The data processing system of claim 25 , wherein the memory address space in the shared memory of the first host system is initialized by processing vital product data (VPD) information for the endpoint, wherein the VPD information includes a field indicating whether the endpoint supports socket connections, and wherein the socket initialization request is posted to the work queue structure only if the VPD information indicates that the endpoint supports socket connections.
27. The data processing system of claim 25 , wherein:
a write operation is performed to a doorbell structure in the shared memory in response to posting the socket initialization request; and
the doorbell structure is polled, by the second host system, to determine if a socket initialization request is available for processing by the second host system.
28. The data processing system of claim 25 , wherein the second host system:
parses the socket initialization request;
determines whether to accept or deny the socket initialization request; and
sends a response to the first host system based on results of determining whether to accept or deny the socket initialization request.
29. The data processing system of claim 28 , wherein the socket initialization request contains a base address, limit, and starting offset into the work queue structure to be used for the first portion of parameters for establishing the socket communication connection.
30. The data processing system of claim 25 , wherein the second portion of parameters for establishing the socket communication connection comprises a base address, limit, and starting offset into a work queue structure provided in a shared memory associated with the second host system.
31. The data processing system of claim 25 , wherein the memory address space in the shared memory of the first host system is initialized by:
traversing links in a communication fabric of the data processing system to gather information about endpoints and root complexes present in the communication fabric;
generating at least one virtual hierarchy identifying at least one endpoint and at least one root complex that are physically or logically associated with each other; and
initializing the memory address space based on the at least one virtual hierarchy such that each endpoint associated with a root complex of the first host system has a corresponding address range in the memory address space.
32. The data processing system of claim 25 , wherein the data processing system is a blade server and the first host system and second host system are blades in the blade server.