Method of accessing a remote device from a host by mapping an address of the device to a memory address of the host
View Patent ↗A protocol for facilitating access to a local input/output device of a remote node across a network. The local input/output device is configured according to a local communication technology (e.g., PCI—Peripheral Component Interconnect), but is accessed remotely (e.g., across a set of InfiniBand (IB) communication links). A host server or other computer (e.g., an InfiniBand host) includes a channel adapter and a nexus driver for conveying or retrieving an instruction of a device driver for the local device across the network. The remote node includes a local/remote bridge (e.g., an IB/PCI bridge). The host connects to the remote node and determines a configuration of the node's local communication bus and the local input/output device. A host memory address is mapped to a local bus address and is used by the input/output device to exchange data with the host.
1. A method of accessing a PCI (Peripheral Component Interconnect) device from an InfiniBand host computer, comprising:
at an InfiniBand host computer, locating a remote node having a PCI segment comprising one or more PCI devices;
loading an InfiniBand/PCI nexus driver configured to enable the host computer to access the PCI segment of the remote node across an InfiniBand network coupling the host computer to the remote node;
establishing a connection with the PCI segment of the remote node;
identifying one or more PCI devices coupled to the PCI segment;
loading a device driver for a first PCI device within the remote node; and
exchanging data between the host computer and the first PCI device by:
registering a host computer memory address with the remote node;
mapping said host computer memory address to a local PCI address; and
transferring the data via the mapped local PCI address.
2. The method of claim 1 , further comprising:
receiving a completion message from the remote host indicating that the data were transferred.
3. The method of claim 2 , wherein the completion message comprises an interrupt.
4. The method of claim 1 , wherein said establishing a connection comprises:
establishing a connection between the host computer and a hot plug controller within the remote node.
5. The method of claim 4 , wherein said identifying comprises:
requesting from the hot plug controller configuration information regarding one or more interface slots of the PCI segment.
6. The method of claim 1 , wherein said establishing a connection comprises:
establishing a connection between the host computer and the PCI segment.
7. The method of claim 1 , wherein said establishing a connection comprises:
establishing a connection between the host computer and the first PCI device.
8. The method of claim 7 , further comprising:
reading a configuration space of the first PCI device to determine one or more resource requirements of the first PCI device.
9. The method of claim 8 , further comprising:
requesting the remote node to allocate a resource for the first PCI device.
10. The method of claim 9 , further comprising:
configuring the first PCI device with said allocated resource.
11. The method of claim 1 , wherein said establishing a connection comprises:
logging into the remote node.
12. The method of claim 1 , further comprising:
registering with the remote node the host computer ability to handle interrupts from the remote node.
13. The method of claim 12 , wherein said interrupts include interrupts generated by a hot plug controller within the remote node.
14. The method of claim 12 , wherein said interrupts include interrupts generated on the PCI segment.
15. The method of claim 12 , wherein said interrupts include interrupts generated by the first PCI device.
16. The method of claim 1 , wherein said registering a host computer memory address comprises:
sending a message to the remote node identifying the host computer memory address and an R_Key for accessing the host computer memory address.
17. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of accessing a PCI (Peripheral Component Interconnect) device from an InfiniBand host computer, the method comprising:
at an InfiniBand host computer, locating a remote node having a PCI segment comprising one or more PCI devices;
loading an InfiniBand/PCI nexus driver configured to enable the host computer to access the PCI segment of the remote node across an InfiniBand network coupling the host computer to the remote node;
establishing a connection with the PCI segment of the remote node;
identifying one or more PCI devices coupled to the PCI segment;
loading a device driver for a first PCI device within the remote node; and
exchanging data between the host computer and the first PCI device by:
registering a host computer memory address with the remote node;
mapping said host computer memory address to a local PCI address; and
transferring the data via the mapped local PCI address.
18. A method of accessing a local input/output device of a node remote from a host computer, comprising:
establishing a connection between the host and a hot plug controller in the node;
receiving from the hot plug controller a status of one or more interface slots of a local communication bus in the node, including a first slot coupled to the local input/output device;
retrieving configuration information from the local input/output device;
configuring the local input/output device for exchanging data with the host; and
mapping a memory address of the host to a local address of the local input/output device;
wherein said mapped memory address is used for transferring data between the host and the local input/output device.
19. The method of claim 18 , further comprising:
registering with the node the host's ability to handle interrupts from the hot plug controller.
20. The method of claim 18 , further comprising:
registering with the node the host's ability to handle interrupts from the local communication bus of the node.
21. The method of claim 18 , further comprising:
registering with the node the host's ability to handle interrupts from the local input/output device.
22. The method of claim 18 , wherein said establishing a connection comprises:
logging the host into a channel adapter of the node.
23. The method of claim 18 , wherein said establishing a connection comprises:
logging the host into the hot plug controller.
24. The method of claim 18 , wherein said establishing a connection comprises:
connecting the host to the local communication bus.
25. The method of claim 18 , further comprising:
loading in the host a device driver for the local input/output device; and
loading in the host a nexus driver configured to translate an instruction from the device driver for transmission to the node.
26. The method of claim 18 , wherein said retrieving configuration information comprises:
reading a configuration space of the local input/output device.
27. The method of claim 26 , wherein said configuring the local input/output device comprises:
writing a local bus memory address to the configuration space from the host; and
writing an input/output address to the configuration space from the host.
28. The method of claim 18 , wherein said mapping a memory address comprises:
registering with the node a host memory address and a key for accessing said host memory address; and
at the node, mapping said host memory address to a first local bus address.
29. The method of claim 28 , further comprising:
writing data from the local input/output device to the first local bus address; and
transferring the data to said host memory address.
30. The method of claim 28 , further comprising:
storing data at said host memory address to be transferred to the local input/output device; and
at the node, retrieving the stored data.
31. The method of claim 18 , further comprising:
issuing a command from the host to the remote node to transfer a set of data from the host to the local input/output device;
facilitating pre-fetching of a portion of said set of data by a network interface of the remote node; and
if said pre-fetched portion of said set of data loses synchronization with the host,
instructing the network interface of the remote node to invalidate said pre-fetched portion of said set of data.
32. The method of claim 18 , further comprising:
issuing a pass-through command from the host to a processor within the remote node;
wherein said pass-through command is received by a network interface of the remote node and passed to the processor.
33. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of accessing a local input/output device of a node remote from a host computer, the method comprising:
establishing a connection between the host and a hot plug controller in the node;
receiving from the hot plug controller a status of one or more interface slots of a local communication bus in the node, including a first slot coupled to the local input/output device;
retrieving configuration information from the local input/output device;
configuring the local input/output device for transferring data with the host; and
mapping a memory address of the host to a local address of the local input/output device;
wherein said mapped address is used for transferring data between the host and the local input/output device.