IP Library Granted Patent US 9,465,760
Granted Patent B2
US 9,465,760 · App. 14/083,206 · Granted Oct 11, 2016

Method and apparatus for delivering MSI-X interrupts through non-transparent bridges to computing resources in PCI-express clusters

Inventors: Norbert EGI (Santa Clara, CA); Robert Lasater (Menlo Park, CA); Thomas Boyle (Santa Clara, CA); John Peters (Sunnyvale, CA); Guangyu Shi (Cupertino, CA)
Assignee: Futurewei Technologies, Inc.
G06F13/32G06F13/4221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,465,760
App. No.
14/083,206
Granted
Oct 11, 2016
Kind
B2
Abstract

An apparatus for initialization. The apparatus includes a management I/O device controller for managing initialization of a plurality of I/O devices coupled to a PCI-Express (PCIe) fabric. The management I/O device controller is configured for receiving a request to register a target interrupt register address of a first worker computing resource, wherein the target interrupt register address is associated with a first interrupt generated by a first I/O device coupled to the PCIe fabric. A mapping module of the management I/O device controller is configured for mapping the target interrupt register address to a mapped interrupt register address of a domain in which the first I/O device resides. A translating interrupt register table includes a plurality of mapped interrupt register addresses in the domain that is associated with a plurality of target interrupt register addresses of a plurality of worker computing resources.

Claims (43)

1. An apparatus, comprising:

a management I/O device controller for managing initialization of a plurality of I/O devices coupled to a PCI Express (PCIe) fabric, wherein said management I/O device controller is configured for receiving a request to register a target interrupt register address of a first worker computing resource, wherein said target interrupt register address is associated with a first interrupt generated by a first I/O device coupled to said PCIe fabric;

a mapping module of said management I/O device controller configured for mapping said target interrupt register address to a mapped interrupt register address of a domain in which said first I/O device resides; and

a translating interrupt register table comprising a plurality of mapped interrupt register addresses in said domain that is associated with a plurality of target interrupt register addresses of a plurality of worker computing resources,

wherein said management I/O device controller is further configured for sending instructions to said first I/O device to register said mapped interrupt register address in association with said first interrupt in a corresponding I/O interrupt vector table of said first I/O device.

2. The apparatus of claim 1 , wherein said management I/O device controller is further configured for determining whether a vector entry is available within said I/O interrupt vector table for registering said first interrupt with said first I/O device and for sending instructions to register said mapped interrupt register address as a first vector in a first vector entry.

3. The apparatus of claim 1 , wherein said management I/O device controller is further configured for receiving information related to a size of a target interrupt register of said first worker computing resource including said target interrupt register address, and configured for allocating a first subset of virtual memory space in said translating interrupt register table controlled by said management I/O controller, wherein said first subset is at least equal to said size, and configured for mapping addresses in said target interrupt register to a plurality of mapped addresses in said virtual memory space.

4. The apparatus of claim 1 , wherein said management I/O device controller is further configured for sending an instruction to said I/O device to enable said first interrupt in said first I/O device, and configured for sending a notification to said first worker computing resource indicating that said first interrupt has been enabled in said first I/O device.

5. The apparatus of claim 1 , wherein said first interrupt is taken from a group consisting essentially of a message signal interrupt (MSI) interrupt, and an MSI-X interrupt.

6. A method for initialization, comprising:

receiving a request to register a target interrupt register address of a first worker computing resource, wherein said target interrupt register address is associated with a first interrupt generated by a first I/O device coupled to a PCI-Express (PCIe) fabric;

mapping said target interrupt register address to a mapped interrupt register address of a domain in which said first I/O device resides;

sending instructions to said first I/O device to register said mapped interrupt register address in association with said first interrupt in a corresponding I/O interrupt vector table of said first I/O device.

7. The method of claim 6 , further comprising:

sending mapping information mapping said mapped interrupt register address to said target interrupt register address to a first non-transparent bridge (NTB) coupling said first worker computing resource to said PCIe fabric.

8. The method of claim 6 , wherein said request is received at a management I/O device controller, wherein said management I/O device controller is coupled to said PCIe fabric to manage initialization of said plurality of I/O devices.

9. The method of claim 6 , wherein said sending instruction further comprises:

determining whether a vector entry is available within said I/O interrupt vector table for registering said first interrupt with said first I/O device and for storing said mapped interrupt register address as a first vector in said I/O interrupt vector table; and

sending said mapped interrupt register address to said first I/O device for storage in said available entry.

10. The method of claim 9 , further comprising:

determining whether said first I/O device can generate said first interrupt.

11. The method of claim 6 , further comprising:

sending an instruction to said first I/O device to enable said first interrupt in said first I/O device.

12. The method of claim 6 , further comprising:

sending a notification to said first worker computing resource indicating that said first interrupt has been enabled in said first I/O device.

13. The method of claim 6 , further comprising:

receiving information related to a size of target interrupt register of said first worker computing resource, wherein said target interrupt registers comprises a plurality of target interrupt register addresses;

allocating a first subset of virtual memory space in a mapped interrupt register comprising a plurality of mapped interrupt register addresses in said domain, wherein said mapped interrupt register is controlled by said management I/O controller, wherein said first subset is at least equal to said size; and

mapping addresses in said target interrupt register to a plurality of mapped addresses in said first subset of virtual memory space.

14. The method of claim 6 , wherein said first interrupt is taken from a group consisting essentially of a message signal interrupt (MSI) interrupt, and an MSI-X interrupt.

15. A system, comprising:

a plurality of worker computing resources;

a plurality of target interrupt registers associated with said plurality of worker computing resources;

a plurality of I/O devices located on said plurality of worker computing resources;

a PCI-Express (PCIe) fabric for enabling communication between said plurality of I/O devices and said plurality of worker computing resources all of which are coupled to said PCIe fabric;

a management I/O device controller configured for managing initialization of said plurality of I/O devices;

a translating interrupt register table comprising a plurality of mapped interrupt register addresses associated with a plurality of target interrupt register addresses of said plurality of worker computing resources;

wherein said management I/O device controller is configured for receiving a request to register a target interrupt register address of a first worker interrupt register of a first worker computing resource, wherein said target interrupt register address is associated with a first interrupt generated by a first I/O device coupled to said PCIe fabric; and

a mapping module of said management I/O device controller configured for mapping said target interrupt register address to a mapped interrupt register address, wherein said management I/O device controller is further configured for sending instructions to said first I/O device to register said mapped interrupt register address in association with said first interrupt in a corresponding I/O interrupt vector table of said first I/O device.

16. The system of claim 15 , further comprising:

a plurality of non-transparent bridges (NTBs) coupled to said plurality of worker computing resources, wherein a first NTB bridge is coupled to said management I/O device controller and is configured for sending said first interrupt originally delivered to said mapped interrupt register address to said target interrupt register address.

17. The system of claim 15 , wherein said management I/O device controller is further configured for determining whether a vector entry is available within said I/O interrupt vector table for registering said first interrupt with said first I/O device and for sending instructions to register said mapped interrupt register address as a first vector in a first vector entry.

18. The system of claim 15 , wherein said management I/O device controller is further configured for receiving information related to a size of said target interrupt register, and configured for allocating a first subset of virtual memory space in a mapped interrupt register controlled by said management I/O controller, wherein said first subset is at least equal to said size, and configured for mapping addresses in said target interrupt register to a plurality of mapped addresses in said virtual memory space.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2013
From: EGI, NORBERT; LASATER, ROBERT; BOYLE, THOMAS; PETERS, JOHN; SHI, GUANGYU
To: FUTUREWEI TECHNOLOGIES, INC.
Reel/Frame 031624/0297 →
Continuity (1)
Related Publication 20150143016A1 · May 21, 2015