Granted Patent B2

US 7,581,033 · App. 10/729,312 · Granted Aug 25, 2009

Abstract

Claims (36)

1. A host computer system comprising:

a first processor;

a first memory;

a data storage device, the data storage device having an operating system stored thereon, the operating system executable by the first processor, the operating system creating an environment in which an application runs, the application communicating with a destination;

a send work queue, stored in the first memory, the send work queue comprising at least one send work queue entry, each send work queue entry comprising at least one of a descriptor and a control-path command;

a receive work queue, stored in the first memory, the receive work queue comprising at least one receive work queue entry, each receive work queue entry comprising at least one of a descriptor and a control-path command;

a Network Interface Card (NIC), the NIC comprising a port through which the application communicates with the destination, the NIC further comprising a second memory and a second processor;

a token table, stored in the first memory and associated with said NIC, the token table communicating descriptors and control-path commands from the send work queue and receive work queue to the NIC;

a Notification Request Area (NRA), stored in the second memory, the NRA allowing the NIC to notify the operating system that a descriptor has completed;

a Master Completion Queue (MCQ), the MCQ being stored in the first memory and being associated with at least one of the send work queue and the receive work queue, the MCQ comprising the completion status of descriptors posted to the associated work queue;

a Memory Deregistration List (MDL), the MDL being stored in the second memory, the MDL containing memory handles to be deregistered;

an MDL Insert Kernel Agent counter running in the operating system, which indicates where in the MDL a next deregistered memory handle should be inserted;

a Memory Region Table (MRT), the MRT being stored in the first memory and maintained by the MDL Insert Kernel Agent, containing all registration information for all memory regions registered on the NIC; and

a shadow MRT, the shadow MRT being stored in the second memory, which is a copy of the MRT entries for said NIC.

2. The host computer system of claim 1 further comprising a master completion queue for each NIC, and each such master completion queue able to reference any completion queue associated with the NIC.

3. The host computer system of claim 1 wherein there is a send and a receive work queue and a completion queue for each application program and wherein a notification request area exists for each completion queue and for said master completion queue, said notification request area providing notice to requestors of completions.

4. A method of optimizing Network Interface Card (NIC) based data communications between an application and a destination, the communications occurring through a NIC wherein the NIC and the application are on a host computer system and the destination is reachable through a port on the NIC, the method comprising:

receiving, by Virtual Interface Architecture Provider Library (VIPL) running on the host computer system, a call from a user application to a function, the call supplying said VIPL with a descriptor;

linking the descriptor onto an appropriate work queue;

atomically incrementing a Last_Inserted_Entry of a token table, and storing a representation of the result in a Local_Insert_Index of the token table;

using the descriptor to fill in a Post_Array field of the token table;

determining if the Post Array field filled by the descriptor is a first unaccepted Post_Array entry and if so, writing a new-entry-posted notification to an Out Post FIFO on an I/O bridge linking the NIC and the host computer system;

operating, on receipt of an interrupt to a Message Unit from the Out Post FIFO on the NIC, a primary Do loop of a Message Unit which copies the entries from the Post array field of the token table and, for each copied entry that is new:

determining whether the new entry contains a descriptor, and if so, fetching memory registration information for the memory region containing the descriptor and storing that information in a shadow copy of a memory registration table on the NIC;

determining if the new entry contains an indirect descriptor reference and if so:

copying the descriptor from application program associated memory;

building an I/O Resource Block (IORB) that represents the copied descriptor; and,

calling a Virtual Interface-to-Fibre Channel (Vito) Protocol component and passing said IORB to said Vito Protocol component;

if the new entry does not contain an indirect descriptor reference, handling the entry as a control command, else incrementing a counter for a last accented entry; and

continuing the primary Do loop until the last copied entry has been handled.

5. The method of claim 4 , contents of the descriptor being validated prior to linking said descriptor onto said appropriate work queue.

6. The method of claim 4 further comprising determining if the token table is full, and if so, causing the system to wait until there is room on the token table before processing the application request, the determining occurring prior to the use of the descriptor to fill in the Post Array field of the token table.

7. The method of claim 4 further comprising:

adjusting the number of token table entries copied per copy operation based upon load.

8. The method of claim 4 further comprising:

adjusting frequency of polling of the token table by the Message Unit based on measured load as determined by number of valid Post Array entries.

Assignments (11)