Core-affine processing on symmetric multiprocessing systems
Embodiments of a symmetric multi-processing (SMP) system can provide full affinity of a connection to a core processor when desired, even when ingress packet distribution, protocol processing layer and applications may autonomously process packets on different cores of the SMP system. In an illustrative embodiment, the SMP system can include a server application that is configured to create a plurality of tasks and bind the plurality of tasks to a plurality of core processors. One or more of the plurality of tasks are configured to create a corresponding listening endpoint socket, bind and listen on a protocol address that is common to the plurality of tasks.
1. A symmetric multi-processing (SMP) system comprising:
a server application configured to create a plurality of tasks and bind the plurality of tasks to a plurality of core processors, the plurality of tasks configured to create a corresponding listening endpoint socket, and bind and listen on a protocol address common to the plurality of tasks; and
a protocol processor configured to respond to an ingress Transmission Control Protocol (TCP) connection on a predetermined core processor of the plurality of core processors by performing look-up of core-local information to locate a listening endpoint socket corresponding to a task local to the predetermined core processor with a core-preference for handling the TCP connection using the listening endpoint socket,
when the listening endpoint is located, establishing the TCP connection on the located listening endpoint socket, applying the TCP connection to the task local and bound to the predetermined core processor, and marking the TCP connection as no-lock required, and
when the listening endpoint is not located, passing the TCP connection to any available task of a matching socket with semantics preserved.
2. The SMP system according to claim 1 wherein:
the plurality of tasks are configured to generate an indication to a stack on the plurality of endpoint sockets respective to the plurality of tasks, the indication specifying that a listened-to endpoint socket accepts endpoint connections arriving on a same core processor.
3. The SMP system according to claim 2 wherein:
the server application is configured wherein the indication is recorded by a Transmission Control Protocol/Internet Protocol (TCP/IP) stack comprising marking an endpoint socket respective to a task of the plurality of tasks indicative of a local core preference, the TCP/IP stack storing core-identifier information within the endpoint socket and storing the endpoint socket in a core-local data structure specific to the respective core processor.
4. The SMP system according to claim 1 wherein:
the located TCP connection applies subsequent packets arriving on the located TCP connection on the predetermined core processor to the task local and bound to the predetermined core processor and processed lock-free.
5. The SMP system according to claim 1 , wherein:
the protocol processor is further configured to respond to arrival of a plurality of packets of an incoming Transmission Control Protocol (TCP) connection on a predetermined core processor of the plurality of core processors by interrogating sockets information to locate an endpoint socket corresponding to a task local to the predetermined core processor with a core preference for handling the TCP connection using the listening endpoint socket.
6. The SMP system according to claim 5 further comprising:
a Transmission Control Protocol/Internet Protocol (TCP/IP) stack configured to interrogate sockets information to find a matching socket with an indicated local core preference and establish a connection on the predetermined core processor and pass the established connection to an application task that executes on the predetermined core processor, the established connection being marked as lockless.
7. The SMP system according to claim 5 wherein:
the protocol processor is configured wherein connection establishment and passing of the established connection to a locally-bound server process are attained without sharing data structures across core processors and without cross-core data sharing and locks.
8. The SMP system according to claim 5 wherein:
the protocol processor is configured in absence of a task local to the predetermined core processor with an indicated core preference to select any task that is waiting for the TCP connection by locating a matching endpoint socket on another core processor or an endpoint socket that does not have the core-preference indicated on the endpoint socket.
9. The SMP system according to claim 5 further comprising:
a classifier configured to send initial and subsequent incoming packets for the TCP/IP connection to the predetermined core processor wherein a task on the predetermined core processor holds the TCP/IP connection and the subsequent packets are processed on the predetermined core processor without cross-core communications, the TCP/IP connection identified for lockless processing by a protocol stack by marking of a lockless attribute.
10. The SMP system according to claim 1 wherein:
the protocol processor is further configured to process egress packets passing from a server to a client and ingress packets passing from the client to the server without cross-core data sharing and locks wherein egress packets originating from the server are bound to a predetermined core processor of the plurality of core processors at which ingress packets of a Transmission Control Protocol (TCP) connection also arrive, and the ingress packets and the egress packets are processed on the same predetermined core processor.
11. The SMP system according to claim 10 wherein:
the protocol processor uses a lockless attribute marked on the TCP connection to specify that locks are not required when processing the TCP connection.
12. A symmetric multi-processing (SMP) system comprising:
a protocol processor configured to
respond to an ingress Transmission Control Protocol (TCP) connection on a predetermined core processor of a plurality of core processors by
performing look-up of core-local information to locate a listening endpoint socket corresponding to a task local to the predetermined core processor with a core-preference for handling the TCP connection using the listening endpoint socket,
when the listening endpoint is located, establishing the TCP connection on the located listening endpoint socket, applying the TCP connection to the task local and bound to the predetermined core processor, and marking the TCP connection as no-lock required, and
when the listening endpoint is not located, passing the TCP connection to any available task of a matching socket with semantics preserved; and
respond to arrival of a plurality of packets of the TCP connection on the predetermined core processor by interrogating sockets information to locate an endpoint socket corresponding to the task local to the predetermined core processor.
13. The SMP system according to claim 12 further comprising:
a Transmission Control Protocol/Internet Protocol (TCP/IP) stack configured to interrogate sockets information to find a matching socket with an indicated local core preference and establish a connection on the predetermined core processor and pass the established connection to an application task that executes on the predetermined core processor, the established connection being marked as lockless.
14. The SMP system according to claim 12 wherein:
the protocol processor is configured wherein connection establishment and passing of the established connection to a locally-bound server process are attained without sharing data structures across core processors and without cross-core data sharing and locks.
15. The SMP system according to claim 12 wherein:
the protocol processor is configured in absence of a task local to the predetermined core processor with an indicated core preference to select any task that is waiting for the TCP connection by locating a matching endpoint socket on another core processor or an endpoint socket that does not have core-preference indicated on the endpoint socket.
16. The SMP system according to claim 12 further comprising:
a classifier configured to send initial and subsequent incoming packets for the TCP/IP connection to the predetermined core processor wherein a task on the predetermined core processor holds the TCP/IP connection and the subsequent packets are processed on the predetermined core processor without cross-core communications, the TCP/IP connection identified for lockless processing by a protocol stack by marking of a lockless attribute.
17. A method for symmetric multi-processing (SMP) comprising:
creating a plurality of tasks;
binding the plurality of tasks to a plurality of core processors;
creating a listening endpoint socket corresponding to the plurality of tasks;
binding and listening on a protocol address common to the plurality of tasks; and
responding to an ingress Transmission Control Protocol (TCP) connection on a predetermined core processor of the plurality of core processors by
performing look-up of core-local information to locate a listening endpoint socket corresponding to a task local to the predetermined core processor with a core-preference for handling the TCP connection using the listening endpoint socket,
if the listening endpoint is located, establishing the TCP connection on the located listening endpoint socket, applying the TCP connection to the task local and bound to the predetermined core processor, and marking the TCP connection as no-lock required, and
if the listening endpoint is not located, passing the TCP connection to any available task of a matching socket with semantics preserved.
18. The method according to claim 17 further comprising:
responding to arrival of a plurality of packets of the TCP connection on the predetermined core processor of the plurality of core processors by interrogating sockets information to locate an endpoint socket corresponding to the task local to the predetermined core processor with an indicated core preference.