Virtual socket for load balancing and failover
View Patent ↗One aspect of the instant application can provide a system and method for balancing load among multiple network sockets established between a local node and a remote node. During operation, the system can encapsulate the multiple network sockets to form a local transport-layer virtual socket comprising a write interface and a read interface. The system can receive, at the write interface of the local transport-layer virtual socket, a packet; select, based on a load-balancing policy, a network socket from the multiple network sockets; and forward the packet to a socket-specific incoming queue associated with the selected network socket to allow the packet to be sent to the read interface of a corresponding remote transport-layer virtual socket via the selected network socket.
1. A computer-implemented method for balancing load among network sockets established between a local node and a remote node, the method comprising:
presenting, to the local node, a single write interface and a single read interface using a virtual socket, wherein:
the single write interface is configured to transmit packets to the remote node, and
the single read interface is configured to receive the packets from the remote node;
encapsulating, using the virtual socket, a set of Transmission Control Protocol (TCP) sockets between the local node and the remote node, wherein:
the virtual socket comprises a set of socket-specific incoming queues and a set of thread-specific outgoing queues,
each socket-specific incoming queue is coupled to a corresponding encapsulated TCP socket and is to queue packets to be transmitted over the corresponding encapsulated TCP socket, and
each thread-specific outgoing queue is coupled to the set of encapsulated TCP sockets and is to queue packets to be received by a corresponding thread executing on the local node;
restricting access to the encapsulated set of TCP sockets to access via the single write interface and the single read interface;
receiving, at the single write interface, a packet sent by the local node; and
applying a load-balancing technique to select an encapsulated TCP socket from the encapsulated set of TCP sockets; and
forwarding the packet to a socket-specific incoming queue corresponding to the selected encapsulated TCP socket.
2. The method of claim 1 , wherein selecting the encapsulated TCP socket comprises:
determining a load of each encapsulated TCP socket; and
selecting an encapsulated TCP socket with least load.
3. The method of claim 2 , wherein determining the load of each encapsulated TCP socket comprises one of:
determining an occupancy rate of the socket-specific incoming queue associated with each encapsulated TCP socket; and
determining a number of packets queued in the socket-specific incoming queue associated with each encapsulated TCP socket.
4. The method of claim 3 , wherein the selected encapsulated TCP socket with the least load corresponds to a socket-specific incoming queue with a lowest occupancy rate or a smallest number of queued packets.
5. The method of claim 1 , further comprising:
in response to detecting that a network interface associated with a particular encapsulated TCP socket fails, removing the particular encapsulated TCP socket from the virtual socket; and
redistributing packets queued in the socket-specific incoming queue associated with the removed particular TCP socket to other socket-specific incoming queues associated with other TCP sockets encapsulated within the virtual socket.
6. The method of claim 5 , further comprising:
in response to detecting a recovery of the failed network interface, creating a new TCP socket bound to the recovered network interface; and
encapsulating the new TCP socket to the virtual TCP-socket.
7. The method of claim 1 , wherein the local and remote node each execute multiple threads, and wherein the method further comprises:
receiving, from the remote node, a packet associated with a particular thread executing on the remote node;
determining a destination thread executing on the local node associated with the packet received from the remote node; and
forwarding the packet received from the remote node to a thread-specific outgoing queue based on the destination thread.
8. The method of claim 7 , further comprising notifying the destination thread to allow the destination thread to read the packet received from the remote node from the thread-specific outgoing queue via the read interface.
9. The method of claim 7 , wherein the thread-specific outgoing queue is thread safe and is accessible to the encapsulated set of TCP sockets.
10. The method of claim 7 , wherein the socket-specific incoming queue is thread safe and is accessible to the multiple threads.
11. The computer-implemented method of claim 1 , further comprising:
providing to the local node, at the single read interface, a second packet sent by the remote node via an encapsulated TCP socket from the encapsulated set of TCP sockets.
12. The computer-implemented method of claim 1 , further comprising:
forwarding the packets received from all threads on a client device to one of the set of socket-specific incoming queues.
13. An apparatus for balancing load among network sockets established between a local node and a remote node, the apparatus comprising:
a local transport-layer virtual socket that is to:
present, to the local node, a single write interface and a single read interface, wherein:
the single write interface is configured to transmit packets to the remote node,
the single read interface is configured to receive packets from the remote node;
encapsulate a set of TCP sockets between the local node and the remote node, wherein:
the local transport-layer virtual socket comprises a set of socket-specific incoming queues and a set of thread-specific outgoing queues,
each socket-specific incoming queue is coupled to a corresponding encapsulated TCP socket and is to queue packets to be transmitted over a corresponding encapsulated TCP socket, and
each thread-specific outgoing queue is coupled to the set of encapsulated TCP sockets and is to queue packets to be received by a corresponding thread executing on the local node; and
restricts access to the encapsulated set of TCP sockets to access via the single write interface and the single read interface; and
a non-transitory computer-readable medium storing instructions, which when executed by one or more processing resources, cause the one or more processing resources to apply a load-balancing technique to select an encapsulated TCP socket from the encapsulated set of TCP sockets and forward the packet to a socket-specific incoming queue corresponding to the selected encapsulated TCP socket.
14. The apparatus of claim 13 , wherein, selecting the encapsulated TCP socket comprises:
determining a load of each encapsulated TCP socket; and
selecting an encapsulated TCP socket with least load.
15. The apparatus of claim 14 , wherein, the instructions further cause the one or more processing resources to determine the load of each encapsulated TCP socket by:
determining an occupancy rate of the socket-specific incoming queue associated with each encapsulated TCP socket; or
determining a number of packets queued in the socket-specific incoming queue associated with each encapsulated TCP socket.
16. The apparatus of claim 15 , wherein the selected encapsulated TCP socket with the least load corresponds to a socket-specific incoming queue with a lowest occupancy rate or a smallest number of queued packets.
17. The apparatus of claim 13 , wherein the instructions further cause the one or more processing resources to:
in response to detecting that a network interface associated with a particular encapsulated TCP socket fails, remove the particular encapsulated TCP socket from the local transport-layer virtual socket; and
cause packets queued in the socket-specific incoming queue associated with the removed particular TCP socket to be redistributed to other socket-specific incoming queues associated with other TCP sockets encapsulated within the local transport-layer virtual socket.
18. The apparatus of claim 17 , wherein the instructions further cause the one or more processing resources to:
in response to detecting a recovery of the failed network interface, create a new TCP socket bound to the recovered network interface; and
encapsulate the new TCP socket to the local transport-layer virtual socket.
19. The apparatus of claim 13 , wherein the selected encapsulated TCP socket:
receives, from the remote node, a packet associated with a particular thread executing on the remote node;
determines a destination thread executing on the local node associated with the packet received from the remote node; and
forwards the packet received from the remote node to a thread-specific outgoing queue based on the destination thread.
20. The apparatus of claim 19 , wherein the selected encapsulated TCP socket:
notifies the destination thread to allow the destination thread to read the packet received from the remote node from the thread-specific outgoing queue via the read interface.
21. The apparatus of claim 19 , wherein the thread-specific outgoing queue is thread safe and is accessible to the encapsulated set of TCP sockets.
22. The apparatus of claim 19 , wherein the socket-specific incoming queue is thread safe and is accessible to multiple threads.