Distributed resilient load-balancing for multipath transport protocols
Techniques are described for providing a distributed application load-balancing architecture that supports multipath transport protocol for client devices connecting to an application service. Rather than having client devices generate new network five-tuples for new subflows to the application servers, the techniques described herein include shifting the burden to the application servers to ensure that the new network five-tuples land in the same bucket in the consistent hashing table. The application servers may receive a hashing function utilized by the load balancers to generate the hash of the network five-tuple. By having the application servers generate the hashes, the load balancers are able to continue stateless, low-level processing of the packets to route them to the correct application servers. In this way, additional subflows can be opened for client devices according to a multipath transport protocol while ensuring that the subflows are routed to the correct application server.
1. A server, disposed in a distributed application architecture, hosting a replicated application of an application service, the server comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, via a load balancer of the distributed application architecture, a request packet indicating a request to establish a multipath connection between the server and a client device, the request packet comprising first values indicating at least a first source port and a first source address of the client device, a first destination port of the server, and a destination address associated with the application service, wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the request packet comprises receiving a TCP SYN packet that includes a five-tuple comprising the first source port and the first source address of the client device, the first destination port of the server, the destination address associated with the application service, and an indication of TCP protocol;
sending an acknowledgment packet to the client device, the acknowledgement packet including a flag to prevent the client device from opening additional subflows on the first destination port;
completing opening of a first subflow on the multipath connection between the first destination port of the client device and the first destination port of the server;
receiving, via the first subflow of the multipath connection, an advertisement packet from the client device indicating a second source port and a second source address associated with the client device;
computing, using a hashing function associated with the load balancer, a second destination port of the server such that a first hash of the first values maps to a same consistent-hashing bucket of the load balancer as a second hash of second values, the second values indicating the second source port and the second source address of the client device, the second destination port of the server, and the destination address associated with the application service, wherein the second destination port is different than the first destination port, wherein the first hash and the second hash mapping to the same consistent-hashing bucket results in the first subflow and second subflow being routed to the server by the load balancer;
sending, to the client device, an address-advertisement message indicating the second destination port; and
completing opening of a second subflow on the multipath connection between the second destination port of the client device and the second destination port of the server.
2. The server of claim 1 , the operations further comprising:
receiving the hashing function from the load balancer, wherein the hashing function is utilized by the load balancer to map network five-tuples to consistent-hashing buckets,
wherein the computing the second destination port of the server comprises:
using the hashing function on the first values resulting in the first hash of the first values; and
determining the second destination port included in the second values such that using the hashing function on the second values results in the second hash that corresponds to the first hash.
3. The server of claim 2 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection;
the first values comprise a first network five-tuple;
the second values comprise a second network five-tuple.
4. The server of claim 1 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
sending the acknowledgement packet to the client device comprises sending a TCP SYN with an MP_CAPABLE option indicating that the server is capable of MP-TCP and a C flag marked to 1.
5. The server of claim 1 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the advertisement packet from the client device comprises receiving an ADD_ADDR packet from the client device via the first subflow, the ADD_ADDR packet indicating the second source port and the second source address associated with the client device.
6. The server of claim 1 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
sending the address-advertisement message indicating the second destination port comprises sending an ADD_ADDR packet to the client device, the ADD_ADDR packet comprising a two-tuple that includes indications of the second destination port and the destination address associated with the application service.
7. The server of claim 1 , wherein computing the second destination port of the server such that the first hash of the first values corresponds to the second hash of second values includes determining that the first hash is equal to the second hash.
8. A method performed at least partly by a server disposed in a distributed application architecture and hosting a replicated application of an application service, the method comprising:
receiving, at a server and via a load balancer of the distributed application architecture, a request packet indicating a request to establish a multipath connection between the server and a client device, the request packet comprising first values indicating at least a first source port and a first source address of the client device, a first destination port of the server, and a destination address associated with the application service, wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the request packet comprises receiving a TCP SYN packet that includes a five-tuple comprising the first source port and the first source address of the client device, the first destination port of the server, the destination address associated with the application service, and an indication of TCP protocol;
sending an acknowledgment packet to the client device, the acknowledgement packet including a flag to prevent the client device from opening additional subflows on the first destination port;
completing opening of a first subflow on the multipath connection between the first destination port of the client device and the first destination port of the server;
receiving, via the first subflow of the multipath connection, an advertisement packet from the client device indicating a second source port and a second source address associated with the client device;
computing, using a hashing function associated with the load balancer, a second destination port of the server such that a first hash of the first values maps to a same consistent-hashing bucket of the load balancer as a second hash of second values, the second values indicating the second source port and the second source address of the client device, the second destination port of the server, and the destination address associated with the application service, wherein the second destination port is different than the first destination port, wherein the first hash and the second hash mapping to the same consistent-hashing bucket results in the first subflow and second subflow being routed to the server by the load balancer;
sending, to the client device, an address-advertisement message indicating the second destination port; and
completing opening of a second subflow on the multipath connection between the second destination port of the client device and the second destination port of the server.
9. The method of claim 8 , the further comprising:
receiving the hashing function from the load balancer, wherein the hashing function is utilized by the load balancer to map network five-tuples to consistent-hashing buckets,
wherein the computing the second destination port of the server comprises:
using the hashing function on the first values resulting in the first hash of the first values; and
determining the second destination port included in the second values such that using the hashing function on the second values results in the second hash that corresponds to the first hash.
10. The method of claim 9 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection;
the first values comprise a first network five-tuple;
the second values comprise a second network five-tuple.
11. The method of claim 8 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
sending the acknowledgement packet to the client device comprises sending a TCP SYN with an MP_CAPABLE option indicating that the server is capable of MP-TCP and a C flag marked to 1.
12. The method of claim 8 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the advertisement packet from the client device comprises receiving an ADD_ADDR packet from the client device via the first subflow, the ADD_ADDR packet indicating the second source port and the second source address associated with the client device.
13. The method of claim 8 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
sending the address-advertisement message indicating the second destination port comprises sending an ADD_ADDR packet to the client device, the ADD_ADDR packet comprising a two-tuple that includes indications of the second destination port and the destination address associated with the application service.
14. A system, disposed in a distributed application architecture, hosting a replicated application of an application service, the system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, via a load balancer of the distributed application architecture, a request packet indicating a request to establish a multipath connection between the system and a client device, the request packet comprising first values indicating at least a first source port and a first source address of the client device, a first destination port of the system, and a destination address associated with the application service, wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the request packet comprises receiving a TCP SYN packet that includes a five-tuple comprising the first source port and the first source address of the client device, the first destination port of the system, the destination address associated with the application service, and an indication of TCP protocol;
sending an acknowledgment packet to the client device, the acknowledgement packet including a flag to prevent the client device from opening additional subflows on the first destination port;
completing opening of a first subflow on the multipath connection between the first destination port of the client device and the first destination port of the system;
receiving, via the first subflow of the multipath connection, an advertisement packet from the client device indicating a second source port and a second source address associated with the client device;
computing, using a hashing function associated with the load balancer, a second destination port of the system such that a first hash of the first values maps to a same consistent-hashing bucket of the load balancer as a second hash of second values, the second values indicating the second source port and the second source address of the client device, the second destination port of the system, and the destination address associated with the application service, wherein the second destination port is different than the first destination port, wherein the first hash and the second hash mapping to the same consistent-hashing bucket results in the first subflow and second subflow being routed to the system by the load balancer;
sending, to the client device, an address-advertisement message indicating the second destination port; and
completing opening of a second subflow on the multipath connection between the second destination port of the client device and the second destination port of the system.
15. The system of claim 14 , the operations further comprising:
receiving the hashing function from the load balancer, wherein the hashing function is utilized by the load balancer to map network five-tuples to consistent-hashing buckets,
wherein the computing the second destination port of the system comprises:
using the hashing function on the first values resulting in the first hash of the first values; and
determining the second destination port included in the second values such that using the hashing function on the second values results in the second hash that corresponds to the first hash.
16. The system of claim 15 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection;
the first values comprise a first network five-tuple;
the second values comprise a second network five-tuple.
17. The system of claim 14 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
sending the acknowledgement packet to the client device comprises sending a TCP SYN with an MP_CAPABLE option indicating that the system is capable of MP-TCP and a C flag marked to 1.
18. The system of claim 14 , wherein:
the multipath connection comprises a Multipath Transmission Control Protocol (MP-TCP) connection; and
receiving the advertisement packet from the client device comprises receiving an ADD_ADDR packet from the client device via the first subflow, the ADD_ADDR packet indicating the second source port and the second source address associated with the client device.