Failure recovery for cloud-based services
The technology disclosed relates to failure recovery in cloud-based services. In particular, the technology disclosed relates to a service instance BA that identifies a service instance BB as having a secondary role for packets carrying a stream affinity code which is specified in a service map distributed to service instances. Service instance BA state information is synchronized with the service instance BB after processing a first packet. After failure of the service instance BA, a service instance AA receives an updated service map, and prepares to forward to the service instance BA a second packet. The second packet includes a same stream affinity code as the first packet forwarded before the failure. The updated service map is used to determine that the service instance BB is available and servicing the same stream affinity code as the service instance BA. The second packet is forwarded to the service instance BB.
1. A computer-implemented method of recovery from failure of a service instance, in a service chain of services that perform at least services A and B, using service instance AA and service instances BA and BB to perform the services A and B, respectively, and performing actions including:
the service instance BA identifying the service instance BB as having a secondary role for packets carrying a stream affinity code which is specified in a service map distributed to service instances, and synchronizing service instance BA state information with the service instance BB after processing a first packet;
after failure of the service instance BA, the service instance AA receiving an updated service map, and preparing to forward to the service instance BA a second packet, wherein the second packet includes a same stream affinity code as the first packet forwarded before the failure;
determining from the updated service map that the service instance BB is available and servicing the same stream affinity code as the service instance BA; and
forwarding the second packet to the service instance BB instead of the service instance BA.
2. The computer-implemented method of claim 1 , wherein the service chain is a security service chain for a subscriber and at least the service B is a security service.
3. The computer-implemented method of claim 1 , wherein the stream affinity code is included in an added header as an added IP header as IP source and destination.
4. The computer-implemented method of claim 1 , further including a packet carrying a service chain for a subscriber in an added packet header and the service B being among services specified in the service chain.
5. The computer-implemented method of claim 1 , wherein instances of the service A and the service B run in containers and the containers are hosted in pods.
6. The computer-implemented method of claim 1 , wherein instances of the service A and the service B are implemented on virtual machines, bare metal servers or custom hardware.
7. The computer-implemented method of claim 1 , wherein failure of the service instance BA is detected by a monitoring agent, and performing actions further including:
monitoring the service instance BA for packet processing activity; and
causing updating of the service map for the service B to remove the service instance BA from availability, should it be inactive for a configurable predetermined amount of time.
8. The computer-implemented method of claim 1 , further including the service instance BB, and performing actions further including:
processing the second packet and based on the processing:
identifying a next service, among at least two additional services to which a subscriber has subscribed, that should next handle the second packet; and
routing the processed second packet to the identified next service upon egress from the service instance BB.
9. The computer-implemented method of claim 1 , further including actions of processing a plurality of packets in a stream through the service chain of services and directing the packets for processing, as a document, to a cloud access security broker (abbreviated CASB) that controls exfiltration of sensitive content in documents stored on cloud-based services in use by users of an organization, by monitoring manipulation of the documents.
10. A tangible non-transitory computer readable storage media, including program instructions loaded into memory that, when executed on processors, cause the processors to implement a computer-implemented method of recovery from failure of a service instance, in a service chain of services that perform at least services A and B, using service instance AA and service instances BA and BB to perform the services A and B, respectively, the computer-implemented method performing actions including:
the service instance BA identifying the service instance BB as having a secondary role for packets carrying a stream affinity code which is specified in a service map distributed to service instances, and synchronizing service instance BA state information with the service instance BB after processing a first packet;
after failure of the service instance BA, the service instance AA receiving an updated service map, and preparing to forward to the service instance BA a second packet, wherein the second packet includes a same stream affinity code as the first packet forwarded before the failure;
determining from the updated service map that the service instance BB is available and servicing the same stream affinity code as the service instance BA; and
forwarding the second packet to the service instance BB instead of the service instance BA.
11. The tangible non-transitory computer readable storage media of claim 10 , wherein the service chain is a security service chain for a subscriber and at least the service B is a security service.
12. The tangible non-transitory computer readable storage media of claim 10 , wherein the stream affinity code is included in an added header as an added IP header as IP source and destination.
13. The tangible non-transitory computer readable storage media of claim 10 , wherein instances of the service A and the service B run in containers and the containers are hosted in pods.
14. The tangible non-transitory computer readable storage media of claim 10 , wherein instances of the service A and the service B are implemented on virtual machines, bare metal servers or custom hardware.
15. The tangible non-transitory computer readable storage media of claim 10 , further including the service instance BB, and performing actions further including:
processing the second packet and based on the processing:
identifying a next service, among at least two additional services to which the subscriber has subscribed, that should next handle the packet; and
routing the processed second packet to the identified next service upon egress from the service instance BB.
16. A system for improved recovery from failure of a service instance, in a service chain of services that perform at least services A and B, using service instance AA and service instances BA and BB to perform the services A and B, respectively, the system including a processor, memory coupled to the processor, and computer instructions from the non-transitory computer readable storage media of claim 10 loaded into the memory.
17. The system of claim 16 , wherein the stream affinity code is included in an added header as an added IP header as IP source and destination.
18. The system of claim 16 , wherein a packet carries a service chain for a subscriber in an added packet header and the service B is among services specified in the service chain.
19. The system of claim 16 , wherein failure of the service instance BA is detected by a monitoring agent, and the system performing actions further including:
monitoring the service instance BA, for packet processing activity; and
causing updating of the service map for the service B to remove the service instance BA from availability should it be inactive for a configurable predetermined amount of time.
20. The system of claim 16 , further configured to process a plurality of packets in a stream through the service chain of services and directing the packets for processing, as a document, to a cloud access security broker (abbreviated CASB) that controls exfiltration of sensitive content in documents stored on cloud-based services in use by users of an organization, by monitoring manipulation of the documents.