Cooperative caching for fast and scalable policy sharing in cloud environments
Systems, methods, and computer-readable media for distributing policies in a SDN environment through chunking. A policy can be chunked into a plurality of policy chunks having corresponding chunk identifications at a controller of a SDN environment. Each of the plurality of policy chunks can be hashed to create corresponding chunk hashes for each of the plurality of policy chunks. Further, the plurality of policy chunks, the chunk identifications of the plurality of policy chunks, and the chunk hashes of the plurality of policy chunks can be distributed from the controller of the SDN environment to an intermediate policy node in a fabric of the SDN environment. The chunk hashes and the chunk identifications of the plurality of policy chunks can be used to control distribution of the plurality of policy chunks to one or more edge nodes in the SDN environment.
1. A method comprising:
chunking, at a controller of a software-defined networking (SDN) environment, a policy into a plurality of policy chunks having corresponding chunk identifications;
hashing each of the plurality of policy chunks to create corresponding chunk hashes for each of the plurality of policy chunks;
distributing, from the controller of the SDN environment, the plurality of policy chunks, the chunk identifications of the plurality of policy chunks, and the chunk hashes of the plurality of policy chunks to an intermediate policy node in a fabric of the SDN environment; and
retrieving, by one or more edge nodes in the SDN environment, the plurality of policy chunks using the chunk identifications and the chunk hashes either or both directly from the intermediate policy node or through local retrieval amongst the one or more edge nodes.
2. The method of claim 1 , further comprising transmitting a chunk identification and a chunk hash of a policy chunk of the plurality of policy chunks from the intermediate policy node to an edge node of the one or more edge nodes in the SDN environment, wherein the edge node is configured to retrieve the policy chunk using the chunk identification and the chunk hash and implement the policy for an endpoint associated with the edge node using the policy chunk.
3. The method of claim 2 , wherein the chunk identification and the chunk hash of the policy chunk is transmitted from the intermediate policy node to the edge node in response to a request for the policy chunk transmitted from the edge node to the intermediate policy node.
4. The method of claim 3 , wherein the policy associated with the policy chunk is specific to the endpoint and the edge node transmits the request for the policy to the intermediate policy node in response to the endpoint joining an endpoint group associated with the edge node.
5. The method of claim 2 , wherein the edge node is configured to locally retrieve the policy chunk from another edge node using the chunk identification and the chunk hash of the policy chunk.
6. The method of claim 5 , wherein the edge node is configured to locally retrieve the policy chunk from the another edge node using a distributed key-value store distributed across the one or more edge nodes, including the edge node and the another edge node, wherein the distributed key-value store uniquely associates the policy chunk with the another edge node based on the policy chunk residing locally at the another edge node.
7. The method of claim 6 , further comprising updating the distributed key-value store to uniquely associate the edge node with the policy chunk based on the policy chunk residing locally at the edge node after being retrieved from the another edge node.
8. The method of claim 2 , wherein the edge node is configured to retrieve the policy chunk from the intermediate policy node.
9. The method of claim 8 , wherein the edge node is configured to locally search for the policy chunk at one or more other edge nodes and retrieve the policy chunk from the intermediate policy node in response to the policy chunk being absent from the one or more other edge nodes.
10. The method of claim 2 , wherein the edge node is further configured to use the chunk hash to verify that the retrieved policy chunk is a correct policy chunk corresponding to the chunk identification.
11. The method of claim 2 , further comprising transmitting, from the intermediate policy node to the edge node, a configuration hash for the policy, wherein the edge node is configured to use the configuration hash to determine whether the edge node has received all policy chunks of the plurality of policy chunks for implementing the policy at the endpoint.
12. The method of claim 2 , further comprising:
transmitting, from the intermediate policy node to the edge node, a notification indicating that the policy chunk is invalid; and
transmitting, from the intermediate policy node to the edge node, a new chunk identification and a new chunk hash of a new policy chunk of the policy, wherein the edge node is configured to retrieve the new policy chunk using the new chunk identification and the new chunk hash.
13. The method of claim 2 , further comprising removing the policy chunk from the edge node and disassociating the edge node with the policy chunk in response to the endpoint disassociating from the edge node.
14. The method of claim 1 , wherein at least a portion of the plurality of policy chunks are fixed size policy chunks.
15. The method of claim 1 , wherein at least a portion of the plurality of policy chunks are variable size policy chunks and the policy is chunked into the variable size policy chunks based on characteristics of the policy.
16. A system comprising:
one or more processors; and
at least one computer-readable storage medium having stored therein instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
chunking, at a controller of a software-defined networking (SDN) environment, a policy into a plurality of policy chunks having corresponding chunk identifications;
hashing each of the plurality of policy chunks to create corresponding chunk hashes for each of the plurality of policy chunks;
distributing, from the controller of the SDN environment, the plurality of policy chunks, the chunk identifications of the plurality of policy chunks, and the chunk hashes of the plurality of policy chunks to an intermediate policy node in a fabric of the SDN environment; and
transmitting a chunk identification and a chunk hash of a policy chunk of the plurality of policy chunks from the intermediate policy node to an edge node of one or more edge nodes in the SDN environment;
retrieving, by the edge node, the policy chunk using the chunk identification and the chunk hash either or both directly from the intermediate policy node or through local retrieval amongst the one or more edge nodes.
17. The system of claim 16 , wherein the edge node is configured to locally retrieve the policy chunk from another edge node using the chunk identification of the policy chunk, the chunk hash of the policy chunk, and a distributed key-value store distributed across the one or more edge nodes, including the edge node and the another edge node, wherein the distributed key-value store uniquely associates the policy chunk with the another edge node based on the policy chunk residing locally at the another edge node.
18. The system of claim 16 , wherein the edge node is configured to locally search for the policy chunk at one or more other edge nodes and retrieve the policy chunk from the intermediate policy node in response to the policy chunk being absent from the one or more other edge nodes.
19. The system of claim 16 , wherein the instructions which, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
transmitting, from the intermediate policy node to the edge node, a notification indicating that the policy chunk is invalid; and
transmitting, from the intermediate policy node to the edge node, a new chunk identification and a new chunk hash of a new policy chunk of the policy, wherein the edge node is configured to retrieve the new policy chunk using the new chunk identification and the new chunk hash.
20. A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform operations comprising:
chunking, at a controller of a software-defined networking (SDN) environment, a policy into a plurality of policy chunks having corresponding chunk identifications, wherein at least a portion of the plurality of policy chunk are variable size policy chunks and the policy is chunked into the variable size policy chunks based on characteristics of the policy;
hashing each of the plurality of policy chunks to create corresponding chunk hashes for each of the plurality of policy chunks; and
distributing, from the controller of the SDN environment, the plurality of policy chunks, the chunk identifications of the plurality of policy chunks, and the chunk hashes of the plurality of policy chunks to an intermediate policy node in a fabric of the SDN environment; and
retrieving, by one or more edge nodes in the SDN environment, the plurality of policy chunks using the chunk identifications and the chunk hashes either or both directly from the intermediate policy node or through local retrieval amongst the one or more edge nodes.