Applying a consistent hash to a distributed domain name server cache
Implementations are provided herein for using a distributed DNS cache that is distributed among nodes of a cluster of nodes operating as a distributed file system. A consistent hash can be used to determine a resolution node for a DNS request asking to resolve a specific FQDN. The inputs to the consistent hashing algorithm can be the FQDN, a set of available nodes, and a set of all possible nodes. By using a consistent hash, the process can reduce the sensitivity of the hashing algorithm output from changing when nodes added or removed as participants in the distributed cache service. As each node in the cluster of nodes can independently calculate a resolution node for specific FQDN, there is no need to send control messages between nodes, as each node only needs to be aware of the set of available nodes in the cluster of nodes.
1. A method comprising:
maintaining a distributed Domain Name Server (“DNS”) cache among nodes of a cluster of nodes operating as a distributed file system;
receiving by a first node among the cluster of nodes a request from a client to resolve a Fully Qualified Domain Name (“FQDN”);
using a consistent hash to associate the request with a resolution node among the cluster of nodes, wherein a set of inputs to the consistent hash include at least the FQDN, a set of available nodes among the cluster of nodes, and the cluster of nodes;
directing the request to the resolution node;
determining by the resolution node a DNS result based on the request; and
sending the DNS result to the client.
2. The method of claim 1 further comprising:
determining whether the FQDN is associated with a previously cached DNS result within the distributed DNS cache of the resolution node; and
in response to the FQDN being associated with the previously cached DNS result, resolving the DNS result as the previously cached DNS result.
3. The method of claim 2 , further comprising:
in response to the FQDN not being associated with the previously cached DNS result:
looking up the DNS result in a DNS server external to the cluster of nodes; and
storing the DNS result in the distributed DNS cache of the resolution node.
4. The method of claim 1 , wherein nodes among the set of available nodes are weighted based on the capacity of nodes to process DNS cache requests.
5. The method of claim 1 , further comprising:
determining a group change among the cluster of nodes wherein the group change indicates at least one of at least one new node among the cluster of nodes or at least one missing node among the cluster of nodes;
asynchronously notifying nodes among the cluster of nodes of the group change; and
using, by the resolution node, the consistent hash to associate the request with a modified resolution node wherein the set of inputs to the consistent hash are modified based on the group change.
6. The method of claim 5 , further comprising:
in response to the modified resolution node and the resolution node being different nodes among the cluster of nodes, letting a previously cached DNS result associated with the request to expire within the distributed DNS cache of the resolution node.
7. The method of claim 1 wherein the set of available nodes is based on a set of permissions associated with the client.
8. A system comprising a cluster of nodes operating as a distributed file system wherein nodes among the cluster of nodes contain at least one storage device and at least one hardware processor configured to:
maintain a distributed Domain Name Server (“DNS”) cache among nodes of the cluster of nodes;
receive by a first node among the cluster of nodes a request from a client to resolve a Fully Qualified Domain Name (“FQDN”);
use a consistent hash to associate the request with a resolution node among the cluster of nodes, wherein a set of inputs to the consistent hash include at least the FQDN, a set of available nodes among the cluster of nodes, and the cluster of nodes;
direct the request to the resolution node;
determine by the resolution node a DNS result based on the request; and
send the DNS result to the client.
9. The system of claim 8 , further configured to:
determine whether the FQDN is associated with a previously cached DNS result within the distributed DNS cache of the resolution node; and
in response to the FQDN being associated with the previously cached DNS result, resolve the DNS result as the previously cached DNS result.
10. The system of claim 9 , further configure to:
in response to the FQDN not being associated with the previously cached DNS result:
look up the DNS result in a DNS server external to the cluster of nodes; and
store the DNS result in the distributed DNS cache of the resolution node.
11. The system of claim 8 , wherein nodes among the set of available nodes are weighted based on the capacity of nodes to process DNS cache requests.
12. The system of claim 8 , further configured to:
determine a group change among the cluster of nodes wherein the group change indicates at least one of at least one new node among the cluster of nodes or at least one missing node among the cluster of nodes;
asynchronously notify nodes among the cluster of nodes of the group change; and
use, by the resolution node, the consistent hash to associate the request with a modified resolution node wherein the set of inputs to the consistent hash are modified based on the group change.
13. The system of claim 12 , further configured to:
in response to the modified resolution node and the resolution node being different nodes among the cluster of nodes, let a previously cached DNS result associated with the request to expire within the distributed DNS cache of the resolution node.
14. The system of claim 8 wherein the set of available nodes is based on a set of permissions associated with the client.
15. A non-transitory computer readable medium with program instructions stored thereon to perform the following acts:
maintaining a distributed Domain Name Server (“DNS”) cache among nodes of a cluster of nodes operating as a distributed file system;
receiving by a first node among the cluster of nodes a request from a client to resolve a Fully Qualified Domain Name (“FQDN”);
using a consistent hash to associate the request with a resolution node among the cluster of nodes, wherein a set of inputs to the consistent hash include at least the FQDN, a set of available nodes among the cluster of nodes, and the cluster of nodes;
directing the request to the resolution node;
determining by the resolution node a DNS result based on the request; and
sending the DNS result to the client.
16. The non-transitory computer readable medium of claim 15 , with program instructions stored thereon to further perform the following acts:
determining whether the FQDN is associated with a previously cached DNS result within the distributed DNS cache of the resolution node; and
in response to the FQDN being associated with the previously cached DNS result, resolving the DNS result as the previously cached DNS result.
17. The non-transitory computer readable medium of claim 16 , with program instructions stored thereon to further perform the following acts:
in response to the FQDN not being associated with the previously cached DNS result:
looking up the DNS result in a DNS server external to the cluster of nodes; and
storing the DNS result in the distributed DNS cache of the resolution node.
18. The non-transitory computer readable medium of claim 15 , wherein nodes among the set of available nodes are weighted based on the capacity of nodes to process DNS cache requests.
19. The non-transitory computer readable medium of claim 15 , with program instructions stored thereon to further perform the following acts:
determining a group change among the cluster of nodes wherein the group change indicates at least one of at least one new node among the cluster of nodes or at least one missing node among the cluster of nodes;
asynchronously notifying nodes among the cluster of nodes of the group change; and
using, by the resolution node, the consistent hash to associate the request with a modified resolution node wherein the set of inputs to the consistent hash are modified based on the group change.
20. The non-transitory computer readable medium of claim 19 , with program instructions stored thereon to further perform the following acts:
in response to the modified resolution node and the resolution node being different nodes among the cluster of nodes, letting a previously cached DNS result associated with the request to expire within the distributed DNS cache of the resolution node.