Fault-tolerant and highly-available configuration of distributed services
Fault-tolerant and highly available configuration of distributed services including a computer-implemented method for role-based configuration discovery comprising receiving a request comprising an identifier of a role; identifying a first key, in a replica of a distributed configuration store, comprising a first value that matches the role identifier; identifying one or more other key-value pairs associated in the replica with the first key; and returning a response to an entity that sent the request comprising the value of at least one key-value pair that is specific to the role the service has. Also disclosed are techniques for log forwarding.
1. A method for role-based configuration discovery, the method comprising:
receiving a request comprising an identifier of a role, wherein the role represents a capability of a given service when executing;
wherein the request does not identify any particular service having the role;
identifying a first key associated with a service, in a replica of a distributed configuration store, comprising a first value that matches the identifier of the role, wherein the service is a software product or software application executing on one or more hosts in a distributed computing environment;
identifying one or more other key-value pairs associated in the replica with the service associated with the first key;
wherein the one or more other key-value pairs comprise configuration information for the service having the role;
wherein the one or more other key-value pairs comprise at least one key-value pair that is specific to the role that the service has; and
returning a response to the request to an entity that sent the request, the response comprising an identifier of the service and comprising the value of the at least one key-value pair that is specific to the role the service has,
wherein the method is performed on one or more computing devices.
2. The method of claim 1 , wherein the entity that sent the request executes at the same host at which the replica is stored.
3. The method of claim 1 , wherein the request specifies a requested host identifier of a requested host at which to find a matching service, and the method further comprising:
identifying the first key associated with the service, in the replica of the distributed configuration store, comprising the first value that matches the identifier of the role, and a second key associated with the service, comprising a second value that matches the requested host identifier, wherein the service is a software product or software application executing on a specific host associated with requested host identifier in the distributed computing environment.
4. The method of claim 1 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises configuration information for a regular expression pattern for matching against log lines of log files produced by the service.
5. The method of claim 1 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises a specification of a format of date/timestamps in log lines in log files produced by the service.
6. One or more non-transitory computer-readable media storing one or more programs for execution by one or more processors, the one or more programs comprising instructions for:
receiving a request comprising an identifier of a role, wherein the role represents a capability of a given service when executing;
wherein the request does not identify any particular service having the role;
identifying a first key associated with a service, in a replica of a distributed configuration store, comprising a first value that matches the identifier of the role, wherein the service is a software product or software application executing installed on one or more hosts in a distributed computing environment;
identifying one or more other key-value pairs associated in the replica with the service associated with the first key;
wherein the one or more other key-value pairs comprise configuration information for the service having the role;
wherein the one or more other key-value pairs comprise at least one key-value pair that is specific to the role that the service has; and
returning a response to the request to an entity that sent the request, the response comprising an identifier of the service and comprising the value of the at least one key-value pair that is specific to the role the service has.
7. The one or more non-transitory computer-readable media of claim 6 , wherein the entity that sent the request executes at the same host at which the replica is stored.
8. The one or more non-transitory computer-readable media of claim 6 , wherein the request specifies a requested host identifier of a requested host at which to find a matching service, and the instructions further comprising: identifying the first key associated with the service, in the replica of the distributed configuration store, comprising the first value that matches the identifier of the role, and a second key associated with the service, comprising a second value that matches the requested host identifier, wherein the service is a software product or software application executing on a specific host associated with requested host identifier in the distributed computing environment.
9. The one or more non-transitory computer-readable media of claim 6 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises a regular expression pattern for matching against log lines of log files produced by the service.
10. The one or more non-transitory computer-readable media of claim 6 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises a specification of a format of date/timestamps in log lines in log files produced by the service.
11. A system, comprising:
one or more processors;
one or more storage media storing one or more programs for execution by the one or more processors, the one or more programs comprising instructions for:
receiving a request comprising an identifier of a role, wherein the role represents a capability of a given service when executing;
wherein the request does not identify any particular service having the role;
identifying a first key associated with a service, in a replica of a distributed configuration store, comprising a first value that matches the identifier of the role, wherein the service is a software product or software application executing on one or more hosts in a distributed computing environment;
identifying one or more other key-value pairs associated in the replica with the service associated with the first key;
wherein the one or more other key-value pairs comprise configuration information for the service having the role;
wherein the one or more other key-value pairs comprise at least one key-value pair that is specific to the role that the service has; and
returning a response to the request to an entity that sent the request, the response comprising an identifier of the service and comprising the value of the at least one key-value pair that is specific to the role the service has.
12. The system of claim 11 , wherein the entity that sent the request executes at the same host at which the replica is stored.
13. The system of claim 11 , wherein the request specifies a requested host identifier of a requested host at which to find a matching service, and the instructions further comprising:
identifying the first key associated with the service, in the replica of the distributed configuration store, comprising the first value that matches the identifier of the role, and a second key associated with the service, comprising a second value that matches the requested host identifier, wherein the service is a software product or software application executing on a specific host associated with requested host identifier in the distributed computing environment.
14. The system of claim 11 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises a regular expression pattern for matching against log lines of log files produced by the service.
15. The system of claim 11 , wherein one or more of the at least one key-value pairs comprise configuration information that is specific to the role the service has comprises a specification of a format of date/timestamps in log lines in log files produced by the service.
16. The system of claim 11 , wherein a value of the first key in the replica names one or more roles that the service has, the value comprising the first value.
17. The system of claim 11 , wherein the first key in the replica comprises a name of the service.
18. The system of claim 11 , wherein each key of the one or more other key-value pairs in the replica comprise the identifier of the role.
19. The system of claim 11 , wherein each key of the one or more other key-value pairs in the replica comprise a name of the service.
20. The system of claim 11 , wherein:
a value in the replica, of the at least one key-value pair that is specific to the role the service has, comprises a variable tag;
the one or more programs further comprise replacing, in a copy of the value in the replica, the variable tag with a name of a host at which the entity executes; and
the value returned in the response is based on the copy of the value in which the variable tag is replaced with the name of the host.