Management and orchestration of microservices
Techniques are provided for microservices management and orchestration. A chart package is selectively retrieved from a chart repository based upon the chart package corresponding to a set of services to host within a cluster and dependencies amongst the set of services. A set of container images may be retrieved from a container repository based upon the set of container images corresponding to the set of services. A cluster may be created within a computing environment. The set of services may be deployed as resources of the computing environment within the cluster and the dependencies may be configured using the chart package and the set of container images.
1 . A method comprising:
receiving, by a job manager associated with a set of services deployed within a cluster as resources in a computing environment, a job request message having a topic subscribed to by the job manager;
invoking, by the job manager, a workflow engine to create an instance of a workflow comprising one or more tasks corresponding to a job associated with the job request message;
implementing, by the workflow engine, the workflow to insert a first task message into an exchange, wherein the first task message comprises a first routing key and corresponds to a first task of the one or more tasks;
routing the first task message to a first service in the set of services based upon the first service corresponding to the routing key;
executing, by the first service, the first task;
in response to the job manager receiving a first message through the exchange that the first task has been executed, updating, by the job manager, a status of the job to indicate that the first task of the job is complete; and
in response to the workflow engine receiving a first completion message through the exchange that the first task has been executed, implementing the workflow to insert a second message comprising a second routing key into the exchange, wherein the second message corresponds to a second task of the one or more tasks of the job.
2 . The method of claim 1 , further comprising:
assigning a field to the job request message; and
utilizing information within the field to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
3 . The method of claim 1 , further comprising receiving, the job request message, from a scheduler through a job exchange, wherein the scheduler generated the job request message based upon a schedule associated with the job.
4 . The method of claim 1 , further comprising:
facilitating communication of messages relating to an asynchronous task over a first communication channel;
facilitating communication of messages relating to a synchronous task over a second communication channel; and
facilitating communication of incoming job request messages to the set of services utilizing REST communication.
5 . The method of claim 1 , further comprising evaluating a job type tag of the job request message against job types of initialized jobs to determine whether the job can be executed.
6 . The method of claim 1 , further comprising assigning a field to the job request message, wherein the field comprises at least one of a job type tag field, an object type field, an object identifier field, a maximum jobs in queue field, a wait for tags field, a wait for job identifiers field, a queue timeout field, a retry count field, or a max retry field, wherein the field is utilized to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
7 . The method of claim 1 , further comprising assigning a field to the job request message, wherein the field comprises a job type tag field, and wherein the field is utilized to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
8 . The method of claim 1 , further comprising assigning a field to the job request message, wherein the field comprises a wait for job identifiers field, and wherein the field is utilized to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
9 . A non-transitory machine readable medium comprising instructions, which when executed by a machine, causes the machine to perform operations comprising:
receiving, by a job manager associated with a set of services deployed within a cluster as resources in a computing environment, a job request message having a topic subscribed to by the job manager;
invoking, by the job manager, a workflow engine to create an instance of a workflow comprising one or more tasks corresponding to a job associated with the job request message;
implementing, by the workflow engine, the workflow to insert a first task message into an exchange, wherein the first task message comprises a first routing key and corresponds to a first task of the one or more tasks;
routing the first task message to a first service in the set of services based upon the first service corresponding to the routing key;
executing, by the first service, the first task;
in response to the job manager receiving a first message through the exchange that the first task has been executed, updating, by the job manager, a status of the job to indicate that the first task of the job is complete; and
in response to the workflow engine receiving a first completion message through the exchange that the first task has been executed, implementing the workflow to insert a second message comprising a second routing key into the exchange, wherein the second message corresponds to a second task of the one or more tasks of the job.
10 . The non-transitory machine readable medium of claim 9 , wherein the operations comprise;
assigning a field to the job request message; and
utilizing information within the field to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
11 . The non-transitory machine readable medium of claim 9 , wherein the operations comprise receiving, the job request message, from a scheduler through a job exchange, wherein the scheduler generated the job request message based upon a schedule associated with the job.
12 . The non-transitory machine readable medium of claim 9 , wherein the operations comprise:
facilitating communication of messages relating to an asynchronous task over a first communication channel;
facilitating communication of messages relating to a synchronous task over a second communication channel; and
facilitating communication of incoming job request messages to the set of services utilizing REST communication.
13 . The non-transitory machine readable medium of claim 9 , wherein the operations comprise evaluating a job type tag of the job request message against job types of initialized jobs to determine whether the job can be executed.
14 . The non-transitory machine readable medium of claim 9 , wherein the operations comprise assigning a field to the job request message, wherein the field comprises at least one of a job type tag field, an object type field, an object identifier field, a maximum jobs in queue field, a wait for tags field, a wait for job identifiers field, a queue timeout field, a retry count field, or a max retry field, wherein the field is utilized to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
15 . A computing device comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the computing device to:
receive, by a job manager associated with a set of services deployed within a cluster as resources in a computing environment, a job request message having a topic subscribed to by the job manager;
invoke, by the job manager, a workflow engine to create an instance of a workflow comprising one or more tasks corresponding to a job associated with the job request message;
implement, by the workflow engine, the workflow to insert a first task message into an exchange, wherein the first task message comprises a first routing key and corresponds to a first task of the one or more tasks;
route the first task message to a first service in the set of services based upon the first service corresponding to the routing key;
executed, by the first service, the first task;
in response to the job manager receiving a first message through the exchange that the first task has been executed, update, by the job manager, a status of the job to indicate that the first task of the job is complete; and
in response to the workflow engine receiving a first completion message through the exchange that the first task has been executed, implement the workflow to insert a second message comprising a second routing key into the exchange, wherein the second message corresponds to a second task of the one or more tasks of the job.
16 . The computing device of claim 15 , wherein the machine executable code further causes the computing device to:
assign a field to the job request message; and
utilize information within the field to determine whether the job can execute, should wait, or be failed based upon other pending jobs.
17 . The computing device of claim 15 , wherein the machine executable code further causes the computing device to receive, the job request message, from a scheduler through a job exchange, wherein the scheduler generated the job request message based upon a schedule associated with the job.
18 . The computing device of claim 15 , wherein the machine executable code further causes the computing device to:
facilitate communication of messages relating to an asynchronous task over a first communication channel;
facilitate communication of messages relating to a synchronous task over a second communication channel; and
facilitate communication of incoming job request messages to the set of services utilizing REST communication.
19 . The computing device of claim 15 , wherein the machine executable code further causes the computing device to evaluate a job type tag of the job request message against job types of initialized jobs to determine whether the job can be executed.
20 . The computing device of claim 15 , wherein the machine executable code further causes the computing device to assign a field to the job request message, wherein the field comprises at least one of a job type tag field, an object type field, an object identifier field, a maximum jobs in queue field, a wait for tags field, a wait for job identifiers field, a queue timeout field, a retry count field, or a max retry field, wherein the field is utilized to determine whether the job can execute, should wait, or be failed based upon other pending jobs.