Reducing start latency of serverless microservices

1 . A computer-implemented method for reducing start latency of serverless microservices, comprising:

detecting an incoming request or event to an application in a serverless microservice environment based on identifying a particular event topic associated with the incoming request or event from one or more potential event topics, wherein identifying the particular event topic initiates invocations of one or more microservices of the application;

monitoring a replica count of an initial microservice, wherein the initial microservice controls the incoming request or event;

selecting an amount of selected microservices from the one or more microservices of the application by detecting a change in the replica count from zero, wherein the amount of selected microservices perform a task of the incoming request or event, and wherein the task applies one or more predefined application-specific rules to one or more elements of the incoming request or event to determine the amount of selected microservices;

triggering scaling up activation of the selected microservices of the application by sending a request to a defined endpoint of each of the selected microservices that activates scaling without causing the selected microservices to perform work; and

simultaneously invoking the selected microservices of the application to match the amount of selected microservices, thereby reducing a latency associated with a cold start to a startup time of a slowest microservice of the selected microservices rather than a cumulative startup time.

2 . The method of claim 1 , wherein selecting the amount of selected microservices of the application further includes:

extracting the one or more elements from a payload or one or more attributes of the incoming request or event, wherein the one or more elements extracted from the payload or the one or more attributes are extracted elements; and

applying the one or more predefined application-specific rules to the extracted elements.

3 . The method of claim 2 , wherein extracting the one or more elements further includes:

parsing a message of an incoming event according to a structured format for the event.

4 . The method of claim 1 , wherein selecting the amount of selected microservices of the application further includes: classifying the incoming request or event, wherein classifying the incoming request or event includes:

applying the predefined application-specific rules to the one or more elements of the incoming request or event; and

mapping a classification to a subset of the one or more microservices of the application by querying an orchestration system API to identify one or more microservice deployments having an orchestration label that matches the classification.

5 . The method of claim 4 , further including:

applying a label to a microservice deployment, the label associated with the pre-defined application-specific rules, wherein a selector uses the label to identify the one or more microservices of the application for activating; and

mapping the label to the subset of the one or more microservices of the application, wherein the label is applied to a list of deployments of the subset the one or more microservices of the application.

6 . The method of claim 5 , further including:

querying a microservice orchestrator application programming interface to obtain one or more replica counts for the amount of selected microservices.

7 . The method of claim 4 , further comprising:

identifying one or more components of the application; and predicting, based on the incoming request or event, at least one component of the one or more components of the application required to process the incoming request or event; and

proactively scaling only the at least one component of the application.

8 . The method of claim 1 , wherein detecting an incoming event further includes:

triggering a scaling service to select the amount of selected microservices by observing a same incoming event queue as an initial microservice.

9 . The method of claim 8 , wherein the scaling service further includes:

sending a wake-up event to the incoming event queues to scale up the amount of selected microservices, wherein the wake-up event is consumed by the microservice to remove the wake-up event from the incoming event queues.

10 . The method of claim 1 , further including:

sending a request to a defined endpoint, provided by the one or more selected microservices, to activate the scaling up activation without causing the amount of selected microservices to perform work.

11 . The method of claim 1 , wherein the monitoring of the replica count of the initial microservice is performed by a dedicated operator microservice running within the serverless microservice environment.

12 . A system for reducing start latency of serverless microservices, the system comprising:

a memory; and

a processor in communication with the memory, the processor being configured to perform operations comprising:

detecting an incoming request or event to an application in a serverless microservice environment, wherein the incoming request or event initiates simultaneous invocations of one or more microservices of the application;

monitoring a replica count of an initial microservice, wherein the initial microservice controls the incoming request or event;

selecting an amount of selected microservices from the one or more microservices of the application by detecting a change in the replica count from zero, wherein the amount of selected microservices perform a task of the incoming request or event, and wherein the task applies one or more predefined application-specific rules to one or more elements of the incoming request or event to determine the amount of selected microservices;

triggering scaling up activation of the selected microservices in response to determining that the selected microservices are needed for the incoming request, wherein said triggering reduces a cold-start time of the application; and

invoking the selected microservices of the application to match the amount of selected microservices.

13 . The system of claim 12 , wherein selecting the amount of selected microservices of the application further includes:

classifying the incoming request or event by applying predefined a set of application-specific rules to elements of the incoming request or event; and

mapping a classification to a subset of the one or more microservices of the application microservices of the application.

14 . The system of claim 13 , the operations further comprise:

querying a microservice orchestrator application programming interface to obtain one or more replica counts for the amount of selected microservices.

15 . The system of claim 12 , wherein detecting an incoming event further includes:

observing incoming event queues using a scaling service, wherein the scaling service selects the amount of selected microservices.

16 . The system of claim 15 , wherein the scaling service further includes:

sending a wake-up event to the incoming event queues to scale up the amount of selected microservices, wherein the wake-up event is consumed by the microservice to remove the wake-up event from the incoming event queues.

17 . The system of claim 12 , wherein the operations further comprise:

performing the selecting of the amount of selected microservices using an activation system implemented as an embedded plugin within an ingress controller.

18 . The system of claim 12 , wherein the operations further comprise:

sending a request to a defined endpoint, provided by the one or more selected microservices, to activate the scaling up activation without causing the amount of selected microservices to perform work.

19 . The system of claim 12 , wherein the operations further comprise:

forwarding and calling one or more proxy components.

20 . A computer program product for reducing start latency of serverless microservices, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processors to perform a function, the function comprising:

detecting an incoming HTTP request to an application in a serverless microservice environment, wherein the incoming HTTP request initiates simultaneous invocations of one or more microservices of the application;

monitoring a replica count of an initial microservice, wherein the initial microservice controls the incoming HTTP request;

selecting an amount of selected microservices from the one or more microservices of the application by detecting a change in the replica count from zero, wherein the amount of selected microservices perform a task of the incoming HTTP request, and wherein the task applies one or more predefined application-specific rules to one or more elements of the incoming HTTP request to determine the amount of selected microservices, and wherein the one or more elements of the incoming HTTP request comprise at least one of an HTTP query parameter or a custom HTTP header;

triggering scaling up activation of the one or more microservices of the application; and

invoking the one or more microservices of the application to match the amount of selected microservices.