Reducing the startup latency of functions in a FaaS infrastructure
Techniques for reducing the startup latency of functions in a Functions-as-a-Service (FaaS) infrastructure are provided. In one set of embodiments, a function manager of the FaaS infrastructure can receive a request to invoke a function uploaded to the infrastructure and can retrieve information associated with the function. The retrieved information can include an indicator of whether instances of the function may be sticky (i.e., kept in host system primary memory after function execution is complete), and a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories. If the indicator indicates that instances of the function may be sticky and if the list identifies at least one host system with an unused sticky instance of the function in its primary memory, the function manager can select the at least one host system for executing the function.
1. A method for reducing the startup latency of functions in a Functions-as-a-Service (FaaS) infrastructure, the method comprising:
receiving by a function manager of the FaaS infrastructure, a request to invoke a function uploaded to the FaaS infrastructure;
retrieving, by the function manager, information associated with the function, the information including:
an indicator of whether instances of the function may be sticky, wherein sticky function instances are kept in host system primary memory after function execution is complete; and
a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories; and
if the indicator indicates that instances of the function may be sticky and if the list identifies at least one host system with an unused sticky instance of the function in its primary memory, selecting, by the function manager, the at least one host system for executing the function.
2. The method of claim 1 wherein if the indicator indicates that instances of the function may be sticky and the list does not identify any host systems with unused sticky instances, the method further comprises:
selecting a host system in the FaaS infrastructure for executing the function; and
if a number of current sticky instances of the function in the FaaS infrastructure is less than a maximum allowed number of sticky instances of the function, instructing the selected host system to maintain a sticky instance of the function in the host system's primary memory after function completion.
3. The method of claim 2 wherein the maximum number of allowed sticky instances of the function is configurable by a developer of the function.
4. The method of claim 1 further comprising:
identifying a function chain that the function is a part of, the function chain corresponding to a sequence of function invocations that is commonly used by a calling entity of the function.
5. The method of claim 4 wherein identifying the function chain comprises, at a time the function is uploaded to the FaaS infrastructure:
parsing program code of the function to identify invocations of one or more other functions; and
building the function chain based on the identified invocations.
6. The method of claim 4 wherein identifying the function chain comprises:
observing function calling patterns that include the function; and
deriving the function chain from the observed function calling patterns.
7. The method of claim 4 further comprising, at the time of receiving the request to invoke the function:
determining that the function is part of the function chain;
identifying a next function in the function that is invoked after the function; and
causing an image of the next function to be pre-loaded on a host system of the FaaS infrastructure.
8. A non-transitory computer readable storage medium having stored thereon program code executable by a computer system in a Functions-as-a-Service (FaaS) infrastructure, the program code embodying a method for reducing the startup latency of functions in the FaaS infrastructure, the method comprising:
receiving a request to invoke a function uploaded to the FaaS infrastructure;
retrieving information associated with the function, the information including:
an indicator of whether instances of the function may be sticky, wherein sticky function instances are kept in host system primary memory of function execution is complete; and
a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories; and
if the indicator indicates that instances of the function may be sticky and if the list identities at least one host system with an unused sticky instance of the function in its primary memory, selecting the at least one host system for executing the function.
9. The non-transitory computer readable storage medium of claim 8 wherein if the indicator indicates that instances of the function may be sticky and the list does not identify any host systems with unused sticky instances, the method further comprises:
selecting a host system in the FaaS infrastructure for executing the function; and
if a number of current sticky instances of the function in the FaaS infrastructure is less than a maximum allowed number of sticky instances of the function, instructing the selected host system to maintain a sticky instance of the function in the host system's primary memory after function completion.
10. The non-transitory computer readable storage medium of claim 8 wherein the maximum number of allowed sticky instances of the function is configurable by a developer of the function.
11. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:
identifying a function chain that the function is, a part of, the function chain corresponding to a sequence of function invocations that is commonly used by a calling entity of the function.
12. The non-transitory computer readable storage medium of claim 11 wherein identifying the function chain comprises, at a time the function is uploaded to the FaaS infrastructure:
parsing program code of the function to identify invocations of one or more other functions; and
building the function chain based on the identified invocations.
13. The non-transitory computer readable storage medium claim 11 wherein identifying the function chain comprises:
observing function calling patterns that include the function; and
deriving the function chain from the observed function calling patterns.
14. The non-transitory computer readable storage medium of claim 11 wherein the method further comprises, at the tame of receiving the request to invoke the function:
determining that the function is part of the function chain;
identifying a next function in the function that is invoked after the function; and
causing an image of the next function to be pre-loaded on a host system of the FaaS infrastructure.
15. A computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed by the processor, causes the processor to:
receive a request to invoke a function uploaded to a Functions-as-a-Service (FaaS) infrastructure;
retrieve information associated with the function, the formation including:
an indicator of whether instances of the function may be sticky, wherein sticky function instances are kept in host system primary memory after function execution is complete; and
a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories; and
if the indicator indicates that instances of the function may be sticky and if the list identifies at least one host system with an unused sticky instance of the function in its primary memory, select the at least one host system for executing the function.
16. The computer system of claim 15 wherein if the indicator indicates that instances of the function may be sticky and the list does not identify any host systems with unused sticky instances, the program code further causes the processor to:
select a host system in the FaaS infrastructure for executing the function; and
if a number of current sticky instances of the function in the FaaS infrastructure is less than a maximum allowed number of sticky instances of the function, instruct the selected host system to maintain a sticky instance of the function in the host system's primary memory after function completion.
17. The computer system of claim 15 wherein the maximum number of allowed sticky instances of the function is configurable by a developer of the function.
18. The computer system of claim 15 wherein the program code further causes the processor to:
identify a function chain that the function is a part of, the function chain corresponding to a sequence of function invocations that is commonly used by a calling entity of the function.
19. The computer system of claim 18 wherein identifying the function chain comprises, at a time the function is uploaded to the FaaS infrastructure:
parsing program code of the function to identify invocations of one or more other functions; and
building the function chain based on the identified invocations.
20. The computer system of claim 18 wherein identifying the function chain comprises:
observing function calling patterns that include the function; and
deriving the function chain from the observed function calling patterns.
21. The computer system of claim 18 wherein the program code further causes the processor to, at the time of receiving the request to invoke the function:
determine that the function is part of the function chain;
identify a next function in the function that is invoked after the function; and
cause an image of the next function to be pre-loaded on a host system of the FaaS infrastructure.