Reducing the startup latency of functions in a FaaS infrastructure
Techniques for reducing the startup latency of functions in a Functions-as-a-Service (FaaS) infrastructure are provided. In one set of embodiments, a function manager of the FaaS infrastructure can receive a request to invoke a function uploaded to the infrastructure and can retrieve information associated with the function. The retrieved information can include an indicator of whether instances of the function may be sticky (i.e., kept in host system primary memory after function execution is complete), and a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories. If the indicator indicates that instances of the function may be sticky and if the list identifies at least one host system with an unused sticky instance of the function in its primary memory, the function manager can select the at least one host system for executing the function.
1. A method for reducing the startup latency of functions in a Functions-as-a-Service (FaaS) infrastructure, the method comprising:
receiving, by a computer system, a request to invoke a first function uploaded to the FaaS infrastructure, the request originating from a calling entity;
selecting, by the computer system, a first host system in the FaaS infrastructure for executing the first function;
sending, by the computer system to the first host system, a first command to load a software image of the first function into a primary memory of the first host system;
determining, by the computer system, that the first function is part of a function chain associated with the calling entity, the function chain defining an ordered sequence of functions including the first function and a second function following the first function;
selecting the first function, by a logic component, to be retained at the primary memory after execution;
recording one or more function invocation sequences that include the first function;
identifying a longest common pattern in the one or more function invocation sequences;
constructing the function chain based on the identified longest common pattern;
selecting, by the computer system, a second host system in the FaaS infrastructure for executing the second function; and
sending, by the computer system to the second host system before the first function has completed execution on the first host system, a second command to load a software image of the second function into a primary memory of the second host system.
2. The method of claim 1 wherein the computer system identifies the function chain by accessing a configuration file that includes a list of function chains associated with the calling entity.
3. The method of claim 1 wherein the computer system constructs the function chain by:
receiving, at a time the first function is uploaded to the FaaS infrastructure, program code for the first function;
identifying, in the program code, invocations of one or more other functions including the second function; and
constructing the function chain based on the identified invocations.
4. The method of claim 1 wherein the calling entity is a particular tenant of the FaaS infrastructure.
5. The method of claim 1 wherein prior to selecting the second host system, the computer system determines whether instances of the second function may be sticky and whether a third host system in the FaaS infrastructure exists that has an unused sticky instance of the second function in its primary memory.
6. The method of claim 5 wherein if the computer system determines that the instances of the second function may be sticky and the third host system exists, the computer system sends a third command to the third host system to execute the second function, without sending the second command to the second host system.
7. A non-transitory computer readable storage medium having stored thereon program code executable by a computer system in a Functions-as-a-Service (FaaS) infrastructure, the program code embodying a method for reducing the startup latency of functions in the FaaS infrastructure, the method comprising:
receiving a request to invoke a first function uploaded to the FaaS infrastructure, the request originating from a calling entity;
selecting a first host system in the FaaS infrastructure for executing the first function;
sending, to the first host system, a first command to load a software image of the first function into a primary memory of the first host system;
determining that the first function is part of a function chain associated with the calling entity, the function chain defining an ordered sequence of functions including the first function and a second function following the first function;
selecting the first function, by a logic component, to be retained at the primary memory after execution;
recording one or more function invocation sequences that include the first function;
identifying a longest common pattern in the one or more function invocation sequences;
constructing the function chain based on the identified longest common pattern;
selecting a second host system in the FaaS infrastructure for executing the second function; and
sending, to the second host system before the first function has completed execution on the first host system, a second command to load a software image of the second function into a primary memory of the second host system.
8. The non-transitory computer readable storage medium of claim 7 wherein the computer system identifies the function chain by accessing a configuration file that includes a list of function chains associated with the calling entity.
9. The non-transitory computer readable storage medium of claim 7 wherein the computer system constructs the function chain by:
receiving, at a time the first function is uploaded to the FaaS infrastructure, program code for the first function;
identifying, in the program code, invocations of one or more other functions including the second function; and
constructing the function chain based on the identified invocations.
10. The non-transitory computer readable storage medium of claim 7 wherein the calling entity is a particular tenant of the FaaS infrastructure.
11. The non-transitory computer readable storage medium of claim 7 wherein prior to selecting the second host system, the computer system determines whether instances of the second function may be sticky and whether a third host system in the FaaS infrastructure exists that has an unused sticky instance of the second function in its primary memory.
12. The non-transitory computer readable storage medium of claim 11 wherein if the computer system determines that the instances of the second function may be sticky and the third host system exists, the computer system sends a third command to the third host system to execute the second function, without sending the second command to the second host system.
13. A computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed by the processor, causes the processor to:
receive a request to invoke a first function uploaded to a Functions-as-a-Service (FaaS) infrastructure, the request originating from a calling entity;
select a first host system in the FaaS infrastructure for executing the first function;
send, to the first host system, a first command to load a software image of the first function into a primary memory of the first host system;
determine that the first function is part of a function chain associated with the calling entity, the function chain defining an ordered sequence of functions including the first function and a second function following the first function;
select the first function, by a logic component, to be retained at the primary memory after execution;
record one or more function invocation sequences that include the first function;
identify a longest common pattern in the one or more function invocation sequences;
construct the function chain based on the identified longest common pattern;
select a second host system in the FaaS infrastructure for executing the second function; and
send, to the second host system before the first function has completed execution on the first host system, a second command to load a software image of the second function into a primary memory of the second host system.
14. The computer system of claim 13 wherein the processor identifies the function chain by accessing a configuration file that includes a list of function chains associated with the calling entity.
15. The computer system of claim 13 wherein the processor constructs the function chain by:
receiving, at a time the first function is uploaded to the FaaS infrastructure, program code for the first function;
identifying, in the program code, invocations of one or more other functions including the second function; and
constructing the function chain based on the identified invocations.
16. The computer system of claim 13 wherein the calling entity is a particular tenant of the FaaS infrastructure.
17. The computer system of claim 13 wherein prior to selecting the second host system, the processor determines whether instances of the second function may be sticky and whether a third host system in the FaaS infrastructure exists that has an unused sticky instance of the second function in its primary memory.
18. The computer system of claim 17 wherein if the processor determines that the instances of the second function may be sticky and the third host system exists, the processor sends a third command to the third host system to execute the second function, without sending the second command to the second host system.