Dynamically routing code for executing
Code may be dynamically routed to computing resources for execution. Code may be received for execution on behalf of a client. Execution criteria for the code may be determined and computing resources that satisfy the execution criteria may be identified. The identified computing resources may then be procured for executing the code and then the code may be routed to the procured computing resources for execution. Permissions or authorization to execute the code may be shared to ensure that computing resources executing the code have the same permissions or authorization when executing the code.
1. A system, comprising:
one or more processors; and
a memory, that stores program instructions that, when executed by the at least one processor, cause the one or more processors to:
receive, via an interface of a cloud computing environment, code to configure execution of an Extract Transform Load (ETL) job to transfer data from a data source to a location on behalf of a client;
evaluate the code to determine a criteria to execute the ETL job, wherein the evaluating comprises:
identify use of one or more Graphics Processing Unit (GPUs) that are utilized via one or more application programming interfaces (APIs) as part of executing the ETL job and identifying that a driver is used to interact with the one or more GPUs in order to execute the ETL job;
procure one or more computing resources with respective capabilities to execute the ETL job that satisfy the determined criteria, wherein the determined criteria includes the use of the one or more GPUs, wherein the procuring causes configuration of a software image for a software container that implements operating system virtualization at the one or more computing resources to use the driver to interact with the one or more GPUs via the one or more APIs to execute the code; and
initiate execution of the ETL job at the procured computing resources.
2. The system of claim 1 , wherein the memory stores further program instructions that when executed by the at least one processor further cause the at least one processor to:
determine that execution of the ETL job has failed; and
provide a failure notification for the ETL job to the client.
3. The system of claim 1 , wherein an additional criteria is determined to execute the ETL job that includes a network configuration.
4. The system of claim 1 , wherein the memory stores further program instructions that when executed by the at least one processor further cause the at least one processor to:
receive a request to cancel execution of the ETL job; and
cancel execution of the ETL job at the at the procured computing resources.
5. The system of claim 1 , wherein the identified computing resources are a cluster of computing resources, wherein at least one of the computing resources is a multi-tenant computing resource available for executing different source code, wherein procuring the identified computing resources to execute the source code comprises configuring a logically isolated network amongst the cluster of computing resources.
6. The system of claim 1 ,
wherein the memory stores further program instructions that when executed by the at least one processor further cause the at least one processor to receive an identity for the client associated with one or more operational permissions; and
wherein to initiate execution of the ETL job at the procured computing resources, the program instructions cause the at least one processor to provide the identity for the client to the procured computing resources.
7. The system of claim 1 , wherein the cloud computing environment is a network-based service and wherein the code for the ETL job is received at a network endpoint via a network-based interface for the network-based service.
8. A method, comprising:
receiving, via an interface of a cloud computing environment, code to configure execution of an Extract Transform Load (ETL) job to transfer data from a data source to a destination on behalf of a client;
evaluating the code to determine a criteria to execute the ETL job, wherein the evaluating comprises:
identifying use of one or more Graphics Processing Unit (GPUs) that are utilized via one or more application programming interfaces (APIs) as part of executing the ETL job and identifying that a driver is used to interact with the one or more GPUs in order to execute the ETL job;
procuring one or more computing resources with respective capabilities to execute the ETL job that satisfy the determined criteria, wherein the determined criteria includes the use of the one or more GPUs, wherein the procuring causes configuration of a software image for a software container that implements operating system virtualization at the one or more computing resources to use the driver to interact with the one or more GPUs via the one or more APIs to execute the code; and
initiating execution of the ETL job at the procured computing resources.
9. The method of claim 8 , further comprising:
determining that execution of the ETL job has failed; and
providing a failure notification for the ETL job to the client.
10. The method of claim 8 , further comprising:
receiving a request to cancel execution of the ETL job; and
canceling execution of the ETL job at the at the procured computing resources.
11. The method of claim 8 , wherein the identified computing resources are a cluster of computing resources, wherein at least one of the computing resources is a multi-tenant computing resource available for executing different source code, wherein procuring the identified computing resources to execute the source code comprises configuring a logically isolated network amongst the cluster of computing resources.
12. The method of claim 8 ,
wherein the method further comprises receiving an identity for the client associated with one or more operational permissions; and
wherein initiating execution of the ETL job at the procured computing resources comprises providing the identity for the client to the procured computing resources.
13. The method of claim 8 , wherein an additional criteria is determined to execute the ETL job that includes a network configuration.
14. The method of claim 8 , wherein the cloud computing environment is a network-based service and wherein the code for the ETL job is received at a network endpoint via a network-based interface for the network-based service.
15. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to implement:
receiving, via an interface of a cloud computing environment, code to configure execution of an Extract Transform Load (ETL) job to transfer data from a data source to a destination on behalf of a client;
evaluating the code to determine a criteria to execute the ETL job, wherein the evaluating comprises:
identifying use of one or more Graphics Processing Unit (GPUs) that are utilized via one or more application programming interfaces (APIs) as part of executing the ETL job and identifying that a driver is used to interact with the one or more GPUs in order to execute the ETL job;
procuring one or more computing resources with respective capabilities to execute the ETL job that satisfy the determined criteria, wherein the determined criteria includes the use of the one or more GPUs, wherein the procuring causes configuration of a software image for a software container that implements operating system virtualization at the one or more computing resources to use the driver to interact with the one or more GPUs via the one or more APIs to execute the code; and
initiating execution of the ETL job at the procured computing resources.
16. The one or more non-transitory computer-readable storage media of claim 15 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement: determining that execution of the ETL job has failed; and providing a failure notification for the ETL job to the client.
17. The one or more non-transitory computer-readable storage media of claim 16 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement:
receiving a request to cancel execution of the ETL job; and
canceling execution of the ETL job at the at the procured computing resources.
18. The one or more non-transitory computer-readable storage media of claim 16 , wherein the identified computing resources are a cluster of computing resources, wherein at least one of the computing resources is a multi-tenant computing resource available for executing different source code, wherein procuring the identified computing resources to execute the source code comprises configuring a logically isolated network amongst the cluster of computing resources.
19. The one or more non-transitory computer-readable storage media of claim 16 ,
wherein the one or more non-transitory computer-readable storage media store further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement receiving an identity for the client associated with one or more operational permissions; and
wherein, in initiating execution of the ETL job at the procured computing resources, the program instructions cause the one or more computing devices to implement providing the identity for the client to the procured computing resources.
20. The one or more non-transitory computer-readable storage media of claim 16 , wherein the cloud computing environment is a network-based service and wherein the code for the ETL job is received at a network endpoint via a network-based interface for the network-based service.