IP Library › Granted Patent US 12670000
Granted Patent B2
US 12670000 · App. 18/680,156 · Granted Jun 30, 2026

Data center resource orchestration using serverless application programming interfaces

Inventors: Bojan Vukojevic (Pleasanton, CA); Cameron Moore (San Jose, CA); Nagesh Prafulla Bhole (Pune, IN); John T. Richards (Wheeling, WV); Frank James Spitulski (Gilroy, CA); Sanjay Saxena (San Jose, CA); Anand Parthasarathi (Apex, NC)
Assignee: NVIDIA Corporation
G06F9/44505G06F9/38G06T1/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12670000
App. No.
18/680,156
Granted
Jun 30, 2026
Kind
B2
Abstract

Disclosed are systems and techniques for a cloud function worker for executing code using graphics processing units (GPUs) in a serverless architecture. The techniques include receiving, at a cloud function worker, a first cloud function execution request from a cloud function queue of a cloud function controller, executing a first instance of a code based on the first cloud function execution request using at least a graphics processing unit (GPU) of a cluster environment hosting the cloud function worker, determining first metrics of the GPU of the cluster environment, responsive to the first metrics of the GPU satisfying a bandwidth criterion, requesting a second cloud function execution request from the cloud function queue of the cloud function controller, and executing, concurrent with execution of the first instance of the code, a second instance of the code based on the second cloud function execution request.

Claims (55)

1 . A method comprising:

receiving, using a worker, a first execution request from a queue managed by a controller;

executing a first instance of a particular code based on the first execution request using at least a graphics processing unit (GPU) of a cluster environment hosting the worker;

during execution of the first instance of the particular code, determining, using the worker, first metrics of the GPU of the cluster environment;

responsive to the first metrics of the GPU satisfying a bandwidth criterion, requesting, using the worker, a second execution request from the queue managed by the controller; and

executing, at least partially concurrently with execution of the first instance of the particular code, a second instance of the particular code based on the second execution request using at least the GPU of the cluster environment.

2 . The method of claim 1 , further comprising:

providing a first execution result corresponding to executing the first instance of the particular code to the controller; and

providing a second execution result corresponding to executing the second instance of the particular code to the controller.

3 . The method of claim 2 , wherein providing the first execution result corresponding to executing the first instance of the particular code to the controller comprises:

causing the first execution result to be transmitted to a storage device; and

causing a storage identifier associated with the first execution result to be transmitted to the controller.

4 . The method of claim 1 , wherein the first execution request comprises input data, and wherein executing the first instance of the particular code based on the first execution request comprises providing the input data of the first execution request to the first instance of the particular code.

5 . The method of claim 1 , wherein the first execution request comprises an input data identifier, the method further comprising:

accessing an input data associated with the input data identifier from a storage device, wherein executing the first instance of the particular code based on the first execution request comprises providing the input data to the first instance of the particular code.

6 . The method of claim 1 , further comprising causing periodic heartbeat requests to be transmitted to the controller.

7 . The method of claim 1 , further comprising:

generating a progress indicator artifact corresponding to a progress of executing the first instance of the particular code; and

causing the progress indicator artifact to be transmitted to the controller.

8 . A system comprising:

one or more processing devices to perform operations comprising:

receiving, using a worker, a first execution request from a queue managed by a controller;

executing a first instance of a particular code based on the first execution request using at least a graphics processing unit (GPU) of a cluster environment hosting the worker;

during execution of the first instance of the particular code, determining, using the worker, first metrics of the GPU of the cluster environment;

responsive to the first metrics of the GPU satisfying a bandwidth criterion, requesting, using the worker, a second execution request from the queue managed by the controller; and

executing, at least partially concurrently with execution of the first instance of the particular code, a second instance of the particular code based on the second execution request using at least the GPU of the cluster environment.

9 . The system of claim 8 , the operations further comprising:

providing a first execution result corresponding to executing the first instance of the particular code to the controller; and

providing a second execution result corresponding to executing the second instance of the particular code to the controller.

10 . The system of claim 9 , wherein providing the first execution result corresponding to executing the first instance of the particular code to the controller comprises:

causing the first execution result to be transmitted to a storage device; and

causing a storage identifier associated with the first execution result to be transmitted to the controller.

11 . The system of claim 8 , wherein the first execution request comprises input data, and wherein executing the first instance of the particular code based on the first execution request comprises providing the input data of the first execution request to the first instance of the particular code.

12 . The system of claim 8 , wherein the first execution request comprises an input data identifier, the operations further comprising:

accessing an input data associated with the input data identifier from a storage device, wherein executing the first instance of the particular code based on the first execution request comprises providing the input data to the first instance of the particular code.

13 . The system of claim 8 , the operations further comprising causing periodic heartbeat requests to be transmitted to the controller.

14 . The system of claim 8 , the operations further comprising:

generating a progress indicator artifact corresponding to a progress of executing the first instance of the particular code; and

causing the progress indicator artifact to be transmitted to the controller.

15 . A processor comprising one or more processing units to:

receive, using a worker, a first execution request from a queue managed by a controller;

execute a first instance of a particular code based on the first execution request using at least a graphics processing unit (GPU) of a cluster environment hosting the worker;

during execution of the first instance of the particular code, determine, using the worker, first metrics of the GPU of the cluster environment;

responsive to the first metrics of the GPU satisfying a bandwidth criterion, request, using the worker, a second execution request from the queue managed by the controller; and

execute, at least partially concurrently with execution of the first instance of the particular code, a second instance of the particular code based on the second execution request using at least the GPU of the cluster environment.

16 . The processor of claim 15 , wherein the one or more processing units are further to:

provide a first execution result corresponding to executing the first instance of the particular code to the controller; and

provide a second execution result corresponding to executing the second instance of the particular code to the controller.

17 . The processor of claim 16 , wherein to provide the first execution result corresponding to executing the first instance of the particular code to the controller, the one or more processing units are to:

cause the first execution result to be transmitted to a storage device; and

cause a storage identifier associated with the first execution result to be transmitted to the controller.

18 . The processor of claim 15 , wherein the first execution request comprises input data, and wherein to execute the first instance of the particular code based on the first execution request, the one or more processing units are to provide the input data of the first execution request to the first instance of the particular code.

19 . The processor of claim 15 , wherein the first execution request comprises an input data identifier, and wherein the one or more processing units are further to:

access an input data associated with the input data identifier from a storage device, wherein to execute the first instance of the particular code based on the first execution request, the one or more processing units are to provide the input data to the first instance of the particular code.

20 . The processor of claim 15 , wherein the one or more processing units are further to cause periodic heartbeat requests to be transmitted to the controller.