Technologies for batching requests in an edge infrastructure
Technologies for batching requests in an edge infrastructure include a compute device including circuitry configured to obtain a request for an operation to be performed at an edge location. The circuitry is also configured to determine, as a function of a parameter of the obtained request, a batch that the obtained request is to be assigned to. The batch includes a one or more requests for operations to be performed at an edge location. The circuitry is also configured to assign the batch to a cloudlet at an edge location. The cloudlet includes a set of resources usable to execute the operations requested in the batch.
1. A compute device comprising:
at least one memory; and
processor circuitry to:
obtain a request for an operation to be performed at an edge location;
generate, as a function of a parameter of the obtained request, a first batch that the obtained request is to be assigned to, wherein the first batch includes a one or more requests for operations to be performed at an edge location;
determine whether to dispatch the first batch, the determination based on whether a number of requests in the first batch satisfies a reference number of requests that can be processed in a burst by an amount of resources included in a cloudlet at an edge location, the cloudlet to include other resources reserved for requests that are not in the first batch;
in response to a determination that the first batch is to be dispatched, assign the first batch to the cloudlet; and
in response to a determination that the first batch is to not be dispatched, generate, as a function of a parameter of the obtained request, a second batch to which the obtained request is to be assigned.
2. The compute device of claim 1 , wherein to determine whether to dispatch the first batch, the processor circuitry is further to determine whether a number of requests in the first batch satisfies a reference number of requests.
3. The compute device of claim 1 , wherein to determine whether to dispatch the first batch, the processor circuitry is further to determine whether an amount of time in which requests have been in the batch satisfies a reference amount of time.
4. The compute device of claim 1 , wherein to determine whether to dispatch the first batch, the processor circuitry is further to determine whether to dispatch the batch based on telemetry data obtained from the cloudlet.
5. The compute device of claim 4 , wherein the circuitry is further to identify, as a function of the telemetry data, a pattern of resource utilization of the cloudlet and wherein to determine whether to dispatch the first batch, the processor circuitry is further to determine to dispatch the batch to the cloudlet in response to a determination that the cloudlet is predicted, as a function of the identified pattern, to have underutilized resources.
6. The compute device of claim 1 , wherein to determine, as a function of a parameter of the obtained request, a first batch that the obtained requests is to be assigned to, the processor circuitry is further to determine the batch as a function of a latency target or a bandwidth target associated with the obtained request.
7. The compute device of claim 1 , wherein to determine, as a function of a parameter of the obtained request, a first batch that the obtained request is to be assigned to, the processor circuitry is further to determine the batch as a function of one or more resources to be utilized to perform the operation.
8. The compute device of claim 1 , wherein the cloudlet is one of a plurality of cloudlets, and wherein to determine, as a function of a parameter of the obtained request, a first batch that the obtained request is to be assigned to, the processor circuitry is further to determine the batch as a function of telemetry data obtained from the plurality of cloudlets.
9. The compute device of claim 1 , wherein the processor circuitry is further to:
receive batched responses produced by the cloudlet in response to performing the operations in the first batch; and
utilize metadata in the batched responses to identify a corresponding request to which a batched response pertains.
10. The compute device of claim 9 , wherein the circuitry is further to send each batched response to a corresponding requestor device.
11. A method comprising:
obtaining, by a compute device, a request for an operation to be performed at an edge location;
generating, by the compute device and as a function of a parameter of the obtained request, a first batch that the obtained request is to be assigned to, wherein the first batch includes a one or more requests for operations to be performed at an edge location;
determining whether to dispatch the first batch, the determination to include whether a number of requests in the first batch satisfies a reference number of requests that can be processed in a burst by an amount of resources included in a cloudlet at an edge location, the cloudlet to include other resources reserved for requests that are not in the first batch;
in response to a determination that the first batch is to be dispatched, assigning, by the compute device, the first batch to the cloudlet; and
in response to a determination that the first batch is not to be dispatched, generating, by the compute device and as a function of a parameter of the obtained request, a second batch to which the obtained request is to be assigned.
12. The method of claim 11 , wherein determining whether to dispatch the first batch includes determining whether a number of requests in the first batch satisfies a reference number of requests.
13. The method of claim 11 , wherein the determining of whether to dispatch the first batch includes determining whether an amount of time in which requests have been in the batch satisfies a reference amount of time.
14. The method of claim 11 , wherein the determining of whether to dispatch the first batch includes determining whether to dispatch the batch on telemetry data obtained from the cloudlet.
15. The method of claim 11 , wherein the parameter is an object.
16. One or more non-transitory machine-readable storage media comprising a plurality of instructions stored thereon that, in response to being executed, cause a device to:
obtain a request for an operation to be performed at a location;
generate, as a function of a parameter of the obtained request, a first batch that the obtained request is to be assigned to, wherein the first batch includes a one or more requests for operations to be performed at an edge location;
determine whether to dispatch the first batch, the determination to include whether a number of requests in the first batch satisfies a reference number of requests that can be processed in a burst by an amount of resources included in a cloudlet at an edge location, the cloudlet to include other resources reserved for requests that are not in the first batch;
in response to a determination that the first batch is to be dispatched, assign the first batch to the cloudlet; and
in response to a determination that the first batch is not to be dispatched, generate, as a function of a parameter of the obtained request, a second batch that the obtained request is to be assigned to.
17. The one or more non-transitory machine-readable storage media of claim 16 , wherein to determine whether to dispatch the first batch, the plurality of instructions, when executed, cause the device to determine whether a number of requests in the first batch satisfies a reference number of requests.