Job allocations to fractions of parallel processing units (PPUs)
In some examples, a controller receives a request to schedule a first job in a system including a plurality of physical parallel processing units (PPUs), where a physical PPU of the plurality of physical PPUs includes multiple PPU fractions. The controller allocates the first job to a first collection of PPU fractions of the multiple PPU fractions based on an operational cost reduction objective to reduce a cost associated with a usage of the plurality of physical PPUs. The controller triggers processing of the first job according to the allocation of the first job to the first collection of PPU fractions, where data isolation is provided between the first job allocated to the first collection of PPU fractions and a second job allocated to a second collection of PPU fractions of the multiple PPU fractions.
1 . A non-transitory machine-readable storage medium comprising instructions, which upon execution, cause a controller to:
receive a request to schedule a first job in a system comprising a plurality of physical parallel processing units (PPUs), wherein a physical PPU of the plurality of physical PPUs comprises multiple PPU fractions, and wherein a first PPU fraction of the multiple PPU fractions comprises a first PPU compute resource and a first PPU memory resource that is separate and isolated from a second PPU compute resource and a second PPU memory resource of a second PPU fraction of the multiple PPU fractions;
allocate the first job to a first collection of PPU fractions of the multiple PPU fractions based on an operational cost reduction objective to reduce a cost associated with a usage of the plurality of physical PPUs; and
trigger processing of the first job according to the allocation of the first job to the first collection of PPU fractions, wherein data isolation is provided between the first job allocated to the first collection of PPU fractions and a second job allocated to a second collection of PPU fractions of the multiple PPU fractions.
2 . The non-transitory machine-readable storage medium of claim 1 , wherein a compute capacity of the first PPU fraction of the multiple PPU fractions is different from a compute capacity of the second PPU fraction of the multiple PPU fractions.
3 . The non-transitory machine-readable storage medium of claim 1 , wherein a memory capacity of the first PPU fraction of the multiple PPU fractions is different from a memory capacity of the second PPU fraction of the multiple PPU fractions.
4 . The non-transitory machine-readable storage medium of claim 1 , wherein the data isolation is based on isolation of PPU compute resources and PPU memory resources between the first collection of PPU fractions and the second collection of PPU fractions.
5 . The non-transitory machine-readable storage medium of claim 1 , wherein the controller is accessible by a plurality of tenants to use the plurality of physical PPUs, wherein the first job is requested by a first tenant, and the second job is requested by a second tenant different from the first tenant, and wherein tenant isolation is provided by allocating the first job to the first collection of PPU fractions of the physical PPU, and allocating the second job to the second collection of PPU fractions of the physical PPU.
6 . The non-transitory machine-readable storage medium of claim 5 , wherein the allocating of the first job to the first collection of PPU fractions is further based on a tenant isolation constraint to provide tenant isolation wherein a single tenant of the plurality of tenants is to use a PPU fraction of the physical PPU at a time.
7 . The non-transitory machine-readable storage medium of claim 6 , wherein the tenant isolation constraint comprises a tenant-job variable to indicate whether a respective PPU fraction of a physical PPU of the plurality of physical PPUs has been allocated to a respective tenant of the plurality of tenants.
8 . The non-transitory machine-readable storage medium of claim 7 , wherein the tenant-job variable is based on variables indicating whether corresponding jobs of the respective tenant have been allocated to the respective PPU fraction.
9 . The non-transitory machine-readable storage medium of claim 8 , wherein the tenant-job variable is based on a sum of the variables indicating whether corresponding jobs of the respective tenant have been allocated to the respective PPU fraction.
10 . The non-transitory machine-readable storage medium of claim 9 , wherein the tenant-job variable is set to a specified value if any job of the respective tenant is assigned to the respective PPU fraction.
11 . The non-transitory machine-readable storage medium of claim 1 , wherein the allocating of the first job to the first collection of PPU fractions is further based on a migration cost reduction objective to reduce a cost associated with migrating jobs between physical PPUs.
12 . The non-transitory machine-readable storage medium of claim 1 , wherein the allocating of the first job to the first collection of PPU fractions is further based on a constraint to ensure that cumulative resources allocated to one or more jobs within a given PPU fraction does not exceed a total capacity of the given PPU fraction.
13 . The non-transitory machine-readable storage medium of claim 1 , wherein the allocating of the first job to the first collection of PPU fractions is further based on a constraint to ensure cumulative resources allocated to one or more jobs across the multiple PPU fractions of the physical PPU does not exceed a total capacity of the physical PPU.
14 . The non-transitory machine-readable storage medium of claim 1 , wherein the controller to execute the instructions is part of an adapter that is separate from a central processing unit (CPU) of a computing system including the plurality of PPUs.
15 . The non-transitory machine-readable storage medium of claim 14 , wherein the adapter is to:
transfer first job data of the first job using a direct memory access (DMA) transfer from the adapter to a memory of the first collection of PPU fractions, and
transfer second job data of the second job using a DMA transfer from the adapter to a memory of the second collection of PPU fractions.
16 . The non-transitory machine-readable storage medium of claim 15 , wherein the adapter is to receive the first job data and the second job data from clients in remote DMA (RDMA) transfers over a network.
17 . An adapter for a system comprising a plurality of physical processing units (PPUs), the adapter comprising:
a network interface to communicate over a network; and
an adapter controller to:
receive, over the network, a request from a first tenant to schedule a first job in the system, wherein a physical PPU of the plurality of physical PPUs comprises multiple PPU fractions, and wherein a first PPU fraction of the multiple PPU fractions comprises a first PPU compute resource and a first PPU memory resource that is separate and isolated from a second PPU compute resource and a second PPU memory resource of a second PPU fraction of the multiple PPU fractions;
allocate the first job to a first collection of PPU fractions of the multiple PPU fractions based on:
an operational cost reduction objective to reduce a cost associated with a usage of the plurality of physical PPUs, and
a tenant isolation constraint to provide tenant isolation wherein a single tenant of a plurality of tenants including the first tenant is to use a PPU fraction of the physical PPU at a time; and
trigger processing of the first job according to the allocation of the first job to the first collection of PPU fractions, wherein data isolation is provided between the first job allocated to the first collection of PPU fractions and a second job of a second tenant allocated to a second collection of PPU fractions of the multiple PPU fractions.
18 . The adapter of claim 17 , wherein the adapter controller is to:
allocate multiple jobs of the first tenant to a common PPU fraction.
19 . A method comprising:
receiving, by a job scheduler executed on a controller, the method, a request from a first tenant to schedule a first job in a system including a plurality of physical processing units (PPUs), wherein a physical PPU of the plurality of physical PPUs comprises multiple PPU fractions, and wherein a first PPU fraction of the multiple PPU fractions comprises a first PPU compute resource and a first PPU memory resource that is separate and isolated from a second PPU compute resource and a second PPU memory resource of a second PPU fraction of the multiple PPU fractions;
allocating, by the job scheduler, the first job to a first collection of PPU fractions of the multiple PPU fractions based on:
an operational cost reduction objective to reduce a cost associated with a usage of the plurality of physical PPUs,
a migration cost reduction objective to reduce a cost associated with migrating jobs between physical PPUs, and
a tenant isolation constraint to provide tenant isolation wherein a single tenant of a plurality of tenants including the first tenant is to use a PPU fraction of a physical PPU at a time; and
processing the first job according to the allocation of the first job to the first collection of PPU fractions, wherein data isolation is provided between the first job allocated to the first collection of PPU fractions and a second job of a second tenant allocated to a second collection of PPU fractions of the multiple PPU fractions.
20 . The method of claim 19 , wherein the plurality of physical PPUs comprise a plurality of graphics processing units (GPUs), and wherein a physical GPU of the plurality of GPUs comprises a first GPU compute resource and a first GPU memory resource of a first GPU fraction that is separate and isolated from a second GPU compute resource and a second GPU memory resource of a second GPU fraction.