Neural processing device and method for job scheduling thereof

1. A neural processing device including circuitry configured to cause:

managing address space IDs (ASID), wherein a number of the ASIDs is predetermined;

determining times of allocating the ASIDs for each of a plurality of contexts;

providing a first scheduling path or a second scheduling path for each of the plurality of contexts; and

consecutively receiving and executing jobs of a set of the plurality of contexts to which the ASIDs are allocated among the plurality of contexts,

wherein a first set of the plurality of contexts are provided the first scheduling path by being allocated the ASIDs at a first time, and

a second set of the plurality of contexts are provided the second scheduling path by being allocated the ASIDs at a second time.

2. The neural processing device of claim 1 , wherein a number of the first set of the plurality of contexts are same as the predetermined number of the ASIDs.

3. The neural processing device of claim 1 , wherein the circuitry is further configured to cause:

receiving ASID allocation requests from the plurality of contexts; and

determining the first set of the plurality of contexts to which the first scheduling path is provided among the plurality of contexts.

4. The neural processing device of claim 3 , wherein the circuitry is further configured to cause:

receiving the ASID allocation requests consecutively from the plurality of contexts according to times of generating jobs for each of the plurality of contexts; and

determining the first set of the plurality of contexts to which the first scheduling path is provided according to an order of receiving the ASID allocation requests.

5. The neural processing device of claim 1 , wherein the first set of the plurality of contexts comprises a first context that performs a first job, and

the second set of the plurality of contexts comprises a second context that performs a second job.

6. The neural processing device of claim 5 , wherein the circuitry is further configured to cause:

unbinding an ASID allocated to the first context; and

allocating the unbound ASID to the second context at the second time.

7. The neural processing device of claim 6 , wherein the circuitry is further configured to cause:

unbinding the ASID allocated to the first context when the first job is executed.

8. The neural processing device of claim 6 , wherein the circuitry is further configured to cause:

allocating a least recently used (LRU) ASID among unbound ASIDs to the second context when there exist a plurality of unbound ASIDs.

9. The neural processing device of claim 5 , wherein the circuitry is further configured to cause:

storing the second job of the second context; and

scheduling an order of the second job stored, and providing the second scheduling path to the second job at the second time,

wherein scheduling the order comprises:

pushing the second job to be stored;

emitting the second job to be provided to a command queue; and

executing the second job.

10. The neural processing device of claim 9 , wherein a first scheduling overhead exists between a time the second job is pushed and a time the second job is emitted.

11. The neural processing device of claim 10 , wherein a second scheduling overhead exists between the time the second job is emitted and a time the second job is executed.

12. The neural processing device of claim 9 , wherein the first context is executed without scheduling overhead.

13. A method for job scheduling of a neural processing device, comprising:

receiving, by address space ID (ASID) manager circuitry, an ASID allocation request from a first context;

determining, by the ASID manager circuitry, whether there is any unbound ASID among ASIDs;

in response to a determination that there is an unbound ASID, allocating the unbound ASID to the first context; and

providing, by the first context, a first job to be executed by the first context directly to a command queue.

14. The method for job scheduling of the neural processing device of claim 13 , further comprising in response to a determination that there is no unbound ASID:

providing, by the first context, the first job to a first entity;

allocating, by job scheduler circuitry, an ASID to the first context via the ASID manager; and

providing, by the job scheduler circuitry, the first job to the command queue via the ASID.

15. The method for job scheduling of the neural processing device of claim 14 , further comprising executing, by the neural processing device, the first job provided to the command queue.

16. The method for job scheduling of the neural processing device of claim 15 , wherein the neural processing device comprises at least one neural processor and a shared memory shared by the at least one neural processor.

17. The method for job scheduling of the neural processing device of claim 13 , wherein allocating the unbound ASID comprises:

selecting unbound ASIDs among the ASIDs;

choosing a least recently used (LRU) ASID that is an oldest previously used among the unbound ASIDs; and

allocating the LRU ASID to the first context.

18. The method for job scheduling of the neural processing device of claim 17 , wherein allocating the unbound ASID further comprises job scheduler circuitry sequentially allocating the ASIDs.

19. A method for job scheduling of a neural processing device including circuitry configured to managing address space IDs (ASID) of a predetermined number, comprising:

receiving ASID allocation requests from a plurality of contexts;

providing a first scheduling path or a second scheduling path to each of the plurality of contexts by determining times of allocating the ASIDs for each of the plurality of contexts; and

executing jobs of contexts of which the ASIDs are allocated among the plurality of contexts,

wherein providing the first scheduling path or the second scheduling path comprises:

providing the first scheduling path to a first set of the plurality of contexts at a first time; and

providing the second scheduling path to a second set of the plurality of contexts at a second time later than the first time in time series.

20. The method for job scheduling of the neural processing device of claim 19 , wherein providing the first scheduling path to the first set of the plurality of contexts comprises:

receiving the ASID allocation requests from the plurality of contexts; and

determining the first set of the plurality of contexts to be provided the first scheduling path among the plurality of contexts from which the ASID allocation requests are received.