Processing device and method for managing tasks thereof
A neural processing device and a method for managing tasks thereof are provided. The neural processing device includes a neural core configured to perform a task and generate a completion signal for completion of the task, a core global configured to transfer task information for the task to the neural core and receive the completion signal of the task from the neural core, and a task manager configured to generate and transmit the task information to the core global, receive the completion signal from the core global, generate a completion report, and transmit the completion report.
1. A neural processing device comprising:
processing circuitry; and
task managing circuitry operably coupled to the processing circuitry,
wherein the task managing circuitry is configured to cause:
receiving a plurality of tasks;
storing the plurality of received tasks in a dependency check waiting memory;
determining whether reference tasks which a task in the dependency check waiting memory depends on are completed based on a dependency of the task in the dependency check waiting memory and a dependency check memory region, wherein the dependency of the task in the dependency check waiting memory represents a list of tasks which have to be completed for the task in the dependency check waiting memory to be transferred to the processing circuitry and the dependency check memory region represents whether reference tasks which tasks in the dependency check waiting memory depend on are completed;
transferring the task in the dependency check waiting memory to a process waiting memory in response to determining that reference tasks which the task in the dependency check waiting memory depends on are completed; and
transferring tasks in the process waiting memory to the processing circuitry, wherein
tasks are stored in a form of task descriptors in the dependency check waiting memory, and
tasks are stored in a form of task descriptors in the process waiting memory.
2. The neural processing device of claim 1 , wherein transferring the tasks in the process waiting memory to the processing circuitry comprises:
determining whether a task in the process waiting memory indicates that a run signal is required for the task in the process waiting memory to be processed,
when the task in the process waiting memory indicates that a run signal is not required, transferring the task in the process waiting memory to the processing circuitry regardless of the run signal, and
when the task in the process waiting memory indicates that that a run signal is required, transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal.
3. The neural processing device of claim 2 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal comprises:
in response to receiving the run signal, changing a state of a register to indicate that the run signal has been received, and
when the task in the process waiting memory indicates that that a run signal is required, transferring the task in the process waiting memory to the processing circuitry in response to determining that the state of the register indicates the run signal has been received.
4. The neural processing device of claim 3 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal further comprises:
in response to transferring the task in the process waiting memory to the processing circuitry, changing the state of the register to indicate that the run signal has not been received.
5. The neural processing device of claim 2 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal comprises:
in response to receiving the run signal, increasing a counter value in a register to indicate how many run signals have been received,
in response to transferring, to the processing circuitry, a task indicating that the run signal is required to be processed, decreasing the counter value in the register, and
when the task in the process waiting memory indicates that the run signal is required transferring the task in the process waiting memory to the processing circuitry in response to determining that the counter value is greater than zero.
6. The neural processing device of claim 2 , wherein the task managing circuitry is further configured to cause:
storing events in an even queue; and
in response to detecting that a number of entries in the even queue is larger than a threshold, ceasing to transfer tasks in the process waiting memory to the processing circuitry regardless of the run signal until transferring tasks to the processing circuitry is resumed.
7. The neural processing device of claim 2 , wherein the task managing circuitry is further configured to cause:
storing tasks which are not completed for a predetermined time in a timeout queue; and
storing tasks which are completed in a completed task queue.
8. The neural processing device of claim 7 , wherein the task managing circuitry is further configured to cause:
in response to detecting that a number of entries in the timeout queue is larger than a threshold, ceasing to transfer tasks in the process waiting memory to the processing circuitry regardless of the run signal until transferring tasks to the processing circuitry is resumed.
9. The neural processing device of claim 7 , wherein the task managing circuitry is further configured to cause:
in response to detecting that a number of entries in the completed task queue is larger than a threshold, ceasing to transfer tasks in the process waiting memory to the processing circuitry regardless of the run signal until transferring tasks to the processing circuitry is resumed.
10. The neural processing device of claim 7 ,
wherein the plurality of tasks are received from an upper-level task managing circuitry of the task managing circuitry,
wherein the task managing circuitry is further configured to cause:
determining whether a task in the completed task queue requires a report, and
transmitting, to the upper-level task managing circuitry, a completion report indicating that a task in the completed task queue is completed, when the task in the completed task queue requires the report.
11. The neural processing device of claim 7 ,
wherein the plurality of tasks are received from an upper-level task managing circuitry of the task managing circuitry,
wherein the task managing circuitry is further configured to cause:
determining whether a task in the timeout queue requires a report, and
transmitting, to the upper-level task managing circuitry, a timeout report indicating that a task in the timeout queue is not completed for a predetermined time, when the task in the timeout queue requires the report.
12. The neural processing device of claim 1 , wherein
the tasks are for a machine learning model,
the processing circuitry processes tasks which are transferred from the process waiting memory to generate task processing results, and
the neural processing device is configured to perform training or inference of the machine learning model based on the task processing results.
13. A method performed by a neural processing device, comprising:
receiving a plurality of tasks;
storing the plurality of received tasks in a dependency check waiting memory;
determining whether reference tasks which a task in the dependency check waiting memory depends on are completed based on a dependency of the task in the dependency check waiting memory and a dependency check memory region, wherein the dependency of the task in the dependency check waiting memory represents a list of tasks which have to be completed for the task in the dependency check waiting memory to be transferred to a processing circuitry and the dependency check memory region represents whether reference tasks which tasks in the dependency check waiting memory depend on are completed;
transferring the task in the dependency check waiting memory to a process waiting memory in response to determining that reference tasks which the task in the dependency check waiting memory depends on are completed; and
transferring tasks in the process waiting memory to the processing circuitry, wherein
tasks are stored in a form of task descriptors in the dependency check waiting memory, and
tasks are stored in a form of task descriptors in the process waiting memory.
14. The method of claim 13 , wherein transferring the tasks in the process waiting memory to the processing circuitry comprises:
determining whether a task in the process waiting memory indicates that a run signal is required for the task in the process waiting memory to be processed,
when the task in the process waiting memory indicates that a run signal is not required, transferring the task in the process waiting memory to the processing circuitry regardless of the run signal, and
when the task in the process waiting memory indicates that that a run signal is required, transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal.
15. The method of claim 14 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal comprises:
in response to receiving the run signal, changing a state of a register to indicate that the run signal has been received, and
when the task in the process waiting memory indicates that that a run signal is required, transferring the task in the process waiting memory to the processing circuitry in response to determining that the state of the register indicates the run signal has been received.
16. The method of claim 15 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal further comprises:
in response to transferring the task in the process waiting memory to the processing circuitry, changing the state of the register to indicate that the run signal has not been received.
17. The method of claim 14 , wherein transferring the task in the process waiting memory to the processing circuitry in response to receiving the run signal comprises:
in response to receiving the run signal, increasing a counter value in a register to indicate how many run signals have been received,
in response to transferring, to the processing circuitry, a task indicating that the run signal is required to be processed, decreasing the counter value in the register, and
when the task in the process waiting memory indicates that the run signal is required transferring the task in the process waiting memory to the processing circuitry in response to determining that the counter value is greater than zero.
18. The method of claim 13 , wherein the tasks are for a machine learning model,
the method further comprises:
processing, using the processing circuitry, tasks which are transferred from the process waiting memory to generate task processing results; and
performing training or inference of the machine learning model based on the task processing results.