Granted Patent B2

US 12,210,959 · App. 17/155,896 · Granted Jan 28, 2025

Inventors: Kenneth W. Waters (San Jose, CA); Christopher L. Mills (Saratoga, CA)

Assignee: APPLE INC.

G06N3/063G06F9/3005G06F9/4843G06N3/045

Abstract

Claims (53)

1. A neural processor circuit, comprising:

one or more neural engine circuits configured to perform convolution operations on input data corresponding to a first task to generate output data;

a data processor circuit coupled to the one or more neural engine circuits, the data processor circuit configured to:

receive the output data from the one or more neural engine circuits; and

generate a branching command from the output data; and

a task manager circuit coupled to the data processor circuit, the task manager circuit configured to:

receive the branching command from the data processor circuit;

enqueue one of two or more segment branches according to the received branching command, the two or more segment branches subsequent to a task segment comprising the first task, wherein the branching command and the one of the two or more segment branches are executable by the neural processor circuit; and

transmit a second task from a selected one of the two or more segment branches to the data processor circuit to perform the second task.

2. The neural processor circuit of claim 1 , wherein the task segment is assigned to a first neural network and the selected one of the two or more segment branches is assigned to a second neural network different from the first neural network.

3. The neural processor circuit of claim 2 , wherein the one or more neural engine circuits are configured to perform the convolution operations on the input data corresponding to the first task via the first neural network, and wherein the task manager circuit is further configured to:

enqueue a context switch task that causes the second neural network to perform convolution operations on input data corresponding to the second task.

4. The neural processor circuit of claim 1 , wherein the branching command is generated based on comparing one or more values in the output data to one or more reference values.

5. The neural processor circuit of claim 1 , wherein the selected one of the two or more segment branches is enqueued to a task queue in which the task segment is enqueued.

6. The neural processor circuit of claim 1 , wherein addresses of the two or more segment branches are stored in the task segment.

7. The neural processor circuit of claim 1 , the first task that determines the branching command is identified by a branching task identifier stored in the task segment.

8. The neural processor circuit of claim 1 , wherein the first task that determines the branching command is a last task in the task segment.

9. The neural processor circuit of claim 1 , wherein the task manager circuit is further configured to:

determine, prior to receiving the branching command, that tasks in the task segment have been transmitted for execution; and

pause the task manager circuit until the branching command is received.

10. The neural processor circuit of claim 1 , wherein the task manager circuit is further configured to:

determine, prior to receiving the branching command, that tasks in the task segment have been transmitted for execution; and

process, prior to receiving the branching command, a separate task segment that is different from the two or more segment branches.

11. A method of performing neural processing operations by a neural processor circuit, comprising:

performing, by one or more neural engine circuits, convolution operations on input data corresponding to a first task to generate output data;

receiving, by a data processor circuit, the output data from the one or more neural engine circuits;

generating a branching command from the output data;

receiving, by a task manager circuit, the branching command from the data processor circuit;

enqueuing one of two or more segment branches according to the branching command, the two or more segment branches subsequent to a task segment comprising the first task, wherein the branching command and the one of the two or more segment branches are executable by the neural processor circuit; and

transmitting a second task from a selected one of the two or more segment branches to the data processor circuit to perform the second task.

12. The method of claim 11 , wherein the pre branch task segment is assigned to a first neural network and the selected one of the two or more segment branches is assigned to a second neural network different from the first neural network.

13. The method of claim 12 , performing the convolution operations on the input data corresponding to the first task comprises performing the convolution operations on the input data corresponding to the first task via the first neural network, and wherein the method further comprises:

enqueuing a context switch task that causes the second neural network to perform convolution operations on input data corresponding to the second task.

14. The method of claim 11 , wherein the branching command is generated based on comparing one or more values in the output data to one or more reference values.

15. The method of claim 11 , wherein the selected one of the two or more segment branches is enqueued to a task queue in which the task segment is enqueued.

16. The method of claim 11 , further comprising:

determining, prior to receiving the branching command, that tasks in the task segment have been transmitted for execution; and

processing, prior to receiving the branching command, a separate task segment that is different from the two or more segment branches.

17. An electronic device, comprising:

a system memory storing one or more machine learning models; and

a neural processor, comprising:

one or more neural engine circuits configured to perform convolution operations on input data corresponding to a first task to generate output data;

a data processor circuit coupled to the one or more neural engine circuits, the data processor circuit configured to:

receive the output data from the one or more neural engine circuits; and

generate a branching command from the output data; and

a task manager circuit coupled to the data processor circuit, the task manager circuit configured to:

receive the branching command from the data processor circuit;

enqueue one of two or more segment branches according to the received branching command, the two or more segment branches subsequent to a task segment comprising the first task, wherein the branching command and the one of the two or more segment branches are executable by the neural processor; and

transmit a second task from a selected one of the two or more segment branches to the data processor circuit to perform the second task.

18. The electronic device of claim 17 , wherein the task segment is assigned to a first neural network and the selected one of the two or more segment branches is assigned to a second neural network different from the first neural network.

19. The electronic device of claim 18 , wherein the one or more neural engine circuits are configured to perform the convolution operations on the input data corresponding to the first task via the first neural network, and wherein the task manager circuit is further configured to:

enqueue a context switch task that causes the second neural network to perform convolution operations on input data corresponding to the second task.

20. The electronic device of claim 17 , wherein the branching command correspond to a prediction result of one of the machine learning models.

Assignments (1)

Continuity (1)

References Cited (33)