Chained buffers in neural network processor
Embodiments of the present disclosure relate to chained buffers in a neural processor circuit. The neural processor circuit includes multiple neural engines, a planar engine, a buffer memory, and a flow control circuit. At least one neural engine operates as a first producer of first data or a first consumer of second data. The planar engine operates as a second consumer receiving the first data from the first producer or a second producer sending the second data to the first consumer. Data flow between the at least one neural engine and the planar engine is controlled using at least a subset of buffers in the buffer memory operating as at least one chained buffer that chains flow of the first data and the second data between the at least one neural engine and the planar engine.
1. A neural processor circuit, comprising:
a plurality of neural engine circuits, at least one of the neural engine circuits configured to produce first data or consume second data by performing at least convolution operations on a channel of data;
a planar engine circuit configured to consume the first data received from the at least one neural engine circuit or produce the second data for the at least one neural engine circuit by performing at least operations on one or more channels of data; and
a buffer memory coupled to the at least one neural engine circuit and the planar engine circuit, the buffer memory including a plurality of buffers, at least one of the buffers configured to control data flow between the at least one neural engine circuit and the planar engine circuit by operating as a chained buffer, wherein the chained buffer:
controls flow of the first data from the at least one neural engine circuit to the planar engine circuit, and
controls flow of the second data from the planar engine circuit to the at least one neural engine circuit.
2. The neural processor circuit of claim 1 , wherein, responsive to a result descriptor for a producing task of the at least one neural engine circuit or the planar engine circuit being set as chained, the chained buffer is configured to:
control data flow produced by the at least one neural engine circuit or the planar engine circuit; and
store the first data or the second data produced by the producing task.
3. The neural processor circuit of claim 2 , wherein the neural processor circuit further comprises:
a neural task manager configured to stall the producing task at the at least one neural engine circuit or the planar engine circuit responsive to storing a defined amount of the first data or the second data in the chained buffer and before starting of a consuming task of the at least one neural engine circuit or the planar engine circuit.
4. The neural processor circuit of claim 1 , wherein, responsive to a source descriptor for a consuming task of the at least one neural engine circuit or the planar engine circuit being set as chained, the chained buffer is configured to:
control data flow to the at least one neural engine circuit or the planar engine circuit; and
provide the first data or the second data to a corresponding circuit of the at least one neural engine circuit or the planar engine circuit.
5. The neural processor circuit of claim 4 , wherein the neural processor circuit further comprises:
a neural task manager configured to start the consuming task after starting the producing task, responsive to the corresponding circuit of the at least one neural engine circuit or the planar engine circuit not operating in relation to another task different than the consuming task.
6. The neural processor circuit of claim 4 , wherein the neural processor circuit further comprises:
a neural task manager configured to stall the consuming task until the producing task writes a threshold amount of the first data or the second data into the chained buffer.
7. The neural processor circuit of claim 4 , wherein the neural processor circuit is further configured to:
map output slices of the first data or the second data produced in the producing task and stored in the chained buffer into input slices of data for the consuming task.
8. The neural processor circuit of claim 1 , wherein the chained buffer is sized such that sizes of tiles and patches of the first data or the second data produced by the at least one neural engine circuit or the planar engine circuit match sizes of tiles and patches of the first data or the second data for consumption by the at least one neural engine circuit or the planar engine circuit.
9. The neural processor circuit of claim 1 , wherein the chained buffer is configured to simultaneously store at most a defined amount of the first data or the second data.
10. The neural processor circuit of claim 1 , wherein, responsive to storing a threshold amount of the first data or the second data in the chained buffer, the chained buffer is configured to provide at least the threshold amount of the first data to the planar engine circuit or provide at least the threshold amount of the second data to the at least one neural engine circuit.
11. A method of operating a neural processor circuit, comprising:
operating at least one neural engine circuit of a plurality of neural engine circuits to produce first data or consume second data by performing at least convolution operations on a channel of data;
operating a planar engine circuit to consume the first data received from the at least one neural engine circuit or produce the second data for the at least one neural engine circuit by performing at least operations on one or more channels of data; and
controlling data flow between the at least one neural engine circuit and the planar engine circuit using at least one of a plurality of buffers in a buffer memory operating as a chained buffer, wherein the chained buffer:
controls flow of the first data from the at least one neural engine circuit to the planar engine circuit, and
controls flow of the second data from the planar engine circuit to the at least one neural engine circuit.
12. The method of claim 11 , further comprising:
setting a result descriptor for a producing task of the at least one neural engine circuit or the planar engine circuit as chained; and
responsive to setting the result descriptor as chained, storing the first data or the second data produced by the producing task into the chained buffer.
13. The method of claim 12 , further comprising:
stalling the producing task at the at least one neural engine circuit or the planar engine circuit, responsive to storing a defined amount of the first data or the second data in the chained buffer and before starting of a consuming task of the at least one neural engine circuit or the planar engine circuit.
14. The method of claim 11 , further comprising:
setting a source descriptor for a consuming task of the at least one neural engine circuit or the planar engine circuit as chained; and
responsive to setting the source descriptor as chained, providing the first data or the second data via the chained buffer to the corresponding circuit of the at least one neural engine circuit or the planar engine circuit.
15. The method of claim 14 , further comprising:
starting the consuming task after starting the producing task, responsive to a corresponding circuit of the at least one neural engine circuit or the planar engine circuit is not operating in relation to another task different than the consuming task.
16. The method of claim 14 , further comprising:
stalling the consuming task until the producing task writes a threshold amount of the first data or the second data into the chained buffer.
17. The method of claim 14 , further comprising:
mapping output slices of the first data or the second data produced in the producing task and stored in the chained buffer into input slices of data for the consuming task.
18. The method of claim 11 , further comprising:
storing in the chained buffer simultaneously at most a defined amount of the first data or the second data; and
after storing a threshold amount of the first data or the second data in the chained buffer,
providing, via the chained buffer, at least the threshold amount of the first data to the planar engine circuit, or
providing, via the chained buffer, at least the threshold amount of the second data to the at least one neural engine circuit.
19. An electronic device, comprising:
a neural processor circuit including:
neural engine circuits at least one of which is configured to produce first data or of consume second data by performing at least convolution operations on a channel of data,
a planar engine circuit configured to consume the first data received from the at least one neural engine circuit or produce the second data for the at least one neural engine circuit by performing at least operations on one or more channels of data, and
a buffer memory coupled to the at least one neural engine circuit and the planar engine circuit, the buffer memory including a plurality of buffers, at least one of the buffers configured to control data flow between the at least one neural engine circuit and the planar engine circuit by operating as a chained buffer, wherein the chained buffer:
controls flow of the first data from the at least one neural engine circuit to the planar engine circuit, and
controls flow of the second data from the planar engine circuit to the at least one neural engine circuit.
20. The electronic device of claim 19 , further comprising a system memory external to the neural processor circuit and coupled to the neural processor circuit, the system memory configured to:
store input data a subset of which is sent to the buffer memory at a time for processing, and
store one or more kernels sent to the at least one neural engine circuit for performing the at least convolution operations.