Explicit scheduling of on-chip operations
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining a first schedule, for a first hardware block of an integrated circuit device, where the first schedule identifies a first set of operations to be performed by the first hardware block. Obtaining a second schedule for a second hardware block of the integrated circuit device, where the second schedule identifies a second set of operations to be performed by the second hardware block and where operations of the second schedule are coordinated with operations of the first schedule such that the first schedule triggers the first hardware block to send data to the second block at a first pre-scheduled value of a counter, and the second schedule triggers the second hardware block to accept the data at an input at a second pre-scheduled value of the counter that is after the first pre-scheduled value. Performing, by the first hardware block, the first set of operations according to the first schedule, and performing, by the second hardware block, the second set of operations according to the second schedule.
1. An integrated circuit device comprising:
a counter;
a first hardware block communicably coupled to the counter and configured to operate according to a first schedule that comprises a first set of operations each of which is scheduled to be executed by the first hardware block at a first respective value of the counter; and
a second hardware block communicably coupled to the counter and to the first hardware block, the second hardware block configured to operate according to a second schedule that comprises a second set of operations each of which is scheduled to be executed by the second hardware block at a second respective value of the counter, and
wherein operations of the second schedule are coordinated with operations of the first schedule such that compute operations in the first schedule are executed concurrently with data exchange operations in the second schedule, and that the first schedule triggers the first hardware block to send data to the second hardware block at a first pre-scheduled value of the counter, and the second schedule triggers the second hardware block to accept the data at an input at a second pre-scheduled value of the counter that is after the first pre-scheduled value.
2. The device of claim 1 , wherein the first set of operations and the second set of operations each comprise a respective portion of a machine learning program.
3. The device of claim 1 , wherein each operation in the first set of operations executes in a predetermined number of clock cycles.
4. The device of claim 1 , wherein operations of the first schedule and the second schedule are coordinated to allow exchange of data between the first hardware block and the second hardware block independent of flow control signals.
5. The device of claim 1 , further comprising a plurality of other hardware blocks, wherein operations of the first schedule are coordinated with respective operation schedules of the other hardware blocks to allow exchange data between the first hardware block and one or more of the other hardware blocks independent of data flow control signals.
6. The device of claim 1 , wherein the first hardware block comprises:
local memory configured to store the first schedule; and
control circuitry coupled to the local memory and configured to execute the first set of operations of the first schedule.
7. The device of claim 6 , wherein the control circuitry is configured to decompress a portion of the first schedule before executing operations included in the portion.
8. The device of claim 1 , wherein the integrated circuit device is an application specific integrated circuit.
9. The device of claim 1 , wherein the first hardware block and the second hardware block are hardware tiles including special purpose circuitry configured to perform neural network operations.
10. The device of claim 9 , wherein the first hardware block comprises:
a computational array of cells; and
local memory coupled to the computational array of cells.
11. The device of claim 1 , wherein the first schedule and the second schedule each comprise a portion of a program executed by the integrated circuit device.
12. An integrated circuit operating method comprising:
obtaining, for a first hardware block of an integrated circuit device, a first schedule that identifies a first set of operations to be performed by the first hardware block;
obtaining, for a second hardware block of the integrated circuit device, a second schedule that identifies a second set of operations to be performed by the second hardware block, wherein operations of the second schedule are coordinated with operations of the first schedule such that compute operations in the first schedule are executed concurrently with data exchange operations in the second schedule, and that the first schedule triggers the first hardware block to send data to the second block at a first pre-scheduled value of a counter, and the second schedule triggers the second hardware block to accept the data at an input at a second pre-scheduled value of the counter that is after the first pre-scheduled value;
performing, by the first hardware block, the first set of operations according to the first schedule; and
performing, by the second hardware block, the second set of operations according to the second schedule.
13. The method of claim 12 , wherein the first schedule and the second schedule each comprise a portion of a program executed by the integrated circuit device.
14. The method of claim 12 , wherein the first set of operations and the second set of operations each comprise a respective portion of a machine learning program.
15. The method of claim 12 , wherein each operation in the first set of operations executes in a predetermined number of clock cycles.
16. The method of claim 12 , wherein operations of the first schedule and the second schedule are coordinated to allow exchange of data between the first hardware block and the second hardware block independent of flow control signals.
17. The method of claim 12 , further comprising decompressing, by the first hardware block, a portion of the first schedule before executing operations included in the portion.
18. The method of claim 12 , wherein the first schedule comprises, for each operation in the first set of operations, a scheduled counter value and data indicating a particular operation to be executed by the first hardware block at the scheduled counter value.
19. The method of claim 12 , wherein performing, by the first hardware block, the first set of operations according to the first schedule comprises:
receiving, from a counter, a first counter value that equals a first scheduled counter value of a first operation in the first set of operations;
in response to receiving the first counter value, causing a first set of one or more computational units of the first hardware block to execute the first operation;
receiving, from the counter, a second counter value that equals a second scheduled counter value of a second operation in the first set of operations; and
in response to receiving the second counter value, causing a second set of one or more computational units of the first hardware block to execute the second operation.
20. The method of claim 12 , wherein the first hardware block and the second hardware block are hardware tiles including special purpose circuitry configured to perform neural network operations.