Distributed synchronization scheme

1. A system comprising:

a machine-learning accelerator (MLA) hardware comprising computation-control units that each have a programmable dependency matrix; and

a compiler computing module configured to generate, based on a machine-learning model, dependency instructions indicating dependencies between the computation-control units;

wherein the computation-control units include at least:

a first computation-control unit configured to generate, after completion of a first operation, a synchronization token representing the completion of the first operation, the synchronization token specifying a recipient identifier for an intended recipient computation-control unit of the synchronization token;

a second computation-control unit configured to:

configure the programmable dependency matrix of the second computation-control unit according to the dependency instructions to include dependency conditions for performing operations;

receive the synchronization token based on the recipient identifier;

update a dependency state to reflect the received synchronization token; and

execute an operation in response to a determination that the dependency state satisfies an associated dependency condition.

2. The system of claim 1 , wherein the first computation-control unit and the second computation-control unit each generates the respective programmable dependency matrix based on the dependencies between the computation-control units.

3. The system of claim 2 , wherein the second computation-control unit updates the dependency state by incrementing a count of the respective programmable dependency matrix that corresponds to the received synchronization token.

4. The system of claim 3 , wherein the second computation-control unit executes the operation based on the count.

5. The system of claim 1 , wherein the synchronization token specifies a recipient identifier for an intended recipient based on a channel identifier (ID) of a channel that the synchronization token is transmitted on between the first computation-control unit and the second computation-control unit.

6. The system of claim 1 , wherein the synchronization token specifies a receipt identifier for an intended receipt based on a client identifier (ID) of the second computation-control unit.

7. The system of claim 1 , wherein the second computation-control unit, after execution of the operation, generates an additional synchronization token representing the completion of the operation, the additional synchronization token specifying a recipient identifier for an intended recipient computation-control unit of the additional synchronization token.

8. The system of claim 1 , wherein the computation-control units:

operate asynchronously without a deterministic timing relationship with one another; and

execute operations according to each unit's respective programmable dependency matrix.

9. A method, comprising:

generating, by a compiler computing module, dependency instructions, based on a machine-learning model, indicating dependencies between computation-control units;

generating, by a first computation-control unit of a machine-learning accelerator (MLA), after completion of a first operation, a synchronization token representing the completion of the first operation, the synchronization token specifying a recipient identifier for an intended recipient computation-control unit of the synchronization token;

configuring, by a second computation-control unit of the MLA, according to the dependency instructions, dependency conditions for performing operations;

receiving, by the second computation-control unit, the synchronization token based on the recipient identifier;

updating, by the second computation-control unit, a dependency state to reflect the received synchronization token; and

executing, by the second computation-control unit, an operation in response to a determination that the dependency state satisfies an associated dependency condition.

10. The method of claim 9 , wherein the first computation-control unit and the second computation-control unit each generates the respective programmable dependency matrix based on the dependencies between the computation-control units.

11. The method of claim 10 , wherein the second computation-control unit updates the dependency state by incrementing a count of the respective programmable dependency matrix that corresponds to the received synchronization token.

12. The method of claim 11 , wherein the second computation-control unit executes the operation based on the count.

13. The method of claim 10 , wherein the synchronization token specifies a recipient identifier for an intended recipient based on a channel identifier (ID) of a channel that the synchronization token is transmitted on between the first computation-control unit and the second computation-control unit.

14. The method of claim 10 , wherein the synchronization token specifies a receipt identifier for an intended receipt based on a client identifier (ID) of the second computation-control unit.

15. The method of claim 10 , wherein the second computation-control unit, after execution of the operation, generates an additional synchronization token representing the completion of the operation, the additional synchronization token specifying a recipient identifier for an intended recipient computation-control unit of the additional synchronization token.

16. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

generating, by a compiler computing module, dependency instructions, based on a machine-learning model, indicating dependencies between the computation-control units;

generating, by a first computation-control unit of a machine-learning accelerator (MLA), after completion of a first operation, a synchronization token representing the completion of the first operation, the synchronization token specifying a recipient identifier for an intended recipient computation-control unit of the synchronization token;

configuring, by a second computation-control unit of the MLA, according to the dependency instructions, dependency conditions for performing operations;

receiving, by the second computation-control unit, the synchronization token based on the recipient identifier;

updating, by the second computation-control unit, a dependency state to reflect the received synchronization token; and

executing, by the second computation-control unit, an operation in response to a determination that the dependency state satisfies an associated dependency condition.

17. The computer-readable non-transitory storage media of claim 16 , wherein the first computation-control unit and the second computation-control unit each generates the respective programmable dependency matrix based on the dependencies between the computation-control units.

18. The computer-readable non-transitory storage media of claim 17 , wherein the second computation-control unit updates the dependency state by incrementing a count of the respective programmable dependency matrix that corresponds to the received synchronization token.

19. The computer-readable non-transitory storage media of claim 18 , wherein the second computation-control unit executes the operation based on the count.

20. The computer-readable non-transitory storage media of claim 16 , wherein the synchronization token specifies a recipient identifier for an intended recipient based on a channel identifier (ID) of a channel that the synchronization token is transmitted on between the first computation-control unit and the second computation-control unit.