Architecture for virtual instructions

1. A system, comprising:

a machine learning accelerator (MLA) hardware configured to perform machine-learning operations according to native instructions;

an interpreter computing module configured to:

generate, based on virtual instructions, machine language instructions configured to be processed by a processing hardware implementing the interpreter computing module; and

cause the processing hardware to perform machine-learning operations according to the machine language instructions; and

a compiler computing module associated with the MLA hardware, the compiler computing module configured to:

receive instructions for performing an inference using a machine-learning model;

based on the received instructions:

generate the native instructions configured to be processed by the MLA hardware, the native instructions specifying first machine-learning operations associated with performing the inference; and

generate the virtual instructions configured to be processed by the interpreter computing module, the virtual instructions specifying second machine-learning operations associated with performing the inference.

2. The system of claim 1 , further comprising a local memory of the MLA hardware, wherein the first machine-learning operations comprise computations using weights associated with the machine-learning model and activations, wherein the native instructions specify memory locations of the activations and the weights within the local memory of the MLA hardware.

3. The system of claim 2 , wherein the second machine-learning operations comprise data transfers between the local memory of the MLA hardware and a system memory of the processing hardware.

4. The system of claim 1 , wherein the compiler computing module is further configured to:

identify first dependencies of the first machine-learning operations of the native instructions, and encode the first dependencies within the native instructions; and

identify second dependencies of the second machine-learning operations of the virtual instructions, and encode the second dependencies within the virtual instructions;

wherein the first machine-learning operations and the second machine-learning operations are synchronized based on the first dependencies and the second dependencies.

5. The system of claim 1 , wherein the MLA hardware is further configured to:

generate a synchronization token indicating a processing completion of one of the first machine-learning operations on which at least one of the second machine-learning operations depends; and

transmit, to the interpreter computing module, the synchronization token.

6. The system of claim 5 , wherein the interpreter computing module is configured to execute the at least one of the second machine-learning operations based on a determination that the synchronization token received from the MLA hardware satisfies one or more dependency criteria associated with the at least one of the second machine-learning operations.

7. The system of claim 1 , wherein the interpreter computing module is further configured to:

generate a synchronization token indicating a processing completion of one of the second machine-learning operations on which at least one of the first machine-learning operations depends; and

transmit, to the MLA hardware, the synchronization token.

8. The system of claim 7 , wherein the MLA hardware is configured to execute the at least one of the first machine-learning operations based on a determination that the synchronization token received from the interpreter computing module satisfies one or more dependency criteria associated with the at least one of the first machine-learning operations.

9. The system of claim 1 , wherein the processing hardware is a central processing unit.

10. The system of claim 1 , wherein the native instructions are generated according to a first instruction set architecture of the MLA, and the virtual instructions are generated according to a second instruction set architecture of the interpreter.

11. A method, comprising:

receiving, by a compiler computing module, instructions for performing an inference using a machine-learning model;

generate, by the compiler computing module and based on the received instructions, native instructions configured to be processed by MLA hardware, the native instructions specifying first machine-learning operations associated with performing the inference;

generate, by the compiler computing module and based on the received instructions, virtual instructions configured to be processed by an interpreter computing module, the virtual instructions specifying second machine-learning operations associated with performing the inference;

performing, by the MLA hardware, machine-learning operations according to native instructions;

generating, by the interpreter computing module and based on the virtual instructions, machine language instructions configured to be processed by a processing hardware implementing the interpreter computing module; and

causing, by the interpreter computing module, the processing hardware to perform machine-learning operations according to the machine language instructions.

12. The method of claim 11 , wherein the first machine-learning operations comprise computations using weights associated with the machine-learning model and activations, wherein the native instructions specify memory locations of the activations and the weights within a local memory of the MLA hardware.

13. The method of claim 12 , wherein the second machine-learning operations comprise data transfers between the local memory of the MLA hardware and a system memory of the processing hardware.

14. The method of claim 11 , further comprising:

identifying, by the compiler computing module, first dependencies of the first machine-learning operations of the native instructions, and encode the first dependencies within the native instructions; and

identifying, by the compiler computing module, second dependencies of the second machine-learning operations of the virtual instructions, and encode the second dependencies within the virtual instructions;

wherein the first machine-learning operations and the second machine-learning operations are synchronized based on the first dependencies and the second dependencies.

15. The method of claim 11 , further comprising:

generating, by the MLA hardware, a synchronization token indicating a processing completion of one of the first machine-learning operations on which at least one of the second machine-learning operations depends; and

transmitting, by the MLA hardware and to the interpreter computing module, the synchronization token.

16. The method of claim 15 , further comprising executing the at least one of the second machine-learning operations based on a determination that the synchronization token received from the MLA hardware satisfies one or more dependency criteria associated with the at least one of the second machine-learning operations.

17. The method of claim 11 , further comprising:

generating, by the interpreter computing module, a synchronization token indicating a processing completion of one of the second machine-learning operations on which at least one of the first machine-learning operations depends; and

transmit, by the interpreter computing module and to the MLA hardware, the synchronization token.

18. The method of claim 17 , further comprising executing the at least one of the first machine-learning operations based on a determination that the synchronization token received from the interpreter computing module satisfies one or more dependency criteria associated with the at least one of the first machine-learning operations.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receiving, by a compiler computing module, instructions for performing an inference using a machine-learning model;

generate, by the compiler computing module and based on the received instructions, native instructions configured to be processed by MLA hardware, the native instructions specifying first machine-learning operations associated with performing the inference;

generate, by the compiler computing module and based on the received instructions, virtual instructions configured to be processed by an interpreter computing module, the virtual instructions specifying second machine-learning operations associated with performing the inference;

performing, by the MLA hardware, machine-learning operations according to native instructions;

generating, by the interpreter computing module and based on the virtual instructions, machine language instructions configured to be processed by a processing hardware implementing the interpreter computing module; and

causing, by the interpreter computing module, the processing hardware to perform machine-learning operations according to the machine language instructions.

20. The computer-readable non-transitory storage media of claim 19 , wherein the first machine-learning operations comprise computations using weights associated with the machine-learning model and activations, wherein the native instructions specify memory locations of the activations and the weights within a local memory of the MLA hardware.