CPU tight-coupled accelerator
An integrated circuit includes: a central processing unit (CPU) core; an accelerator; and an acceleration instruction queue connected to the CPU core and the accelerator. The CPU core is to: fetch and decode one or more instructions from among an instruction sequence in a programmed order; determine an instruction from among the one or more instructions containing an acceleration workload encoded therein; and queue the instruction containing the acceleration workload encoded therein in the acceleration instruction queue.
1. An integrated circuit comprising:
a central processing unit (CPU) core;
an accelerator; and
an acceleration instruction queue connected to the CPU core and the accelerator,
wherein the CPU core is configured to:
fetch and decode one or more instructions from among an instruction sequence in a programmed order, the one or more instructions comprising an acceleration workload for the accelerator and a CPU workload for the CPU core;
determine a first instruction from among the one or more instructions containing the acceleration workload encoded therein based on an instruction type of the first instruction indicating the acceleration workload;
queue the first instruction containing the acceleration workload encoded therein in the acceleration instruction queue;
determine a second instruction from among the one or more instructions containing the CPU workload therein based on an instruction type of the second instruction indicating the CPU workload; and
dispatch the second instruction to a CPU data path for the CPU core, and
wherein the instruction type indicating the acceleration workload comprises one or more tensor operations, and the instruction type indicating the CPU workload comprises at least one of a scalar workload, a vector workload, or a memory workload.
2. The integrated circuit of claim 1 , wherein the accelerator is configured to:
dequeue the first instruction containing the acceleration workload from the acceleration instruction queue;
receive operands associated with the acceleration workload from scratch memory of the CPU core; and
compute a result based on the operands and the dequeued first instruction.
3. The integrated circuit of claim 2 , wherein the accelerator is configured to dequeue instructions from the acceleration instruction queue in a first-in-first-out method.
4. The integrated circuit of claim 2 , wherein the accelerator is further configured to store the result in embedded memory of the accelerator.
5. The integrated circuit of claim 4 , wherein the CPU core, the accelerator, the scratch memory, and the embedded memory are integrated on the same chip as each other.
6. The integrated circuit of claim 4 , wherein the CPU core is configured to retrieve the result from the embedded memory of the accelerator, and store the result in the scratch memory of the CPU core.
7. The integrated circuit of claim 1 , wherein the accelerator instruction queue comprises a plurality of instruction queues defining different priorities from each other for the accelerator.
8. A computing system comprising:
an accelerator;
one or more processors integrated with the accelerator in the same integrated circuit; and
memory comprising instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to:
identify a programmed order for executing one or more CPU instructions, the one or more CPU instructions comprising an acceleration workload for the accelerator and a CPU workload from the one or more processors; and
execute the one or more CPU instructions according to the programmed order,
wherein to execute the one or more CPU instructions, the instructions cause the one or more processors to:
fetch and decode a first instruction in the programmed order from among the one or more CPU instructions;
dispatch the decoded first instruction to an accelerator data path from among a CPU pipeline based on an instruction type of the first instruction indicating the acceleration workload;
fetch and decode a second instruction in the programmed order from among the one or more CPU instructions; and
dispatch the decoded second instruction to a CPU data path from among the CPU pipeline based on an instruction type of the second instruction indicating the CPU workload, and
wherein the instruction type indicating the acceleration workload comprises one or more tensor operations, and the instruction type indicating the CPU workload comprises at least one of a scalar workload, a vector workload, or a memory workload.
9. The computing system of claim 8 , wherein the first instruction comprises an accelerator workload encoded therein to be dispatched to the accelerator data path, and
wherein the instructions further cause the one or more processors to:
enqueue the first instruction in an acceleration instruction queue; and
provide corresponding operands to the accelerator for compute based on the first instruction.
10. The computing system of claim 9 , wherein the accelerator is configured to:
dequeue the first instruction from the acceleration instruction queue;
compute a result based on the corresponding operands and the first instruction dequeued from the acceleration instruction queue; and
store the result in embedded memory of the accelerator.
11. The computing system of claim 10 , wherein the accelerator is configured to dequeue instructions from the acceleration instruction queue in a first-in-first-out method.
12. The computing system of claim 10 , wherein the instructions further cause the one or more processors to:
retrieve the result from the embedded memory of the accelerator; and
store the result in scratch memory.
13. The computing system of claim 12 , wherein the accelerator, the one or more processors, the embedded memory, and the scratch memory are integrated in the same integrated circuit.
14. A method for accelerating instructions, comprising:
identifying, by one or more processors, a programmed order for executing one or more instructions, the one or more instructions comprising an acceleration workload and a CPU workload;
determining, by the one or more processors, the acceleration workload encoded in a first instruction of the one or more instructions in the programmed order based on an instruction type of the first instruction indicating the acceleration workload;
dispatching, by the one or more processors, the first instruction to an accelerator data path from among a plurality of data paths of a CPU pipeline based on the determining that the acceleration workload is encoded in the first instruction;
determining, by the one or more processors, the CPU workload encoded in a second instruction of the one or more instructions in the programmed order based on an instruction type of the second instruction indicating the CPU workload; and
dispatching, by the one or more processors, the second instruction to a CPU data path from among the plurality of data paths of the CPU pipeline based on the determining that the CPU workload is encoded in the second instruction,
wherein the instruction type indicating the acceleration workload comprises one or more tensor operations, and the instruction type indicating the CPU workload comprises at least one of a scalar workload, a vector workload, or a memory workload.
15. The method of claim 14 , wherein the dispatching of the first instruction comprises:
enqueueing, by the one or more processors, the first instruction in an acceleration instruction queue; and
providing, by the one or more processors, corresponding operands to the accelerator data path for compute based on the first instruction.
16. The method of claim 15 , wherein the accelerator data path comprises an accelerator integrated with the one or more processors in the same integrated circuit, and the method further comprises:
dequeuing, by the accelerator, the first instruction from the acceleration instruction queue;
computing, by the accelerator, a result based on the corresponding operands and the first instruction dequeued from the acceleration instruction queue; and
storing, by the accelerator, the result in embedded memory of the accelerator.
17. The method of claim 16 , wherein the accelerator is configured to dequeue instructions from the acceleration instruction queue in a first-in-first-out method.
18. The method of claim 16 , further comprising:
retrieving, by the one or more processors, the result from the embedded memory of the accelerator; and
storing, by the one or more processors, the result in scratch memory of the one or more processors.
19. The method of claim 16 , wherein the accelerator and the one or more processors are co-processors or multi-processors of the same integrated circuit.