Node topology employing recirculating ring command and data buffers for executing thread scheduling
The claimed invention discloses system comprising a plurality of logical nodes comprised in a single or plurality of stages, with defined properties and resources associated with each node, for reducing compute resources, said system further comprising: at least a recirculating ring buffer holding only any one of a control information, input, and, or out data necessary to stream a temporary data between node and, or nodes in an execution graph, thereby reducing size of said recirculating ring buffer; said recirculating ring buffer being sufficiently reduced in size to reside in an on-chip cache, such that any one of the control information, input, and, or out data between node and, or nodes need not be stored in memory; wherein the control information further comprises a command related to invalidating any one of the input and, or out data held in a recirculating ring data buffer, clearing the buffer of tasked data; and wherein a producer is stalled from writing any more control information into a recirculating ring command buffer upon the buffer being full, preventing command buffer over-writing, and thereby reducing compute resources associated with a DRAM memory transaction.
1. A machine configured for increased efficiency processing flow, comprising an arrangement of:
a plurality of stages configured to process an execution graph that includes a plurality of logical nodes with defined properties and resources associated with each logical node of the plurality of logical nodes;
a recirculating ring buffer, wherein the recirculating ring buffer is configured to holding only any one of a control information, input, and, or out data necessary to stream a temporary data between each logical node of the execution graph, thereby reducing a size of said recirculating ring buffer;
wherein said recirculating ring buffer is sufficiently reduced in size to reside in an on-chip cache, and thereby, any one of the control information, input, and, or out data between node and, or nodes need not be stored in memory;
wherein the control information is capable of executing a command related to invalidating any one of the input and, or out data held in the recirculating ring data buffer, clearing the recirculating buffer of tasked data;
wherein the invalidation of any one of the data held in the recirculating ring buffer is by a command held in a command buffer; and
a data producer, wherein the data producer is configured to stall from writing control information into the command buffer upon the command buffer being full, preventing command buffer over-writing, and thereby reducing compute resources associated with a DRAM memory transaction.
2. The machine of claim 1 , wherein the control information further comprises any of, or combination of, an execution command and, or a command related to synchronization between the logical nodes.
3. The machine of claim 1 , wherein the invalidation of any one of the data held in the recirculating ring buffer is by a command held in the command buffer and, or program.
4. The machine of claim 3 , wherein the invalidation of any one of the data held in the recirculating ring buffer is based on completion of processing a graph task.
5. The machine of claim 1 , wherein the recirculating ring buffer residing in the on-chip cache are data buffers holding input data and, or out data.
6. The machine of claim 5 , wherein the input data and, or out data is generated by a producer and read by a consumer, whereby the producer and consumer are closer in a memory hierarchy due to the input data and, or out data held in the recirculating ring buffer residing in the on-chip cache.
7. The machine of claim 1 , wherein the recirculating ring buffer is a command buffer, whereby commands are written into the recirculating ring buffer by a producer and commands are read by a consumer.
8. The machine of claim 1 , wherein the writing of input and, or out data is by updating a write-only pointer in a context buffer by the producer; and the reading of input and, or out data is by processing a read-only pointer in a context buffer by the consumer.
9. The machine of claim 8 , wherein the writing of informational control data is by updating a write-only pointer in a context buffer by the producer; and the reading of informational control data by processing a read-only pointer in a context buffer by the consumer.
10. The machine of claim 8 , wherein the context buffer is prefetched.
11. The machine of claim 9 , wherein the context buffer is prefetched.
12. The machine of claim 1 , wherein the recirculating ring buffer utilizes addressing that wraps to the beginning of the recirculating ring buffer once the end has been reached.
13. The machine of claim 1 , wherein multiple graph task executions occur concurrently, thereby not requiring a sequential stage-stage and, or node at a time processing.
14. The machine of claim 1 , wherein the size of the recirculating ring buffer may vary.
15. A system encoded as a set of machine-readable instructions stored on non-transitory machine-readable media, the system to employ re-circulating ring buffers to reduce computing power, said system further comprising instructions for implementing:
at least a set of logical nodes, wherein said logical nodes are processed by a plurality of stages, connected together in an arbitrary topology;
at least one recirculating ring buffer residing in an on-chip cache, said at least one recirculating ring buffer further comprising any one of at least a control information, input and, or out data delivering temporary data between a stage and, or a logical node in an execution graph, such that any one of the control information, input, and, or out data between logical nodes need not be stored in memory;
an on-chip buffer management module, said on-chip buffer management module comprising a write pointer queue and a read pointer queue coupled via a cache gauge, thereby managing buffer content and traffic;
wherein the read pointer queue alerts a cache gauge of an impending completion of a graph task and, or a maximum capacity of any one the recirculating ring buffer and the cache gauge updates the write pointer queue; and
based on the write pointer queue updates, direct compute resources to prepare for a data producer stall and, or buffer clear.
16. A non-transitory, machine-readable media, for re-configuring a machine for increased efficiency in processing flow comprising steps of:
reducing compute resources via a plurality of logical nodes being processed on plurality of stages, with defined properties and resources associated with each logical node, for reducing compute resources;
reducing a size of a recirculating ring buffer by having the recirculating ring buffer hold only any one of a control information, input, and, or out data necessary to stream a temporary data between logical nodes in an execution graph;
residing said recirculating ring buffer in an on-chip cache, such that any one of the control information, input, and, or out data between nodes need not be stored in memory;
invalidating any one of the inputs and, or out data held in a recirculating ring data buffer, clearing the buffer of tasked data; and
stalling a data producer of the machine from writing any more control information into a recirculating ring command buffer upon the buffer being full, preventing command buffer over-writing, and thereby reducing compute resources associated with a DRAM memory transaction.