System and method for providing cache-aware lightweight producer consumer queues
View Patent ↗A multiprocessor system or a system of hardware accelerators is provided to reduce cache ping-ponging and to provide improved single producer single consumer (SPSC) queues and methods. The systems are configured for specifying separate cache attributes for inner (e.g., local) cache and outer (e.g., shared) cache for promoting lower system overhead. Separate cache attributes are specified such that shared variables are cacheable only in a cache level shared by multiple processors.
1. A method of operation of a multiprocessor system, the method comprising:
determining via a first processor of the system if advancing a first variable of a buffer from a first location in the buffer to a second location in the buffer would make the first variable equal to a second variable, the first variable and the second variable being processor local variables;
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the second variable, performing steps of:
storing requested data to the buffer at a pointer specified by the first variable; and
advancing the first variable to the second location in the buffer; and
when determining indicates that advancing the first variable from the first location to the second location would make the first variable equal to the second variable, performing steps of:
determining whether advancing the first variable from the first location to the second location would make the first variable equal to a shared variable; and
storing the shared variable to the second variable when advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable.
2. The method as claimed in claim 1 , wherein the first processor is a producer processor and a second processor is a consumer processor having access to the buffer concurrent with the producer processor, wherein the first variable includes a memory pointer or an index and the second variable includes a memory pointer or an index.
3. The method as claimed in claim 1 , further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the second variable, further performing a step of:
executing a data memory barrier instruction for allowing prior writes to memory of the system to be detected before a next instruction executes; and
storing the first variable to shared cache of the system to provide an indication to a second processor of the system that the first variable is available for consumption by the second processor, shared cache being shared by the first processor and the second processor.
4. The method as claimed in claim 1 , further comprising:
associating the memory pointer or the index for the first variable with a local buffer, associating the memory pointer or the index for the second variable with the local buffer, and associating a memory pointer or an index of the shared variables with a shared buffer.
5. The method as claimed in claim 2 , further comprising:
managing memory access such that variables written and not read by at least one of the first processor and the second processor are cacheable only in shared cache.
6. The method as claimed in claim 1 , further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable, further performing a step of:
storing requested data to the buffer at a pointer specified by the first variable.
7. The method as claimed in claim 6 , further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable, further performing a step of:
advancing the first variable to the second location in the buffer.
8. The method as claimed in claim 7 , further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable, further performing a step of:
executing a data memory barrier instruction.
9. The method as claimed in claim 8 , wherein the buffer is a first buffer and further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable, further performing a step of:
storing the first variable to shared cache of the system to provide an indication in a second buffer, the indication in the second buffer indicating to a second processor of the system that the first variable is available for consumption by the second processor.
10. The method as claimed in claim 1 , further comprising:
when determining indicates that advancing the first variable from the first location to the second location would make the first variable equal to the shared variable, further performing a step of:
stalling processing by the first processor until a time when advancing the first variable of the buffer from the first location to the second location would not make the first variable equal to the shared variable.
11. The method as claimed in claim 1 , wherein the shared variable is cacheable only in shared cache of the system, shared cache being shared by the first processor and a second processor of the system.
12. The method as claimed in claim 1 , wherein the processor local variables are not shared between the first processor and a second processor of the system.
13. A non-transitory computer-readable medium having computer-executable instructions for performing a method of operation of a multiprocessor system, the method comprising:
determining via a first processor of the system if advancing a first variable of a buffer from a first location in the buffer to a second location in the buffer would make the first variable equal to a second variable, the first variable and the second variable being processor local variables;
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the second variable, performing steps of:
storing requested data to the buffer at a pointer specified by the first variable;
advancing the first variable to the second location in the buffer; and
executing a data memory barrier instruction for allowing prior writes to memory of the system to be detected before a next instruction executes; and
when determining indicates that advancing the first variable from the first location to the second location would make the first variable equal to the second variable, performing steps of:
determining whether advancing the first variable from the first location to the second location would make the first variable equal to a shared variable; and
storing the shared variable to the second variable when advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable.
14. The non-transitory computer-readable medium as claimed in claim 13 , the method further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the second variable, further performing a step of:
storing the first variable to shared cache of the system to provide an indication to a second processor of the system that the first variable is available for consumption by the second processor, shared cache being shared by the first processor and the second processor.
15. The non-transitory computer-readable medium as claimed in claim 14 , wherein the pointer is a first pointer associated with local cache, and wherein storing the first variable to the shared cache of the system includes storing the first pointer specified by the first variable to a second pointer associated with the shared cache.
16. The non-transitory computer-readable medium as claimed in claim 13 , the method further comprising:
when determining indicates that advancing the first variable from the first location to the second location would not make the first variable equal to the shared variable, further performing steps of:
storing requested data to the buffer at a pointer specified by the first variable;
advancing the first variable to the second location in the buffer;
executing a data memory barrier instruction for allowing prior writes to memory of the system to be detected before a next instruction executes; and
storing the first variable to shared cache of the system to provide an indication to a second processor of the system that the first variable is available for consumption by the second processor.
17. The non-transitory computer-readable medium as claimed in claim 13 , the method further comprising:
when determining indicates that advancing the first variable from the first location to the second location would make the first variable equal to the shared variable, further performing a step of:
stalling processing by the first processor until a time when advancing the first variable of the buffer from the first location to the second location would not make the first variable equal to the shared variable.
18. The non-transitory computer-readable medium as claimed in claim 13 , wherein the shared variable is cacheable only in shared cache of the system, shared cache being shared by the first processor and a second processor of the system.
19. The non-transitory computer-readable medium as claimed in claim 13 , wherein the processor local variables are not shared between the first processor and a second processor of the system.
20. A multiprocessor system, comprising:
a first processor, the first processor including local cache accessible only by the first processor;
a second processor communicatively couple with the first processor, the second processor including local cache accessible only by the second processor;
a memory, the memory being communicatively coupled with the first processor and the second processor;
a shared cache, the shared cache configured for being accessible by the first processor and the second processor,
wherein the system is a distributed processing system and is configured for allocating a first copy of a single producer single consumer queue and a copy of shared variables to the shared cache and at least a second copy of the single producer single consumer queue and a copy of local variables respectively to the first local cache and the second local cache when the memory is first accessed by the processors, the shared variables being control variables used by the processors to signal each other that data is available in the first copy of the single producer single consumer queue and being cacheable only in the shared cache, and the local variables being control variables used by the processors to respectively point to a location in the at least the second copy of the single producer single consumer queue and are cacheable only in the respective local cache; and
wherein the system is further configured for determining via at least one of the first processor or the second processor of the system if a first variable of a buffer is equal to a second variable of the buffer, the first variable and the second variable being processor local control variables;
when determining indicates that the first variable is not equal to the second variable, performing steps of:
reading requested data from the buffer at a pointer specified by the second variable; and
advancing the second variable to a second location in the buffer; and
when determining indicates that the first variable would be equal to the second variable, performing steps of:
determining whether the second variable would be equal to a shared variable; and
storing the shared variable to the first variable when the second variable would not be equal to the shared variable.