Efficient work unit processing in a multicore system
Techniques are described in which a system having multiple processing units processes a series of work units in a processing pipeline, where some or all of the work units access or manipulate data stored in non-coherent memory. In one example, this disclosure describes a method that includes identifying, prior to completing processing of a first work unit with a processing unit of a processor having multiple processing units, a second work unit that is expected to be processed by the processing unit after the first work unit. The method also includes processing the first work unit, and prefetching, from non-coherent memory, data associated with the second work unit into a second cache segment of the buffer cache, wherein prefetching the data associated with the second work unit occurs concurrently with at least a portion of the processing of the first work unit by the processing unit.
1. A system comprising:
processing circuitry having a cache, wherein the processing circuitry is configured to process a first stream fragment and generate first stream data in a first cache segment in the cache;
a buffer to store data; and
a load store unit configured to:
determine that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment,
prefetch data associated with the second stream fragment into a second segment of the cache, wherein at least some of the prefetching occurs before the processing circuitry finishes processing the first stream fragment, and
flush the first cache segment of the cache after the processing circuitry finishes processing the first stream fragment, wherein flushing the first cache segment includes storing the first stream data in the buffer.
2. The system of claim 1 , wherein the load store unit is further configured to:
generate a message indicating that the first stream data can be accessed from the buffer.
3. The system of claim 2 , wherein the processing circuitry is a first processing unit, wherein generating the message indicating that the first stream data can be accessed from the buffer occurs before the processing circuitry finishes processing the first stream segment, and wherein the load store unit is further configured to:
deliver the message to a second processing unit, wherein delivering the message is gated by completion of flushing the first cache segment.
4. The system of claim 1 , wherein to determine that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment, the load store unit is further configured to:
determine, based on information stored in a queue, that the second stream fragment is expected to be processed by the processing circuitry after the first stream fragment.
5. The system of claim 1 , wherein to prefetch data associated with the second stream fragment into the second segment of the cache, the load store unit is further configured to:
prefetch data associated with the second stream fragment concurrently with processing the first stream fragment.
6. The system of claim 1 , wherein the processing circuitry is further configured to:
process the second stream fragment by accessing the data associated with the second stream fragment from the second segment of the cache.
7. The system of claim 6 , wherein to process the second stream fragment, the processing circuitry is further configured to:
generate second stream data in the second segment of the cache.
8. The system of claim 7 , wherein the load store unit is further configured to:
determine that a third stream fragment is expected to be processed by the processing circuitry after the second stream fragment;
prefetch data associated with the third stream fragment into the first cache segment of the cache, wherein at least some of the prefetching occurs before the processing circuitry finishes processing the second stream fragment;
flush the second cache segment of the cache after the processing circuitry finishes processing the second stream fragment, wherein flushing the second cache segment includes storing the second stream data in the buffer; and
generating a message indicating that the second stream data can be accessed in the buffer.
9. The system of claim 1 , wherein the processing circuitry is a processor within a multi-processor computing system.
10. The system of claim 1 , wherein the processing circuitry is a processing core within a multi-core processor.
11. The system of claim 3 , wherein the buffer is non-coherent memory, and wherein delivering the message indicating that the first stream data can be accessed from the buffer transfers ownership of at least a portion of the non-coherent memory.
12. The system of claim 1 , wherein to prefetch data associated with the second stream fragment includes masking invalid addresses.
13. A method comprising:
processing, by processing circuitry having a cache, a first stream fragment;
generating first stream data, by the processing circuitry, when processing the first stream fragment;
storing the first stream data in a first cache segment of the cache;
determining that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment;
prefetching data associated with the second stream fragment into a second segment of the cache, wherein at least some of the prefetching of the data associated with the second stream fragment occurs before the processing circuitry finishes processing the first stream fragment; and
flushing the first cache segment after the processing circuitry finishes processing the first stream fragment, wherein flushing the first cache segment includes storing the first stream data in the buffer.
14. The method of claim 13 , further comprising:
generating a message indicating that the first stream data is stored in the buffer.
15. The method of claim 13 , wherein the processing circuitry is a first processing unit, wherein generating the message indicating that the first stream data can be accessed from the buffer occurs before the processing circuitry finishes processing the first stream segment, the method further comprising:
delivering the message to a second processing unit, wherein delivering the message is gated by completion of flushing the first cache segment.
16. The method of claim 13 , wherein determining that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment includes:
determining, based on information stored in a queue, that the second stream fragment is expected to be processed by the processing circuitry after the first stream fragment.
17. The method of claim 13 , wherein determining that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment includes:
determining, prior to completing processing of the first stream fragment by the processing circuitry, that the second stream fragment is expected to be processed by the processing circuitry after the first stream fragment.
18. The method of claim 13 , wherein prefetching data associated with the second stream fragment into the second segment of the cache includes:
prefetch data associated with the second stream fragment concurrently with processing the first stream fragment.
19. The method of claim 13 , further comprising:
processing the second stream fragment by accessing the data associated with the second stream fragment from the second segment of the cache.
20. A computing system having a cache, an instruction storage system, and a processing circuitry, wherein the processing circuitry is configured to: process a first stream fragment; generate first stream data when processing the first stream fragment; store the first stream data in a first cache segment of the cache; determine that a second stream fragment is expected to be processed by the processing circuitry after the first stream fragment; prefetch data associated with the second stream fragment into a second segment of the cache, wherein at least some of the prefetching of the data associated with the second stream fragment occurs before the processing circuitry finishes processing the first stream fragment; and flush the first cache segment after the processing circuitry finishes processing the first stream fragment, wherein flushing the first cache segment includes storing the first stream data in the buffer.