Apparatus and method with cache control
A computing apparatus is provided. The computing apparatus is configured to receive control information from a host device to control a cache area, generate a cache configuration based on the received control information, determine a first cache area and a second cache area in a memory in the computing apparatus based on the generated cache configuration, cache one or more instructions to the first cache area and cache data to the second cache area, and process a thread based on the one or more cached instructions and the cached data.
1. A method implemented by a processor, the method comprising:
receiving control information for controlling sizing of a cache area from a host device, wherein the control information comprises a data size and an instruction size corresponding to a thread as determined by a compiler while compiling code;
generating a cache configuration based on the received control information;
setting a size of an instruction cache area and a size of a data cache area in a memory in the processor based on the generated cache configuration;
caching some of the instructions of the thread stored in an external memory of the processor to the instruction cache area and caching data of the thread stored in the external memory to the data cache area;
monitoring usage, by execution of the thread, of the cached instructions and the cached data;
generating a cache performance profile based on a result of the monitoring;
modifying the generated cache configuration based on the generated cache performance profile; and
based on the modified cache configuration, changing the size of the instruction cache area and the size of the data cache area.
2. The method of claim 1 , wherein the generating of the cache configuration is performed upon execution of the thread.
3. The method of claim 2 , wherein the monitoring the usage of the cached instructions and the cached data comprises monitoring stalls.
4. The method of claim 1 , wherein the monitoring comprises determining a total number of first stalls due to cache misses of the instruction cache and determining a total number of second stalls due to cache misses, and
wherein the modifying of the generated cache configuration comprises modifying the size of the instruction cache area and modifying the size of the data cache area such that a total number of the first stalls and a total number of second stalls are balanced.
5. The method of claim 4 , wherein the modifying of the size of the instruction cache area and the modifying of the size of the data cache area comprises:
increasing the size of the instruction cache area and decreasing the size of the data cache area in response to the total number of first stalls being greater than the total number of second stalls; and
increasing the size of the data cache area size and decreasing the size of the instruction cache area in response to the total number of second stalls being greater than the total number of first stalls.
6. The method of claim 1 , further comprising:
determining an optimized ratio between the size of instruction cache area and the size of the data cache area based on the generated cache performance profile; and
setting the size of the instruction cache area and the size of the data cache area according to the determined optimized ratio.
7. The method of claim 1 , wherein the received instruction size and data size are inferred from an input given by a neural network model in the host device.
8. The method of claim 1 , wherein the memory comprises static random-access memory (SRAM).
9. A computing apparatus, comprising:
a memory;
a cache controller configured to: receive control information for controlling a cache area from a host device, generate a cache configuration based on the control information, and determine a size of an instruction cache area and a size of a data cache area in the memory based on the generated cache configuration, wherein the control information comprises a data size and an instruction size corresponding to a thread as determined by a compiler while compiling code;
a fetcher configured to fetch instructions cached in the instruction cache area and decode the fetched instructions;
an arithmetic logic unit configured to perform operations on the cached data in the data cache area and process the thread based on the decoded instructions; and
a profiler configured to generate a cache performance profile based on a result of monitoring performance of the instruction cache area and the data cache area with respect to the operations of processing the thread, the cache performance profile indicating efficiency of use of the instruction cache area and the data cache area,
wherein the cache controller is configured to modify the generated cache configuration based on the generated cache performance profile, and, according to the modified cache configuration, update the size of the instruction cache area and the size of the data cache area.
10. The apparatus of claim 9 , wherein the cache controller is configured to modify the size of the instruction cache area and the size of the data cache area such that a total number of first stalls due to cache misses of the instruction cache area and a total number of second stalls due to cache misses of the data cache area are balanced.
11. The apparatus of claim 10 , wherein the cache controller is configured to:
increase the size of the instruction cache area and decrease the size of the data cache area in response to the total number of first stalls being greater than the total number of second stalls, and increase the size the data cache area and decrease size of the instruction cache area in response to the total number of second stalls being greater than the total number of first stalls.
12. The apparatus of claim 9 , wherein the cache controller is configured to determine an optimized ratio between the size of the instruction cache area and the size of the data cache area based on the generated cache performance profile and apply the determined optimized ratio to the instruction cache area and the data cache area.
13. The apparatus of claim 9 , wherein the instruction size and the data size are inferred from an input given by a neural network model in the host device.
14. The apparatus of claim 9 , wherein the memory comprises static random-access memory (SRAM).
15. An electronic apparatus comprising:
a plurality of processors, each processor comprising a respective memory; and
wherein each of the processors is configured to:
receive control information for controlling a cache area from a host device, the control information comprising a data size and an instruction size corresponding to a thread, the thread being determined by a compiler while compiling code,
generate a cache configuration based on the control information,
determine a size of an instruction cache area and a size of a data cache area in the memory of the corresponding processor based on the generated cache configuration,
cache instructions of the thread to the instruction cache area and cache data of the thread to the data cache area,
process the thread based on the one or more cached instructions and the cached data,
monitor usage of the cached instructions and the cached data;
determine a performance of the instruction cache area and the data cache area based on a result of the monitoring, the performance corresponding to the processing of the thread, and
based on the performance of the instruction cache area and the data cache area, update the cache configuration, and update the size of the instruction cache area and the size of the data cache area based on the updated cache configuration.