Trace cache that supports multiple different trace lengths
Disclosed techniques relate to storing wide traces in a trace cache. In some embodiments, prediction circuitry predicts directions of control transfer instructions. Instruction cache circuitry may cache instructions sequentially in cache lines that store up to a first number of instructions. Trace cache circuitry may identify traces of instruction, store identified traces having up to the first number of instructions in first trace storage circuitry, and store identified traces having up to a second, greater number of instructions in second trace storage circuitry. This may advantageously allow longer traces to be executed from the second trace storage circuitry (without extensive changes to the instruction cache and processor front end, in some embodiments, due to the limited trace length for the first trace storage circuitry).
1 . An apparatus, comprising:
processor circuitry configured to execute control transfer instructions;
prediction circuitry configured to predict directions of control transfer instructions;
instruction cache circuitry configured to cache instructions sequentially in cache lines that store up to a first number of instructions;
trace cache circuitry configured to:
identify traces of instructions that satisfy one or more criteria, wherein a given trace of the identified traces includes at least one internal taken control transfer instruction;
store first one or more of the identified traces having up to the first number of instructions in first trace storage circuitry, wherein the first trace storage circuitry is at a first level in a cache hierarchy; and
store second one or more of the identified traces having up to a second number of instructions, greater than the first number of instructions, in second trace storage circuitry, wherein the second trace storage circuitry is at a second level in the cache hierarchy that is further from the processor circuitry than the first level;
prefetch circuitry configured to:
predict that a trace will be executed by the processor circuitry; and
prefetch the predicted trace from the second trace storage circuitry to the first trace storage circuitry based on the prediction; and
prefetch buffer circuitry configured to store the prefetched predicted trace from the second trace storage circuitry and provide the prefetched predicted trace for execution, wherein the prefetch buffer circuitry supports traces having up to the second number of instructions.
2 . The apparatus of claim 1 , wherein the trace cache circuitry is configured to promote entries from the prefetch buffer circuitry, having up to the first number of instructions, to the first trace storage circuitry and is configured not to promote entries from the prefetch buffer circuitry having more than the first number of instructions.
3 . The apparatus of claim 1 , wherein:
the second trace storage circuitry includes:
overflow buffer circuitry; and
entries configured to store up to the first number of instructions and a pointer to an entry of the overflow buffer circuitry to store remaining instructions for a trace having more than the first number of instructions; and
the prefetch buffer circuitry includes entries configured to store up to the second number of instructions.
4 . The apparatus of claim 1 , wherein the second trace storage circuitry includes:
overflow buffer circuitry; and
entries configured to store up to the first number of instructions and information that indicates an entry of the overflow buffer circuitry to store remaining instructions for a trace having more than the first number of instructions.
5 . The apparatus of claim 1 , further comprising:
pipeline instruction buffer circuitry configured to buffer instructions from the instruction cache circuitry and instructions from the trace cache circuitry, including to store a trace from the second trace storage circuitry in multiple entries of the pipeline instruction buffer circuitry and store a trace from the first trace storage circuitry in a single entry of the pipeline instruction buffer circuitry.
6 . The apparatus of claim 1 , further comprising:
sequential fetch control circuitry configured to:
store a set of sequential instructions having up to the second number of instructions in the second trace storage circuitry, including up to a threshold number of predicted-not-taken control transfer instructions; and
fetch the set of sequential instructions from the second trace storage circuitry for execution in response to a hit.
7 . The apparatus of claim 6 , wherein the set of sequential instructions includes up to the second number of stable not-taken control transfer instructions and up to the threshold number of unstable predicted-not-taken control transfer instructions.
8 . The apparatus of claim 6 , further comprising usefulness control circuitry configured to:
track use of cached traces and sets of sequential instructions in the second trace storage circuitry; and
select entries for eviction based on the tracked use.
9 . The apparatus of claim 6 , wherein:
for the set of sequential instructions stored in the second trace storage circuitry, the prediction circuitry is configured to maintain prediction information, wherein the prediction information encodes one of multiple states, including
two or more states that indicate which control transfer instruction was taken among control transfer instructions in the set of sequential instructions; and
a state that indicates that none of the control transfer instructions in the set of sequential instructions was taken.
10 . The apparatus of claim 9 , wherein the prediction information includes at least two states that encode the same predicted directions for control transfer instructions in the set of sequential instructions, wherein different states of the at least two states correspond to different prediction confidence levels.
11 . A method, comprising:
predicting, by a computing system, directions of control transfer instructions executed by a processor of the computing system;
caching, by the computing system in an instruction cache, sequences of instructions in instruction cache entries that store up to a first number of instructions;
identifying, by the computing system, traces of instructions that satisfy one or more criteria, wherein one or more of the identified traces include at least one internal taken control transfer instruction;
storing, by the computing system, first one or more identified traces having up to the first number of instructions in first trace storage circuitry, wherein the first trace storage circuitry is at a first level in a cache hierarchy;
storing, by the computing system, second one or more identified traces having up to a second number of instructions, greater than the first number of instructions, in second trace storage circuitry, wherein the second trace storage circuitry is at a second level in the cache hierarchy that is further from the processor than the first level;
predicting, by the computing system, that a first trace having the first number of instructions will be executed by the processor and that a second trace having the second number of instructions will be executed by the processor;
prefetching, by the computing system, the first trace from the second trace storage circuitry to the first trace storage circuitry and providing the first trace from the first trace storage circuitry for execution; and
prefetching, by the computing system, the second trace from the second trace storage circuitry to a prefetch buffer and providing the second trace from the prefetch buffer for execution, wherein the prefetch buffer supports traces having up to the second number of instructions.
12 . The method of claim 11 , further comprising:
storing, by the computing system, a first portion of the second trace in an entry of the second trace storage circuitry;
storing, by the computing system, a pointer to an overflow buffer entry in the entry of the second trace storage circuitry; and
storing, by the computing system, a second portion of the second trace in the overflow buffer entry.
13 . The method of claim 11 , further comprising:
buffering, by the computing system, instructions from an instruction cache entry in a single entry of a buffer;
buffering, by the computing system, a trace from the first trace storage circuitry in a single entry of the buffer; and
buffering, by the computing system, a trace from the second trace storage circuitry in multiple entries of the buffer.
14 . The method of claim 11 , further comprising:
storing, by the computing system, a set of sequential instructions having up to the second number of instructions in the second trace storage circuitry, including up to a threshold number of predicted-not-taken control transfer instructions; and
fetching, by the computing system, the set of sequential instructions from the second trace storage circuitry for execution in response to a hit.
15 . The method of claim 14 , wherein the set of sequential instructions includes up to the second number of stable not-taken control transfer instructions and up to the threshold number of unstable predicted-not-taken control transfer instructions.
16 . The method of claim 14 , further comprising:
for the set of sequential instructions stored in the second trace storage circuitry, the computing system maintaining prediction information, wherein the prediction information encodes one of multiple states, including:
two or more states that indicate which control transfer instruction was taken among control transfer instructions in the set of sequential instructions; and
a state that indicates that none of the control transfer instructions in the set of sequential instructions was taken.
17 . A non-transitory computer-readable medium having instructions of a hardware description programming language stored thereon that, when processed by a computing system, program the computing system to generate a computer simulation model, wherein the model represents a hardware circuit that includes:
processor circuitry configured to execute control transfer instructions;
prediction circuitry configured to predict directions of control transfer instructions;
instruction cache circuitry configured to cache instructions sequentially in cache lines that store up to a first number of instructions;
trace cache circuitry configured to:
identify traces of instructions that satisfy one or more criteria, wherein a given trace of the identified traces includes at least one internal taken control transfer instruction;
store first one or more of the identified traces having up to the first number of instructions in first trace storage circuitry, wherein the first trace storage circuitry is at a first level in a cache hierarchy; and
store second one or more of the identified traces having up to a second number of instructions, greater than the first number of instructions, in second trace storage circuitry, wherein the second trace storage circuitry is at a second level in the cache hierarchy that is further from the processor circuitry than the first level;
prefetch circuitry configured to:
predict that a trace will be executed by the processor circuitry; and
prefetch the predicted trace from the second trace storage circuitry to the first trace storage circuitry based on the prediction; and
prefetch buffer circuitry configured to store the prefetched predicted trace from the second trace storage circuitry and provide the prefetched predicted trace for execution, wherein the prefetch buffer circuitry supports traces having up to the second number of instructions.
18 . The non-transitory computer-readable medium of claim 17 , wherein the hardware circuit represented by the model further includes:
sequential fetch control circuitry configured to:
store a set of sequential instructions having up to the second number of instructions in the second trace storage circuitry, including up to a threshold number of predicted-not-taken control transfer instructions; and
fetch the set of sequential instructions from the second trace storage circuitry for execution in response to a hit.
19 . An apparatus, comprising:
processor circuitry configured to execute control transfer instructions;
prediction circuitry configured to predict directions of control transfer instructions;
instruction cache circuitry configured to cache instructions sequentially in cache lines that store up to a first number of instructions;
trace cache circuitry configured to:
identify traces of instructions that satisfy one or more criteria, wherein a given trace of the identified traces includes at least one internal taken control transfer instruction;
store first one or more of the identified traces having up to the first number of instructions in first trace storage circuitry; and
store second one or more of the identified traces having up to a second number of instructions, greater than the first number of instructions, in second trace storage circuitry;
sequential fetch control circuitry configured to:
store a set of sequential instructions having up to the second number of instructions in the second trace storage circuitry, including up to a threshold number of predicted-not-taken control transfer instructions; and
fetch the set of sequential instructions from the second trace storage circuitry for execution in response to a hit.