Decoupling atomicity from operation size
In an embodiment, a processor implements a different atomicity size (for memory consistency order) than the operation size. More particularly, the processor may implement a smaller atomicity size than the operation size. For example, for multiple register loads, the atomicity size may be the register size. In another example, the vector element size may be the atomicity size for vector load instructions. In yet another example, multiple contiguous vector elements, but fewer than all the vector elements in a vector register, may be the atomicity size for vector load instructions.
1 . A processor, comprising:
a decoder circuit configured to decode a load instruction into a load memory operation; and
a load/store unit configured to:
execute the load memory operation having an operation size that is larger than an atomicity size implemented for compliance with a memory ordering model;
verify, for compliance with the memory ordering model, that each of a plurality of atomic elements associated with the load memory operation is fully sourced from either a store memory operation in a store queue or from a different source, wherein the plurality of atomic elements are of the atomicity size; and
permit forwarding from the store queue responsive to a verification that each of the plurality of atomic elements is fully sourced from either a store memory operation in the store queue or from a different source.
2 . The processor of claim 1 , wherein the load/store unit is configured to:
prevent completion of the load memory operation responsive to a detection that:
at least one of the plurality of atomic elements is not fully sourced from a single source; and
at least one of the plurality of atomic elements is sourced from the store queue.
3 . The processor of claim 2 , wherein the load/store unit is configured to prevent the completion of the load memory operation until one or more store memory operations targeted by the load memory operation have drained from the store queue.
4 . The processor of claim 1 , wherein the load memory operation targets a plurality of registers, wherein the atomicity size is a size of a register, and wherein a given atomic element comprises bytes written into a given register of the plurality of registers.
5 . The processor of claim 1 , wherein the load memory operation is a vector load memory operation, and the atomicity size is a size of a vector element in a vector that is read by the load memory operation.
6 . The processor of claim 1 , wherein at least one of the plurality of atomic elements is fully sourced from a store memory operation in the store queue and at least one of the plurality of atomic elements is fully sourced from the different source.
7 . The processor of claim 1 , further comprising a data cache that corresponds to the different source.
8 . The processor of claim 1 , wherein the load/store unit is configured to ensure ordering of the plurality of atomic elements responsive to the load memory operation receiving bytes from a plurality of cache lines.
9 . The processor of claim 1 , further comprising:
a reorder buffer, wherein the load memory operation is one of a plurality of load memory operations corresponding to the load instruction, wherein the load/store unit is configured to signal a flush of the load memory operation if an ordering violation is detected, and wherein the reorder buffer is configured to enforce in order execution of the plurality of load memory operations responsive to detecting that the load memory operation is within a threshold number of entries of a head of the reorder buffer when the flush is signaled.
10 . A load/store unit, comprising:
a store queue configured to queue one or more store memory operations that write data to one or more memory locations; and
an execution circuit coupled to the store queue and configured to:
execute a load memory operation having an operation size that is larger than an atomicity size implemented for compliance with a memory ordering model;
verify, for compliance with the memory ordering model, that each of a plurality of atomic elements associated with the load memory operation is fully sourced from either a store memory operation in the store queue or from a different source, wherein the plurality of atomic elements are of the atomicity size; and
permit forwarding of data from a particular store memory operation in the store queue for the load memory operation responsive to a verification that each of the plurality of atomic elements is fully sourced from either a store memory operation in the store queue or from a different source.
11 . The load/store unit of claim 10 , wherein the execution circuit is configured to, in response to a detection that at least one of the plurality of atomic elements is not fully sourced from either a store memory operation in the store queue or from a different source, stall the load memory operation until the store queue has emptied.
12 . The load/store unit of claim 10 , wherein the load memory operation targets a plurality of registers, and the atomicity size is a size of a register.
13 . The load/store unit of claim 10 , wherein the load memory operation is a vector load memory operation, and the atomicity size is based on a vector element size of a vector element in a vector that is read by the load memory operation.
14 . The load/store unit of claim 13 , wherein the atomicity size is the vector element size, and a given atomic element of the plurality of atomic elements corresponds to a vector element in the vector.
15 . The load/store unit of claim 13 , wherein the atomicity size is a multiple of the vector element size, and a given atomic element of the plurality of atomic elements corresponds to a plurality of adjacent vector elements in the vector.
16 . The load/store unit of claim 10 , wherein the operation size is an integer multiple of the atomicity size.
17 . A method, comprising:
executing, by a processor, a load memory operation, wherein the load memory operation has an operation size that is larger than an atomicity size implemented for compliance with a memory ordering model, and wherein the load memory operation is one of a plurality of load memory operations corresponding to a load instruction;
detecting, by the processor, an ordering violation based on a plurality of atomic elements associated with the load memory operation;
in response to the detecting, the processor signaling a flush of the load memory operation; and
enforcing, by the processor, in order execution of the plurality of load memory operations responsive to detecting that the load memory operation is within a threshold number of entries of a head of a reorder buffer of the processor when the flush is signaled.
18 . The method of claim 17 , wherein the operation size and an address accessed by the load memory operation cause a crossing of a cache line boundary between a plurality of cache lines, and wherein the ordering violation is detected in response to the plurality of atomic elements receiving bytes from the plurality of cache lines out of order.
19 . The method of claim 17 , wherein the enforcing includes:
tagging, by the processor, the plurality of load memory operations with an in-order-load indication to force the plurality of load memory operations to execute in order.
20 . The method of claim 17 , wherein the threshold number is programmable in the reorder buffer.