Network attached memory using selective resource migration
Maintaining cache coherency in the presence of a network attached memory is disclosed. A computer system includes a plurality of physical nodes. An operating system is run collectively across the plurality of physical nodes. The physical nodes are configured to communicate with a network attached memory. Based at least in part on an operation to be performed with respect to page on a first physical node included in the plurality of physical nodes, the network attached memory is configured to receive a message. The network attached memory is configured to perform an action based at least in part on the received message.
1. A computer system, comprising:
a plurality of physically interconnected computing nodes, wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the physically interconnected computing nodes;
a network-attached memory comprising a centralized appliance including an array of physical memory, wherein the network-attached memory communicates with at least some of the physically interconnected computing nodes over a network according to a coherency protocol;
wherein, in response to a remote read request made by a first computing node to a second computing node that has a primary copy of a page of memory, the second computing node, according to the coherency protocol:
transitions the copy of the page of memory on the second computing node from being designated as a primary copy to being designated as a secondary copy;
performs a first transmission of a copy of the page of memory to the first computing node, wherein the copy of the page of memory on the first computing node is marked as primary, and wherein the page of memory is subsequently requested from the first computing node based at least in part on the copy of the page of memory on the first computing node being marked as primary; and
performs a second transmission of a copy of the page of memory to the centralized network-attached memory, wherein the copy of the page of memory on the network-attached memory is marked as valid and secondary.
2. The computer system of claim 1 wherein portions of the physical memory in the network-attached memory correspond to guest physical addresses in the virtual environment that is defined by the set of hyper-kernels running on the at least portion of the physically interconnected computing nodes.
3. The computer system of claim 1 wherein based at least in part on detection of a failure, the network-attached memory requests copies of one or more pages from the computing nodes in the plurality of physically interconnected computing nodes.
4. The computer system of claim 3 wherein the one or more pages for which copies are requested comprise pages determined to be invalid on the network-attached memory.
5. The computer system of claim 4 wherein the network-attached memory updates an invalid page with a corresponding obtained copy of a requested page.
6. The computer system of claim 1 wherein the network-attached memory includes at least one of flash memory, phase change memory, and 3D-Xpoint memory.
7. The computer system of claim 1 wherein the network-attached memory stores a read-only copy of the page.
8. A method, comprising:
according to a coherency protocol, in response to a remote read request made by a first computing node in a plurality of physically interconnected computing nodes to a second computing node that has a primary copy of a page of memory:
transitioning, by the second computing node, the copy of the page of memory on the second computing node from being designated as a primary copy to being designated as a secondary copy;
performing a first transmission, from the second computing node, of a copy of the page of memory to the first computing node, wherein the copy of the page of memory on the first computing node is marked as primary, and wherein the page of memory is subsequently requested from the first computing node based at least in part on the copy of the page of memory on the first computing node being marked as primary; and
performing a second transmission, from the second computing node, of a copy of the page of memory to a centralized network-attached memory comprising a centralized appliance including an array of physical memory, wherein the copy of the page of memory on the network-attached memory is marked as valid and secondary; and
wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the physically interconnected computing nodes, and wherein the network-attached memory communicates with at least some of the physically interconnected computing nodes over a network according to the coherency protocol.
9. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
according to a coherency protocol, in response to a remote read request made by a first computing node in a plurality of physically interconnected computing nodes to a second computing node that has a primary copy of a page of memory:
transitioning, by the second computing node, the copy of the page of memory on the second computing node from being designated as a primary copy to being designated as a secondary copy;
performing a first transmission, from the second computing node, of a copy of the page of memory to the first computing node, wherein the copy of the page of memory on the first computing node is marked as primary, and wherein the page of memory is subsequently requested from the first computing node based at least in part on the copy of the page of memory on the first computing node being marked as primary; and
performing a second transmission, from the second computing node, of a copy of the page of memory to a centralized network-attached memory comprising a centralized appliance including an array of physical memory, wherein the copy of the page of memory on the network-attached memory is marked as valid and secondary; and
wherein a guest operating system is run on a virtual environment that is defined by a set of hyper-kernels running on at least a portion of the physically interconnected computing nodes, and wherein the network-attached memory communicates with at least some of the physically interconnected computing nodes over a network according to the coherency protocol.