IP Library Granted Patent US 8,095,617
Granted Patent B2
US 8,095,617 · App. 12/495,635 · Granted Jan 10, 2012

Caching data in a cluster computing system which avoids false-sharing conflicts

Assignee: Oracle America Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,095,617
App. No.
12/495,635
Granted
Jan 10, 2012
Kind
B2
Abstract

Managing operations in a first compute node of a multi-computer system. A remote write may be received to a first address of a remote compute node. A first data structure entry may be created in a data structure, which may include the first address and status information indicating that the remote write has been received. Upon determining that the local cache of the first compute node has been updated with the remote write, the remote write may be issued to the remote compute node. Accordingly, the first data structure entry may be released upon completion of the remote write.

Claims (85)

1. A method for managing operations in a first compute node of a multi-computer system, wherein the multi-computer system comprises a plurality of compute nodes coupled together by a network, the method comprising:

receiving a remote write to a first address of a remote compute node, wherein the remote write is issued by a first process executing on the first compute node;

creating a first data structure entry in a data structure, wherein the first data structure entry comprises the first address and status information indicating that the remote write has been received;

receiving a writeback operation to the first address prior to a local cache of the first compute node being updated with the remote write, wherein the writeback operation comprises writeback data;

maintaining at least one first memory map which represents prefetchable, cacheable remote memory;

maintaining at least one second memory map which represents uncached remote memory;

determining if the first address of the writeback operation is in the first memory map, wherein the writeback operation is not propagated onto the network if the first address of the writeback operation is determined to be in the first memory map;

updating the status information in the first data structure entry to indicate that the writeback operation has been received for the first address;

saving the writeback data in the first data structure entry;

determining that the local cache of the first compute node has been updated with the remote write;

issuing the remote write to the remote compute node in response to determining that the local cache of the first compute node has been updated with the remote write;

releasing the first data structure entry upon completion of the remote write.

2. The method of claim 1 ,

wherein said issuing the remote write to the remote compute node after determining that the local cache of the first compute node has been updated with the remote write inhibits a remote CPU of the remote compute node from seeing the remote write before the remote write is visible to a local CPU of the first compute node.

3. The method of claim 1 ,

wherein said determining that the local cache of the first compute node has been updated with the remote write comprises determining that a write has occurred to a physical memory address with a predefined association with the first address.

4. The method of claim 1 , further comprising:

receiving a remote write to the first address from a second process executing on the first compute node;

creating a second data structure entry in the data structure, wherein the second data structure entry comprises the first address and status information indicating that the remote write has been received for the first address.

5. The method of claim 4 , further comprising:

in response to receiving the writeback operation, updating the status information in the second data structure entry to indicate that the writeback operation has been received; and

saving the writeback data of the writeback operation in the second data structure entry.

6. The method of claim 1 , further comprising:

receiving a read of the first address;

determining that at least one data structure entry exists for the first address;

determining that the status information indicates that the writeback operation has been received for the first address;

returning the writeback data of the writeback operation from the at least one data structure entry in response to the read of the first address.

7. The method of claim 1 , further comprising:

receiving a read of the first address prior to receiving the writeback operation;

determining that at least one data structure entry exists for the first address;

determining that the status information indicates that a writeback operation has not been received for the first address;

performing a remote fetch of the first address; and

returning data from the remote fetch in response to the read of the first address.

8. A system comprising a plurality of compute nodes coupled together by a network, wherein each compute node comprises a processor, one or more memory mediums, and a network adapter, wherein the system comprises:

a first compute node; and

a second compute node, wherein the first compute node stores information at a first address in the one or more memory mediums; and

wherein the first compute node is configured to:

receive a remote write to the first address of the second compute node, wherein the remote write is issued by a first process executing on the first compute node;

create a first data structure entry in a data structure, wherein the first data structure entry comprises the first address and status information indicating that the remote write has been received;

receive a writeback operation to the first address prior to a local cache of the first compute node being updated with the remote write, wherein the writeback operation comprises writeback data

maintain at least one first memory map which represents prefetchable, cacheable remote memory;

maintain at least one second memory map which represents uncached remote memory;

determine if the first address of the writeback operation is in the first memory map, wherein the writeback operation is not propagated onto the network if the first address of the writeback operation is determined to be in the first memory map;

update the status information in the first data structure entry to indicate that the writeback operation has been received for the first address;

save the writeback data in the first data structure entry;

determine that the local cache of the first compute node has been updated with the remote write;

issue the remote write to the second compute node in response to determining that the local cache of the first compute node has been updated with the remote write; and

release the first data structure entry upon completion of the remote write.

9. The system of claim 8 ,

wherein said issue of the remote write to the second compute node after determining that the local cache of the first compute node has been updated with the remote write inhibits a remote CPU of the second compute node from seeing the remote write before the remote write is visible to a local CPU of the first compute node.

10. The system of claim 8 ,

wherein said determination that the local cache of the first compute node has been updated with the remote write comprises determining that a write has occurred to a physical memory address with a predefined association with the first address.

11. The system of claim 8 , wherein the first compute node is further configured to:

receive a remote write to the first address from a second process executing on the first compute node;

create a second data structure entry in the data structure, wherein the second data structure entry comprises the first address and status information indicating that the remote write has been received for the first address.

12. The system of claim 11 , wherein the first compute node is further configured to:

in response to receiving the writeback operation, update the status information in the second data structure entry to indicate that the writeback operation has been received; and

save the writeback data of the writeback operation in the second data structure entry.

13. The system of claim 8 , wherein the first compute node is further configured to:

receive a read of the first address;

determine that at least one data structure entry exists for the first address;

determine that the status information indicates that the writeback operation has been received for the first address; and

return the writeback data of the writeback operation from the at least one data structure entry in response to the read of the first address.

14. The system of claim 8 , wherein the first compute node is further configured to:

receive a read of the first address prior to receiving the writeback operation;

determine that at least one data structure entry exists for the first address;

determine that the status information indicates that a writeback operation has not been received for the first address;

perform a remote fetch of the first address; and

return data from the remote fetch in response to the read of the first address.

15. A compute node configured to couple to a network, wherein the compute node comprises:

a processor;

one or more memory mediums;

a network adapter;

wherein the compute node is configured to:

receive a remote write to a first address of a second compute node, wherein the remote write is issued by a first process executing on the compute node;

create a first data structure entry in a data structure, wherein the first data structure entry comprises the first address and status information indicating that the remote write has been received;

receive a writeback operation to the first address prior to a local cache of the first compute node being updated with the remote write, wherein the writeback operation comprises writeback data

maintain at least one first memory map which represents prefetchable, cacheable remote memory;

maintain at least one second memory map which represents uncached remote memory;

determine if the first address of the writeback operation is in the first memory map, wherein the writeback operation is not propagated onto the network if the first address of the writeback operation is determined to be in the first memory map;

update the status information in the first data structure entry to indicate that the writeback operation has been received for the first address;

save the writeback data in the first data structure entry;

determine that the local cache of the compute node has been updated with the remote write;

issue the remote write to the second compute node in response to determining that the local cache of the compute node has been updated with the remote write; and

release the first data structure entry upon completion of the remote write.

Assignments (2)
MERGER AND CHANGE OF NAME Recorded Dec 16, 2015
From: ORACLE USA, INC.; SUN MICROSYSTEMS, INC.; ORACLE AMERICA, INC.
To: ORACLE AMERICA, INC.
Reel/Frame 037311/0134 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 17, 2009
From: JOHNSEN, BJORN DAG; SUGUMAR, RABIN A.; SUM, BEN; HUSE, LARS PAUL
To: SUN MICROSYSTEMS, INC.
Reel/Frame 023247/0222 →
Continuity (1)
Related Publication 20100332612A1 · Dec 30, 2010