Sharing data fabric for coherent-distributed caching of multi-node shared-distributed flash memory

1. A global shared flash-memory system, comprising:

a plurality of compute nodes, wherein each compute node of the plurality of compute nodes is a distinct physical machine and comprises:

a flash memory for storing homed objects in non-volatile solid-state memory cells;

a processor for executing an application program;

a volatile memory, coupled to the processor, for storing cached objects, wherein the cached objects include copies of remote homed objects stored at remote compute nodes in the plurality of compute nodes, wherein each homed object, including the homed objects stored in the flash memory and the remote homed objects stored at remote compute nodes, has a respective single home node;

a network interface controller (NIC), coupled by a network to other compute nodes, and responsive to the processor, that sends local copies of the homed objects stored in the flash memory to a first compute node in the plurality of compute nodes, and that receives from a second compute node in the plurality of compute nodes remote copies of the remote homed objects stored in the flash memory of the second compute node;

a Sharing Data Fabric (SDF) component executing on the processor, that enables storing the remote copies of the remote homed objects received by the NIC into the volatile memory as the cached objects,

wherein the SDF component stores objects using a single global address space that is also used by SDF components residing on respective other nodes of the plurality of compute nodes,

wherein the application program executing on the processor accesses the cached objects from the volatile memory, including cached objects that are remote copies of the remote homed objects stored in the flash memory of the remote compute nodes in the plurality of compute nodes;

wherein the SDF component is configured to determine the home node for the homed objects; and

wherein the SDF component sends copies of homed objects stored in the flash memory to other compute nodes in the plurality of compute nodes through the network for storage as cached objects in said other compute nodes.

2. The global shared flash-memory system of claim 1 , wherein each compute node in the plurality of compute nodes further comprises:

a local switch, coupled to the processor, the flash memory, and the NIC, for allowing the processor to communicate with other compute nodes through the NIC, and for accessing the homed objects in the flash memory to generate the local copies of the homed objects for transfer to the first compute node via the NIC.

3. The global shared flash-memory system of claim 2 , wherein the local switch connects to the flash memory and to the NIC using a Peripheral Component Interconnect Express (PCIe) bus; and

wherein the network is an Ethernet network that connects together the plurality of compute nodes.

4. The global shared flash-memory system of claim 1 , further comprising:

an applications-programming interface (API) to the SDF component, the API being used by the application program to get a requested cached object that is not present in the volatile memory, the SDF component getting a remote copy of a homed object stored in the flash memory of a remote home node in the plurality of compute nodes and storing the remote copy as the requested cached object for access by the application program;

wherein the application program is not aware that the requested cached object is located at the remote home node, and the application program uses the API to activate the SDF component to transparently retrieve objects from the remote home node.

5. The global shared flash-memory system of claim 1 , wherein each compute node in the plurality of compute nodes further comprises:

a plurality of flash memory chips, wherein the flash memory comprises the plurality of flash memory chips;

a plurality of flash controllers that activate the plurality of flash memory chips to read and write flash blocks in the plurality of flash memory chips;

a flash manager for assigning the flash blocks for writing using a wear-leveling scheme to reduce wear of the flash blocks storing the homed objects.

6. The global shared flash-memory system of claim 1 , wherein each of the plurality of compute nodes acting as a home node further comprises:

a global cache directory having a sharing list, the sharing list being a list of ones of the plurality of compute nodes storing the cached objects that are copies of the homed objects stored in the flash memory of the home node;

a flash object map being used by the SDF component to locate the homed objects within the non-volatile solid-state memory cells of the flash memory in response to a request for the cached objects from another compute node in the plurality of compute nodes.

7. The global shared flash-memory system of claim 6 , further comprising:

coherency means, in the SDF component, for fetching a modified copy of a remote homed object from a sharing node in the plurality of compute nodes identified by the global cache directory, and for sending the modified copy instead of the remote homed object in response to a request for the cached object.

8. The global shared flash-memory system of claim 6 , further comprising:

coherency means, in the SDF component, for maintaining coherency of each homed object stored in the flash memory and of the remote homed objects stored at the remote compute nodes in the plurality of compute nodes.

9. The global shared flash-memory system of claim 1 , wherein each of the plurality of compute nodes acting as an action node with a requested cached object further comprises:

a transaction table that stores a list of objects touched by a transaction and an initial state for all objects touched by the transaction;

wherein the initial state is a state before the transaction begins;

transaction recovery means for aborting the transaction by restoring the objects touched by the transaction to their initial state stored in the transaction table;

wherein the transaction is a series of operations that are all committed together or all aborted.

10. The global shared flash-memory system of claim 9 , wherein each of the plurality of compute nodes acting as the action node with the requested cached object further comprises:

transaction-commit means for committing homed object updates, activated after all operations in the transaction are successfully completed and all prior outstanding requests for objects touched by the transaction are completed;

flash-update means, activated by the transaction-commit means, for sending modified objects touched by the transaction to home nodes in the plurality of compute nodes, the home nodes updating the homed objects in the flash memory using the modified objects with changes from the transaction.

11. The global shared flash-memory system of claim 1 , wherein the non-volatile solid-state memory cells consume less power than a rotating disk when writing the homed objects.

12. The global shared flash-memory system of claim 11 , wherein the flash memory stores objects as non-volatile data stored in electrically-erasable programmable read-only memory (EEPROM) memory cells or in phase-change memory (PCM) cells.

13. The global shared flash-memory system of claim 1 , wherein the flash memory of the plurality of compute nodes collectively are accessible as the global address space.

14. A method for managing a globally shared flash-memory system, the method comprising:

at each compute node in a computer system comprising a plurality of compute nodes, performing operations comprising:

storing homed objects in a flash memory;

storing cached objects in a volatile memory, wherein the cached objects include copies of remote homed objects stored at remote compute nodes in the plurality of compute nodes; wherein each homed object, including the homed objects stored in the flash memory and the remote homed objects stored at remote compute nodes, has a respective single home node;

sending, via a network interface controller (NIC) coupled by a network to other compute nodes, local copies of the homed objects stored in the flash memory to a first compute node in the plurality of compute nodes, and receiving, via the NIC, from a second compute node in the plurality of compute nodes remote copies of the remote homed objects stored in the flash memory of the second compute node; and

executing a Sharing Data Fabric (SDF) component to enable storing the remote copies of the remote homed objects received by the NIC into the volatile memory as the cached objects,

wherein the SDF component stores objects using a single global address space that is also used by SDF components residing on respective other nodes of the plurality of compute nodes,

wherein the SDF component is configured to determine the home node for the homed objects, and

wherein executing the SDF component includes sending copies of homed objects stored in the flash memory to other compute nodes in the plurality of compute nodes through the network for storage as cached objects in said other compute nodes.

15. The method of claim 14 , wherein each compute node in the plurality of compute nodes further comprises:

a local switch, coupled to the processor, the flash memory, and the NIC, for allowing the processor to communicate with other compute nodes through the NIC, and for accessing the homed objects in the flash memory to generate the local copies of the homed objects for transfer to the NIC for sending to the first compute node via the NIC.

16. The method of claim 15 , wherein the local switch connects to the flash memory and to the NIC using a Peripheral Component Interconnect Express (PCIe) bus; and

wherein the network is an Ethernet network that connects together the plurality of compute nodes.

17. The method of claim 14 , further comprising:

executing an application program, including obtaining a requested cached object that is not present in the volatile memory using an applications-programming interface (API) to the SDF component, the SDF component obtaining a remote copy of a homed object stored in the flash memory of a remote home node in the plurality of compute nodes and storing the remote copy as the requested cached object for access by the application program;

wherein the application program is not aware that the requested cached object is located at the remote home node, and the application program uses the API to activate the SDF component to transparently retrieve objects from the remote home node.

18. The method of claim 14 , wherein each compute node in the plurality of compute nodes further comprises:

a plurality of flash memory chips, wherein the flash memory comprises the plurality of flash memory chips;

a plurality of flash controllers that activate the plurality of flash memory chips to read and write flash blocks in the plurality of flash memory chips;

a flash manager for assigning the flash blocks for writing using a wear-leveling scheme to reduce wear of the flash blocks storing the homed objects.

19. The method of claim 14 , wherein each of the plurality of compute nodes acting as a home node further comprises:

a global cache directory having a sharing list, the sharing list being a list of ones of the plurality of compute nodes storing the cached objects that are copies of the homed objects stored in the flash memory of the home node;

a flash object map being used by the SDF component to locate the homed objects within the non-volatile solid-state memory cells of the flash memory in response to a request for the cached objects from another compute node in the plurality of compute nodes,

whereby the homed objects are located with the flash object map and compute nodes storing the remote copies of the homed objects are located using the global cache directory.

20. The method of claim 19 , further comprising:

fetching, in the SDF component, a modified copy of a remote homed object from a sharing node in the plurality of compute nodes identified by the global cache directory; and

sending the modified copy instead of the remote homed object in response to a request for the cached object.

21. The method of claim 14 , wherein each of the plurality of compute nodes acting as an action node with a requested cached object further comprises:

a transaction table that stores a list of objects touched by a transaction and an initial state for all objects touched by the transaction;

wherein the initial state is a state before the transaction begins;

transaction recovery means for aborting the transaction by restoring the objects touched by the transaction to their initial state stored in the transaction table;

wherein the transaction is a series of operations that are all committed together or all aborted.

22. The method of claim 21 , wherein each of the plurality of compute nodes acting as the action node with the requested cached object further comprises:

transaction-commit means for committing homed object, activated after all operations in the transaction are successfully completed and all prior outstanding requests for objects touched by the transaction are completed;

flash-update means, activated by the transaction-commit means, for sending modified objects touched by the transaction to home nodes in the plurality of compute nodes, the home nodes updating the homed objects in the flash memory using the modified objects with changes from the transaction.

23. The method of claim 14 , wherein the non-volatile solid-state memory cells consume less power than a rotating disk when writing the homed objects.

24. The method of claim 23 , wherein the flash memory stores objects as non-volatile data stored in electrically-erasable programmable read-only memory (EEPROM) memory cells or in phase-change memory (PCM) cells.