Global pointers for scalable parallel applications
View Patent ↗Mapping of cacheable memory pages from other processes in a parallel job provides a very efficient mechanism for inter-process communication. A trivial address computation can then be used to look up a virtual address that allows the use of cacheable loads and stores to directly access or update the memory of other processes in the job for communication purposes. When an interconnection network permits the cacheable access of one host's memory from another host in the cluster, kernel and library software can map memory from processes on other hosts, in addition to the memory on the same host. This mapping can be done at the start of a parallel job using a system library interface. A function in an application programming interface provides a user-level, fast lookup of a virtual address that references data regions residing on all of the processes in a parallel job running across multiple hosts.
1. A method of implementing a multi-processing programming model, comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, the hosts using a message passing interface for communication between processes; and
transferring messages through mapped memory regions by a copy request from a user process for an inter-host transfer of a cache line of memory initiated by an object program generated without specifically requesting the inter-host transfer.
2. A method as recited in claim 1 ,
wherein each of the multiple hosts includes at least one processor executing at least one of the cooperating processes, and
wherein said providing can make all memory assigned to any of the cooperating processes accessible to any of the processes.
3. A method as recited in claim 2 , wherein said providing includes defining pointers to arbitrary data structures from arbitrary processes in multiple hosts.
4. A method as recited in claim 3 , wherein the pointers are unique across all of the hosts.
5. A method as recited in claim 2 , further comprising mapping portions of the memory upon initial start up of each process for subsequent inter-host access.
6. A method as recited in claim 5 , wherein said mapping of portions of the memory includes mapping segments of virtual memory fixed in size and changeable in size.
7. A method as recited in claim 6 , wherein said mapping of portions of the memory includes mapping at least one of a static area, private heap, symmetric heap, stack area, and internal buffers for each process in each host.
8. A method as recited in claim 1 wherein said providing includes defining pointers that are unique across all of the hosts.
9. A method as recited in claim 1 , wherein said transferring includes short-strided transfers or irregularly structured data areas, where each stride is a gap between consecutive memory words that are accessed.
10. A method as recited in claim 1 , wherein said providing comprises
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data indicating that the area of memory is to be shared and restrictions on access to the area of memory;
responding to an access permission request from a second process for permission to access the area of memory by the second process by determining whether the second process meets the restrictions on access; and
mapping the area of memory into virtual address space of the second process if permission is granted.
11. A method as recited in claim 10 , wherein said setting of restrictions on access to the area of memory is based on user configurable permission control.
12. A method as recited in claim 10 , wherein said setting, determining and mapping are performed in response to calls to library functions in user memory space.
13. A method of implementing a multi-processing programming model, comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, said providing including
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data indicating that the area of memory is to be shared and restrictions on access to the area of memory,
responding to an access permission request from a second process for permission to access the area of memory by the second process by determining whether the second process meets the restrictions on access,
mapping the area of memory into virtual address space of the second process if permission is granted, and
responding to a fault generated upon an initial attempt to access the area of memory by handling the fault to finish mapping at least a portion of data from the area of memory of the first process into virtual address space of the second process, so that a user load/store operation will succeed.
14. A method of implementing a multi-processing programming model, comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, said providing including
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data, in response to calls to library functions in user memory space, indicating that the area of memory is to be shared and restrictions on access to the area of memory, where the share request includes returning a handle identifying the area of memory in the virtual address space of the first process and the first process uses inter-host communication to transmit the handle to a second process,
responding to an access permission request, from the second process and including a get library function call using the handle as an argument, for permission to access the area of memory by the second process by determining, in response to calls to library functions in user memory space, whether the second process meets the restrictions on access, and returning an identifier if permission to access is determined, and
mapping, in response to calls to library functions in user memory space, the area of memory into virtual address space of the second process if permission is granted, where said mapping includes responding to an attach library function call using the identifier as an argument.
15. A method as recited in claim 14 , wherein the handle is unique across all of the hosts.
16. At least one computer readable medium storing at least one program to control a computer system to implement a multi-processing programming model according to a method comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, the hosts using a message passing interface for communication between processes; and
transferring messages through mapped memory regions by a copy request from a user process for an inter-host transfer of a cache line of memory initiated by an object program generated without specifically requesting the inter-host transfer.
17. At least one computer readable medium as recited in claim 16 ,
wherein each of the multiple hosts includes at least one processor executing at least one of the cooperating processes, and
wherein said providing can make all memory assigned to any of the cooperating processes accessible to any of the processes.
18. At least one computer readable medium as recited in claim 17 , wherein said providing includes defining pointers to arbitrary data structures from arbitrary processes in multiple hosts.
19. At least one computer readable medium as recited in claim 18 , wherein the pointers are unique across all of the hosts.
20. At least one computer readable medium as recited in claim 17 , further comprising mapping portions of the memory upon initial start up of each process for subsequent inter-host access.
21. At least one computer readable medium as recited in claim 20 , wherein said mapping of portions of the memory includes mapping segments of virtual memory fixed in size and changeable in size.
22. At least one computer readable medium as recited in claim 21 , wherein said mapping of portions of the memory includes mapping at least one of a static area, private heap, symmetric heap, stack area, and internal buffers for each process in each host.
23. At least one computer readable medium as recited in claim 16 , wherein the pointers are unique across all of the hosts.
24. At least one computer readable medium as recited in claim 16 , wherein said transferring includes short-strided transfers or irregularly structured data areas, where each stride is a gap between consecutive memory words that are accessed.
25. At least one computer readable medium as recited in claim 16 , wherein said providing comprises
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data indicating that the area of memory is to be shared and restrictions on access to the area of memory;
responding to an access permission request from a second process for permission to access the area of memory by the second process by determining whether the second process meets the restrictions on access; and
mapping the area of memory into virtual address space of the second process if permission is granted.
26. At least one computer readable medium as recited in claim 25 , wherein said setting of restrictions on access to the area of memory is based on user configurable permission control.
27. At least one computer readable medium as recited in claim 25 , wherein said setting, determining and mapping are performed in response to calls to library functions in user memory space.
28. At least one computer readable medium as recited in claim 25 , further comprising disabling access to the area of memory in the virtual address space of the first process in response to one of requesting revocation of permission to access by the first process and requesting detachment of the area of memory by the second process.
29. At least one computer readable medium as recited in claim 28 , wherein said requesting detachment is performed automatically upon termination of the second process.
30. At least one computer readable medium as recited in claim 28 ,
wherein said responding to the share request includes returning a handle identifying the area of memory in the virtual address space of the first process;
wherein said requesting revocation includes calling a remove library function with the handle as an argument, and
wherein said requesting detachment includes calling a detach library function using a virtual address, corresponding to the handle, as a argument.
31. At least one computer readable medium as recited in claim 28 , further comprising requesting release of the area of memory by the second process when access is no longer needed.
32. At least one computer readable medium storing at least one program to control a computer system to implement a multi-processing programming model according to a method comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, said providing including
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data indicating that the area of memory is to be shared and restrictions on access to the area of memory;
responding to an access permission request from a second process for permission to access the area of memory by the second process by determining whether the second process meets the restrictions on access;
mapping the area of memory into virtual address space of the second process if permission is granted; and
responding to a fault generated upon an initial attempt to access the area of memory by handling the fault to finish mapping at least a portion of data from the area of memory of the first process into virtual address space of the second process, so that a user load/store operation will succeed.
33. At least one computer readable medium storing at least one program to control a computer system to implement a multi-processing programming model according to a method comprising:
providing cacheable memory access to shared memory by cooperating processes executing on multiple hosts, said providing including
responding to a share request from a first process to share an area of memory in virtual address space of the first process by setting data, in response to calls to library functions in user memory space, indicating that the area of memory is to be shared and restrictions on access to the area of memory, where the share request includes returning a handle identifying the area of memory in the virtual address space of the first process and the first process uses inter-host communication to transmit the handle to a second process,
responding to an access permission request, from the second process and including a get library function call using the handle as an argument, for permission to access the area of memory by the second process by determining, in response to calls to library functions in user memory space, whether the second process meets the restrictions on access, and returning an identifier if permission to access is determined, and
mapping, in response to calls to library functions in user memory space, the area of memory into virtual address space of the second process if permission is granted, where said mapping includes responding to an attach library function call using the identifier as an argument.
34. At least one computer readable medium as recited in claim 33 , wherein the handle is unique across all of the hosts.
35. A computer system implementing a multi-processing programming model, comprising:
cacheable memory; and
processors programmed to provide access to said cacheable memory by cooperating processes executing on multiple hosts using a message passing interface for communication between the cooperating processes through mapped memory regions via an inter-host transfer of a cache line of memory that is initiated by an object program generated without specifically requesting the inter-host transfer.
36. A computer system as recited in claim 7 , wherein said processors are further programmed to define pointers that are unique across all of the hosts.
37. A system implementing a multi-processing programming model, comprising:
cacheable memory; and
means for providing access to said cacheable memory by cooperating processes executing on multiple hosts using a message passing interface for communication between the cooperating processes through mapped memory regions via an inter-host transfer of a cache line of memory that is initiated by an object program generated without specifically requesting the inter-host transfer.