Providing distributed array containers for programming objects
View Patent ↗Embodiments are directed to establishing registration objects for distributed processes, to managing memory on worker processes of a distributed software application and to using object serialization to communicate references to shim objects. In one scenario, a computer system accesses distributed process instances in a distributed runtime and creates a registration object for each of the process instances in the distributed runtime. The registration object includes a key value pair, where the key includes a unique identifier (ID) that identifies a distributed array instance associated with the distributed process, and the value includes a reference to a local portion of the distributed array instance. The computer system then maintains a mapping between the unique ID and the distributed array instance using the registration object. As such, the key value refers to the local portion of the same distributed array instance on each distributed process of the distributed runtime.
1. A method, implemented at a computer system that includes one or more hardware processors, for managing memory on worker processes of a distributed software application, including the following:
determining that a distributed software function has finished processing, the distributed software function including one or more shim objects instantiated on a master rank and one or more worker ranks of the distributed application, the shim objects being configured to store a reference to a local object instance of a distributed array object, the reference including a unique identifier for the distributed array object;
upon determining that the distributed software application has finished processing, allowing the shim objects instantiated on the worker ranks to go out of scope; and
retaining the distributed object reference to prevent garbage collection from recovering local data storage associated with the distributed software function.
2. The method of claim 1 , wherein the shim object includes one or more portions of metadata describing at least one of global shape and distribution information for the distributed array object.
3. The method of claim 1 , wherein the shim objects allow communication among one or more ranks of the distributed software function.
4. The method of claim 1 , wherein the shim objects are implemented in a thin-client-dependent environment.
5. The method of claim 1 , wherein the shim objects comprise classes, allowing users to specify which serializers and deserializers are to be used in communication.
6. The method of claim 1 , further comprising updating one or more of the references upon determining that the reference's corresponding data location has changed due to garbage collection.
7. The method of claim 1 , further comprising using the unique identifier for the distributed array object to serialize the distributed array object and convert the distributed array object into a key.
8. The method of claim 7 , further comprising reconstituting the local object instance of the distributed array object based on the key.
9. A computer program product comprising one or more hardware storage devices having stored thereon computer-executable instructions that, when executed by one or more processors of a computing system, cause the computing system manage memory on worker processes of a distributed software application, including the following:
determining that a distributed software function has finished processing, the distributed software function including one or more shim objects instantiated on a master rank and one or more worker ranks of the distributed application, the shim objects being configured to store a reference to a local object instance of a distributed array object, the reference including a unique identifier for the distributed array object;
upon determining that the distributed software application has finished processing, allowing the shim objects instantiated on the worker ranks to go out of scope; and
retaining the distributed object reference to prevent garbage collection from recovering local data storage associated with the distributed software function.
10. The computer program product of claim 9 , wherein the shim object includes one or more portions of metadata describing at least one of global shape and distribution information for the distributed array object.
11. The computer program product of claim 9 , wherein the shim objects allow communication among one or more ranks of the distributed software function.
12. The computer program product of claim 9 , wherein the shim objects are implemented in a thin-client-dependent environment.
13. The computer program product of claim 9 , wherein the shim objects comprise classes, allowing users to specify which serializers and deserializers are to be used in communication.
14. The computer program product of claim 9 , further comprising updating one or more of the references upon determining that the reference's corresponding data location has changed due to garbage collection.
15. A computer system comprising the following:
one or more hardware processors;
system memory;
one or more computer-readable storage media having stored thereon computer-executable instructions that, when executed by the one or more processors, causes the computing system to perform a method for managing memory on worker processes of a distributed software application, the method comprising the following:
determining that a distributed software function has finished processing, the distributed software function including one or more shim objects instantiated on a master rank and one or more worker ranks of the distributed application, the shim objects being configured to store a reference to a local object instance of a distributed array object, the reference including a unique identifier for the distributed array object;
upon determining that the distributed software application has finished processing, allowing the shim objects instantiated on the worker ranks to go out of scope; and
retaining the distributed object reference to prevent garbage collection from recovering local data storage associated with the distributed software function.
16. The computer system of claim 15 , wherein the shim object includes one or more portions of metadata describing at least one of global shape and distribution information for the distributed array object.
17. The computer system of claim 15 , wherein the shim objects allow communication among one or more ranks of the distributed software function.
18. The computer system of claim 15 , wherein the shim objects are implemented in a thin-client-dependent environment.
19. The computer system of claim 15 , wherein the shim objects comprise classes, allowing users to specify which serializers and deserializers are to be used in communication.
20. The computer system of claim 15 , further comprising updating one or more of the references upon determining that the reference's corresponding data location has changed due to garbage collection.