Method and system for managing memory of data processing accelerators

1. A computer-implemented method for memory management, the method comprising:

performing a secure boot using a security module of a host system;

establishing a trusted execution environment (TEE) associated with one or more processors of the host system;

launching a memory manager, wherein the memory manager is configured to manage memory resources of a data processing (DP) accelerator coupled to the host system over a bus, including maintaining memory usage information at least by recording, in a memory usage data structure, allocation of memory blocks of a global memory residing within the DP accelerator to one or more applications to map, in the memory data structure, which of the memory blocks are allocated to which of the one or more applications, wherein the memory manager and the memory usage data structure are implemented as a part of a runtime library associated with the DP accelerator and the memory manager and the memory usage data structure are executed within the TEE of the host system; and

in response to a request received from an application running within the TEE for accessing a memory location of the DP accelerator, allowing or denying the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information.

2. The method of claim 1 , wherein allowing the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information includes sending, by the memory manager, a copy of the memory usage data structure and an allocation command to the DP accelerator.

3. The method of claim 1 , wherein maintaining memory usage information of global memory of the DP accelerator comprises maintaining a memory usage data structure to record memory allocation of memory blocks of the global memory of the DP accelerator.

4. The method of claim 3 , wherein the memory usage data structure comprises a plurality of entries, each entry recording a memory block of the global memory of the DP accelerator that has been allocated.

5. The method of claim 4 , wherein each entry stores a starting memory address of a corresponding memory block, a size of the corresponding memory block, and a flag indicating whether the corresponding memory block has been allocated.

6. The method of claim 3 , further comprising:

receiving a first request from the application to allocate a first memory block from the global memory of the DP accelerator;

in response to the first request, determining whether the first memory block has been allocated based on the memory usage information stored in the memory usage data structure, without having to interrogate the DP accelerator; and

allocating the first memory block of the global memory to the application, in response to determining that the first memory block has not been allocated.

7. The method of claim 6 , further comprising denying the first request, in response to determining that the first memory block has been allocated.

8. The method of claim 3 , further comprising:

receiving a second request from the application to deallocate a second memory block from the global memory of the DP accelerator;

in response to the second request, determining whether the second memory block has been allocated to the application based on the memory usage information stored in the memory usage data structure;

deallocating the second memory block from the global memory, in response to determining that the second memory block has been allocated to the application; and

otherwise denying the second request.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

performing a secure boot using a security module of a host system;

establishing a trusted execution environment (TEE) associated with one or more processors of the host system;

launching a memory manager, wherein the memory manager is configured to manage memory resources of a data processing (DP) accelerator coupled to the host system over a bus, including maintaining memory usage information at least by recording, in a memory usage data structure, allocation of memory blocks of a global memory residing within the DP accelerator to one or more applications to map, in the memory data structure, which of the memory blocks are allocated to which of the one or more applications, wherein the memory manager and the memory usage data structure are implemented as a part of a runtime library associated with the DP accelerator and the memory manager and the memory usage data structure are executed within the TEE of the host system; and

in response to a request received from an application running within the TEE for accessing a memory location of the DP accelerator, allowing or denying the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information.

10. The machine-readable medium of claim 9 , wherein allowing the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information includes sending, by the memory manager, a copy of the memory usage data structure and an allocation command to the DP accelerator.

11. The machine-readable medium of claim 9 , wherein maintaining memory usage information of global memory of the DP accelerator comprises maintaining a memory usage data structure to record memory allocation of memory blocks of the global memory of the DP accelerator.

12. The machine-readable medium of claim 11 , wherein the memory usage data structure comprises a plurality of entries, each entry recording a memory block of the global memory of the DP accelerator that has been allocated.

13. The machine-readable medium of claim 12 , wherein each entry stores a starting memory address of a corresponding memory block, a size of the corresponding memory block, and a flag indicating whether the corresponding memory block has been allocated.

14. The machine-readable medium of claim 11 , wherein the operations further comprise:

receiving a first request from the application to allocate a first memory block from the global memory of the DP accelerator;

in response to the first request, determining whether the first memory block has been allocated based on the memory usage information stored in the memory usage data structure, without having to interrogate the DP accelerator; and

allocating the first memory block of the global memory to the application, in response to determining that the first memory block has not been allocated.

15. A data processing host system, comprising:

a security module;

one or more processors coupled to the security module; and

a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:

performing a secure boot using the security module of the host system,

establishing a trusted execution environment (TEE) associated with one or more processors of the host system,

launching a memory manager, wherein the memory manager is configured to manage memory resources of a data processing (DP) accelerator coupled to the host system over a bus, including maintaining memory usage information at least by recording, in a memory usage data structure, allocation of memory blocks of a global memory residing within the DP accelerator to one or more applications to map, in the memory data structure, which of the memory blocks are allocated to which of the one or more applications, wherein the memory manager and the memory usage data structure are implemented as a part of a runtime library associated with the DP accelerator and the memory manager and the memory usage data structure are executed within the TEE of the host system, and

in response to a request received from an application running within the TEE for accessing a memory location of the DP accelerator, allowing or denying the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information.

16. The system of claim 15 , wherein allowing the request based on the allocation of the global memory residing within the DP accelerator of the memory usage information includes sending, by the memory manager, a copy of the memory usage data structure and an allocation command to the DP accelerator.

17. The system of claim 15 , wherein maintaining memory usage information of global memory of the DP accelerator comprises maintaining a memory usage data structure to record memory allocation of memory blocks of the global memory of the DP accelerator.

18. The system of claim 17 , wherein the memory usage data structure comprises a plurality of entries, each entry recording a memory block of the global memory of the DP accelerator that has been allocated.

19. The system of claim 18 , wherein each entry stores a starting memory address of a corresponding memory block, a size of the corresponding memory block, and a flag indicating whether the corresponding memory block has been allocated.

20. The system of claim 17 , wherein the operations further comprise:

receiving a first request from the application to allocate a first memory block from the global memory of the DP accelerator;

in response to the first request, determining whether the first memory block has been allocated based on the memory usage information stored in the memory usage data structure, without having to interrogate the DP accelerator; and

allocating the first memory block of the global memory to the application, in response to determining that the first memory block has not been allocated.