Utilizing memory coherency to improve bandwidth performance

1. A computer-implemented method for rerouting device allocation commands to a GPU having GPU memory through a driver and a software stack, wherein the rerouting further comprises:

updating the software stack, wherein the updated software stack allows for adding drivers to monitor for relevant device allocation commands to be redirected to an address translation service;

subsequent to the updating, performing the steps of:

receiving, a call from an application on a host having a host memory;

determining that the call includes a relevant device allocation command, wherein the device allocation command is configured to allocate a set of data to the graphical processing unit (GPU);

dynamically intercepting the call to prevent the call from being passed to the driver on the host;

initiating an alternate data allocation command, wherein the alternate data allocation command allocates data to a coherent memory, and the coherent memory uses a portion of the GPU memory and is physically located on the GPU, and further the data stored in the coherent memory is accessible to the host through the address translation service located on the host without mirroring the data to the host memory, wherein the address translation service allows the host to access the data in the coherent memory by translating a virtual GPU address into a physical address usable by the host;

completing the alternate data allocation command; and

returning the completed alternate data allocation command to the application.

2. The method of claim 1 , wherein the application has a first state at a first time prior to the updating, and a second state at a second time after the updating; and

wherein the first state and the second state are the same.

3. The method of claim 1 where the alternate data allocation command includes a first command and a second command.

4. The method of claim 3 , wherein the first command allocates data on the host, and the second command allocates data on the GPU.

5. The method of claim 4 , wherein the first command includes an mmap command, and the second command includes a mbind command.

6. The method of claim 4 , wherein the first command includes a glibc malloc command, and the second command includes a cudaMemPrefetchAsynce command.

7. The method of claim 1 , wherein the call is a cudaMalloc call.

8. The method of claim 1 , wherein the call is a cudaMallocManaged call.

9. The method of claim 1 , wherein the host transfers data to the GPU via a NVLink.

10. A system for rerouting device allocation commands to a GPU having GPU memory through a driver and a software stack, wherein the system comprises:

a processor; and

a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, are configured to cause the processor to:

update the software stack, wherein the undated software stack allows for adding drivers to monitor for relevant device allocation commands to be redirected to an address translation service;

subsequent to the updating, performing the steps of:

receive, a call from an application on a host having a host memory;

determine that the call includes a relevant device allocation command, wherein the device allocation command is configured to allocate a set of data to the graphical processing unit (GPU);

dynamically intercept the call to prevent the call from being passed to the driver on the host;

initiate an alternate data allocation command, wherein the alternate data allocation command allocates data to a coherent memory and the coherent memory uses a portion of the GPU memory and is physically located on the GPU, and further the data stored in the coherent memory is accessible to the host though the address translation service located on the host without mirroring the data to the host memory, wherein the address translation service allows the host to access the data in the coherent memory by translating a virtual GPU address into a physical address usable by the host;

complete the alternate data allocation command; and

return the completed alternate data allocation command to the application.

11. The system of claim 10 , wherein the alternate data allocation command includes a first command and a second command.

12. The system of claim 11 , wherein the first command allocates data on the host, and the second command allocates data on the GPU.

13. A computer program product for rerouting device allocation commands to a GPU having GPU memory though a driver and a software stack, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing unit to cause the processing unit to:

update the software stack, wherein the updated software stack allows for adding drivers to monitor for relevant device allocation commands to be redirected to an address translation service;

subsequent to the updating, performing the steps of:

receive, a call from an application on a host having a host memory;

determine that the call includes a relevant device allocation command, wherein the device allocation command is configured to allocate a set of data to the graphical processing unit (GPU), wherein the host transfers data to the GPU via a NVLink;

dynamically intercept the call to prevent the call from being passed to the driver on the host;

initiate an alternate data allocation command, wherein the alternate data allocation command allocates data to a coherent memory, and the coherent memory uses a portion of the GPU memory and is physically located on the GPU, and further the data stored in the coherent memory is accessible to the host through the address translation service located on the host without mirroring the data to the host memory, wherein the address translation service allows the host to access the data in the coherent memory by translating a virtual GPU address into a physical address usable by the host;

complete the alternate data allocation command; and

return the completed alternate data allocation command to the application.

14. The computer program product of claim 13 , wherein the alternate data allocation command includes a first command and a second command, the first command allocates data on the host, and the second command allocates data on the GPU.