3D API redirection for virtual desktop infrastructure using a client-side shadow window
Techniques for implementing 3DI API redirection for VDI desktops are provided. In one set of embodiments, a server system can intercept a call to a 3D API made by a 3D application running within a VM on the server system, where the VM hosts a desktop that is presented to a user of a client system. The server system can determine metadata associated with the call, where the metadata including a name of the 3D API and one or more input parameter values to the call, and can transmit the metadata to the client system. In response, the client system can reconstruct the call to the 3D API using the metadata and execute the call using one or more physical GPUs residing on the client system.
1. A method comprising:
receiving, by a client system from a server system, first metadata for a first 3D application programming interface (API) call by a 3D application running within a virtual machine (VM) on the server system, the first 3D API call being directed to creation of a server-side targeting window in a server-side desktop hosted by the VM, the first metadata including a first identifier identifying a handler of the server-side targeting window;
in response to receiving the first metadata, creating, by the client system, a client-side shadow window corresponding to the server-side targeting window within a client-side desktop window, the client-side desktop window enabling a user of the client system to remotely interact with the server-side desktop;
determining, by the client system, a second identifier identifying a handler of the client-side shadow window;
storing, by the client system, an association between the first identifier and the second identifier;
receiving, by the client system from the server system, second metadata for a second 3D API call that is directed to the server-side targeting window and that is intercepted on the server system; and
using the stored association between the first identifier and the second identifier to process, by a physical graphics processing unit (GPU) of the client system, a 3D rendering operation of the second 3D API call, wherein processing the 3D rendering operation of the second 3D API call on the client system includes reconstructing a client-side API call corresponding to the second 3D API call by substituting the second identifier of the handler of the client-side shadow window for the first identifier of the handler of the server-side targeting window in the second 3D API call based on the stored association.
2. The method of claim 1 , wherein the client-side shadow window is invisible in the client-side desktop window upon being created, and wherein the client-side shadow window is made visible upon execution of the 3D rendering operation by the physical GPU of the client system.
3. The method of claim 1 further comprising:
receiving, from the server system, third metadata for a third 3D API call by the 3D application, the third 3D API call being directed to movement or scaling of the server-side targeting window in the server-side desktop, the third metadata for the third 3D API call including the first identifier and a first area covered by the server-side targeting window within the server-side desktop;
in response to receiving the third metadata for the third 3D API call, determining a second area covered by the client-side shadow window within the client-side desktop window;
determining, based on the first area and the second area, a new location or size for the client-side shadow window; and
causing the client-side shadow window to be moved or scaled in the client-side desktop window in accordance with the new location or size.
4. The method of claim 3 wherein determining the second area comprises:
obtaining a window identifier of the client-side desktop window from an operating system (OS) of the client system; and
requesting, from the OS, the second area using the window identifier.
5. The method of claim 3 wherein the new location or size for the client-side shadow window corresponds to a current location or size of the server-side targeting window within the server-side desktop in view of the first area.
6. The method of claim 1 further comprising:
capturing user inputs directed to the client-side shadow window; and
transmitting the user inputs and the second identifier to the server system for processing by the 3D application.
7. The method of claim 1 , wherein processing the 3D rendering operation of the second 3D API call on the client system further comprises:
passing the reconstructed client-side API call including the second identifier of the handler of the client-side shadow window to a GPU driver of the physical GPU of the client system.
8. A non-transitory computer readable storage medium having stored thereon program code executable by a client system, the program code embodying a method comprising:
receiving, from a server system, first metadata for a first 3D application programming interface (API) call by a 3D application running within a virtual machine (VM) on the server system, the first 3D API call being directed to creation of a server-side targeting window in a server-side desktop hosted by the VM, the first metadata including a first identifier identifying a handler of the server-side targeting window;
in response to receiving the first metadata, creating a client-side shadow window corresponding to the server-side targeting window within a client-side desktop window, the client-side desktop window enabling a user of the client system to remotely interact with the server-side desktop;
determining a second identifier identifying a handler of the client-side shadow window;
storing an association between the first identifier and the second identifier;
receiving, by the client system from the server system, second metadata for a second 3D API call that is directed to the server-side targeting window and that is intercepted on the server system; and
using the stored association between the first identifier and the second identifier to process, by a physical graphics processing unit (GPU) of the client system, a 3D rendering operation of the second 3D API call, wherein processing the 3D rendering operation of the second 3D API call on the client system includes reconstructing a client-side API call corresponding to the second 3D API call by substituting the second identifier of the handler of the client-side shadow window for the first identifier of the handler of the server-side targeting window in the second 3D API call based on the stored association.
9. The non-transitory computer readable storage medium of claim 8 wherein the client-side shadow window is invisible in the client-side desktop window upon being created, and wherein the client-side shadow window is made visible upon execution of the 3D rendering operation by the physical GPU of the client system.
10. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:
receiving, from the server system, third metadata for a third 3D API call by the 3D application, the third 3D API call being directed to movement or scaling of the server-side targeting window in the server-side desktop, the third metadata for the third 3D API call including the first identifier and a first area covered by the server-side targeting window within the server-side desktop;
in response to receiving the third metadata for the third 3D API call, determining a second area covered by the client-side shadow window within the client-side desktop window;
determining, based on the first area and the second area, a new location or size for the client-side shadow window; and
causing the client-side shadow window to be moved or scaled in the client-side desktop window in accordance with the new location or size.
11. The non-transitory computer readable storage medium of claim 10 wherein determining the second area comprises:
obtaining a window identifier of the client-side desktop window from an operating system (OS) of the client system; and
requesting, from the OS, the second area using the window identifier.
12. The non-transitory computer readable storage medium of claim 10 wherein the new location or size for the client-side shadow window corresponds to a current location or size of the server-side targeting window within the server-side desktop in view of the first area.
13. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:
capturing user inputs directed to the client-side shadow window; and
transmitting the user inputs and the second identifier to the server system for processing by the 3D application.
14. The non-transitory computer readable storage medium of claim 8 , wherein processing the 3D rendering operation of the second 3D API call on the client system further comprises:
passing the reconstructed client-side API call including the second identifier of the handler of the client-side shadow window to a GPU driver of the physical GPU of the client system.
15. A client system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed by the processor, causes the processor to:
receive, from a server system, first metadata for a first 3D application programming interface (API) call by a 3D application running within a virtual machine (VM) on the server system, the first 3D API call being directed to creation of a server-side targeting window in a server-side desktop hosted by the VM, the first metadata including a first identifier identifying a handler of the server-side targeting window;
in response to receiving the first metadata, create a client-side shadow window corresponding to the server-side targeting window within a client-side desktop window, the client-side desktop window enabling a user of the client system to remotely interact with the server-side desktop;
determine a second identifier identifying a handler of the client-side shadow window;
store an association between the first identifier and the second identifier;
receive, by the client system from the server system, second metadata for a second 3D API call that is directed to the server-side targeting window and that is intercepted on the server system; and
use the stored association between the first identifier and the second identifier to process, by a physical graphics processing unit (GPU) of the client system, a 3D rendering operation of the second 3D API call, wherein processing the 3D rendering operation of the second 3D API call on the client system includes reconstructing a client-side API call corresponding to the second 3D API call by substituting the second identifier of the handler of the client-side shadow window for the first identifier of the handler of the server-side targeting window in the second 3D API call based on the stored association.
16. The client system of claim 15 wherein the client-side shadow window is invisible in the client-side desktop window upon being created, and wherein the client-side shadow window is made visible upon execution of the 3D rendering operation by the physical GPU of the client system.
17. The client system of claim 15 wherein the program code further causes the processor to:
receive, from the server system, third metadata for a third 3D API call by the 3D application, the third 3D API call being directed to movement or scaling of the server-side targeting window in the server-side desktop, the third metadata for the third 3D API call including the first identifier and a first area covered by the server-side targeting window within the server-side desktop;
in response to receiving the third metadata for the third 3D API call, determine a second area covered by the client-side shadow window within the client-side desktop window;
determine, based on the first area and the second area, a new location or size for the client-side shadow window; and
cause the client-side shadow window to be moved or scaled in the client-side desktop window in accordance with the new location or size.
18. The client system of claim 17 wherein the program code that causes the processor to determine the second area comprises program code that causes the processor to:
obtain a window identifier of the client-side desktop window from an operating system (OS) of the client system; and
request, from the OS, the second area using the window identifier.
19. The client system of claim 17 wherein the new location or size for the client-side shadow window corresponds to a current location or size of the server-side targeting window within the server-side desktop in view of the first area.
20. The client system of claim 15 wherein the program code further causes the processor to:
capture user inputs directed to the client-side shadow window; and
transmit the user inputs and the second identifier to the server system for processing by the 3D application.
21. The client system of claim 15 , wherein processing the 3D rendering operation of the second 3D API call on the client system further comprises:
passing the reconstructed client-side API call including the second identifier of the handler of the client-side shadow window to a GPU driver of the physical GPU of the client system.