Method and system for identifying drawing primitives for selective transmission to a remote display
Remote desktop servers include a display encoder that maintains a secondary framebuffer that contains display data to be encoded and transmitted to a remote client display and a list of display primitives effectuating updated display data in the secondary framebuffer. The display encoder submits requests to receive the list of drawing primitives to a video adapter driver that receives and tracks drawing primitives that, when executed, update a primary framebuffer.
1. A method comprising:
receiving a request from an application executing on a virtual machine to update a graphical user interface of the application displayed on a client device;
identifying drawing primitives corresponding to drawing commands that when executed update the graphical user interface;
determining, in response to the request to update the graphical user interface, whether to transmit to the client device i) the identified drawing primitives or ii) updated display data generated by applying the drawing primitives to prior interface data held in a framebuffer, the determining whether to transmit comprising:
determining a first bandwidth usage for transmitting one or more drawing primitives, and
determining a second bandwidth usage for transmitting the updated display data corresponding to one or more drawing primitives; and
transmitting the one or more drawing primitives to the client device in response to determining the first bandwidth usage is lesser than the second bandwidth usage or the updated display data to the client device in response to determining the second bandwidth usage is lesser than the first bandwidth usage.
2. The method of claim 1 , wherein the request to update a graphical user interface of the application is converted by a graphical drawing interface layer into drawing commands understood by a video adapter driver.
3. The method of claim 2 , wherein the video adapter driver marks entries of a driver blitmap data structure to indicate that at least a portion of pixel values in region of a first framebuffer will be updated by executing the drawing commands.
4. The method of claim 2 , wherein the video adapter driver converts the drawing commands to drawing primitives and inserts the drawing primitives into a queue.
5. The method of claim 4 , wherein identifying the drawing primitives comprises extracting the drawing primitives from the queue.
6. The method of claim 1 , wherein the one or more drawing primitives correspond to the drawing primitives that change pixels of a framebuffer relative to a prior state of the graphical user interface.
7. One or more non-transitory computer-readable media including instructions that, when executed by a processing unit of a computer, causes the processing unit to performing the steps of:
receiving a request from an application executing on a virtual machine to update a graphical user interface of the application displayed on a client device;
identifying drawing primitives corresponding to drawing commands that when executed update the graphical user interface;
determining, in response to the request to update the graphical user interface, whether to transmit to the client device i) the identified drawing primitives or ii) updated display data generated by applying the drawing primitives to prior interface data held in a framebuffer, the determining whether to transmit comprising:
determining a first bandwidth usage for transmitting one or more drawing primitives, and
determining a second bandwidth usage for transmitting the updated display data corresponding to one or more drawing primitives; and
transmitting the one or more drawing primitives to the client device in response to determining the first bandwidth usage is lesser than the second bandwidth usage or the updated display data to the client device in response to determining the second bandwidth usage is lesser than the first bandwidth usage.
8. The non-transitory computer-readable media of claim 7 , wherein the request to update a graphical user interface of the application is converted by a graphical drawing interface layer into drawing commands understood by a video adapter driver.
9. The non-transitory computer-readable media of claim 8 , wherein the video adapter driver marks entries of a driver blitmap data structure to indicate that at least a portion of pixel values in region of a first framebuffer will be updated by executing the drawing commands.
10. The non-transitory computer-readable media of claim 8 , wherein the video adapter driver converts the drawing commands to drawing primitives and inserts the drawing primitives into a queue.
11. The non-transitory computer-readable media of claim 10 , wherein identifying the drawing primitives comprises extracting the drawing primitives from the queue.
12. The non-transitory computer-readable media of claim 7 , wherein the one or more drawing primitives correspond to the drawing primitives that change pixels of a framebuffer relative to a prior state of the graphical user interface.
13. A system comprising one or more computers having one or more processors and one or more computer-readable media, wherein the one or more computers are configured to perform operations comprising:
receiving a request from an application executing on a virtual machine to update a graphical user interface of the application displayed on a client device;
identifying drawing primitives corresponding to drawing commands that when executed update the graphical user interface;
determining, in response to the request to update the graphical user interface, whether to transmit to the client device i) the identified drawing primitives or ii) updated display data generated by applying the drawing primitives to prior interface data held in a framebuffer, the determining whether to transmit comprising:
determining a first bandwidth usage for transmitting one or more drawing primitives, and
determining a second bandwidth usage for transmitting the updated display data corresponding to one or more drawing primitives; and
transmitting the one or more drawing primitives to the client device in response to determining the first bandwidth usage is lesser than the second bandwidth usage or the updated display data to the client device in response to determining the second bandwidth usage is lesser than the first bandwidth usage.
14. The system of claim 13 , wherein the request to update a graphical user interface of the application is converted by a graphical drawing interface layer into drawing commands understood by a video adapter driver.
15. The system of claim 14 , wherein the video adapter driver marks entries of a driver blitmap data structure to indicate that at least a portion of pixel values in region of a first framebuffer will be updated by executing the drawing commands.
16. The system of claim 14 , wherein the video adapter driver converts the drawing commands to drawing primitives and inserts the drawing primitives into a queue.
17. The system of claim 16 , wherein identifying the drawing primitives comprises extracting the drawing primitives from the queue.
18. The system of claim 13 , wherein the one or more drawing primitives correspond to the drawing primitives that change pixels of a framebuffer relative to a prior state of the graphical user interface.