System, method, and computer program product for applying different transport mechanisms for user interface and image portions of a remotely rendered image
View Patent ↗A system and method for remote rendering of computer graphics wherein user transactions are reliable and the transmission of rendered graphics is relatively fast. The invention is implemented in a client server context, where a computer graphics application and rendering resources are located at a server. A user controls the graphics application through a client machine connected to the server through a computer network. The user's commands are sent from the client to the server, while rendered computer graphics are transmitted from the server to a display at the client. Different transport protocols are used, depending on the requirements of a particular transmission. Data related to user interactions is transmitted using a relatively reliable transport protocol, such as TCP. Rendered subject graphics data is transmitted from the server to the client using a less reliable but faster transport protocol, such as UDP.
1. A system for remote rendering of graphics data, the system comprising:
a client; and
a server in communication with the client, comprising a compositor and a plurality of rendering pipelines, the plurality of rendering pipelines comprising a graphical operation module and a rasterizer,
wherein the client transmits a rendering command to a graphics application at the server using a reliable transport protocol,
wherein a first rendering pipeline of the plurality of rendering pipelines renders graphical user interface (GUI) graphics data, and a second rendering pipeline of the plurality of rendering pipelines renders subject graphics data,
wherein the server composites, using the compositor, the rendered subject graphics data and the rendered GUI graphics data, and
wherein the server transmits the composited graphics data using an unreliable transport protocol.
2. The system of claim 1 , wherein the client comprises an output device.
3. The system of claim 2 , wherein the output device comprises a display.
4. The system of claim 1 , wherein the implementation of the compositor comprises hardware.
5. The system of claim 1 , wherein the implementation of the compositor comprises software.
6. The system of claim 1 , wherein the reliable transport protocol is a TCP transport protocol.
7. The system of claim 1 , wherein the unreliable transport protocol is a UDP transport protocol.
8. A system for remote rendering of graphics data, the system comprising:
a client: and
a server in communication with the client, comprising a compositor and a plurality of rendering pipelines, the plurality of rendering pipelines comprising a graphical operation module and a rasterizer,
wherein the client transmits a rendering command to a graphics application at the server using a first transport protocol,
wherein a first rendering pipeline of the plurality of rendering pipelines renders graphical user interface (GUI) graphics data, and a second rendering pipeline of the plurality of rendering pipelines renders subject graphics data,
wherein the server composites, using the compositor, the rendered subject graphics data and the rendered GUI graphics data,
wherein the server transmits the composited subject graphics data using a second transport protocol, and
wherein the first transport protocol is more reliable than the second transport protocol.
9. A method by which a server communicates with a client, comprising:
receiving a rendering command from a client using a reliable transport protocol;
processing the rendering command to render graphical user interface (GUI) graphics data in a first rendering pipeline, and to render subject graphics data in a second rendering pipeline, wherein the first and second rendering pipelines comprise a graphical operation module and a rasterizer;
compositing, using a compositor, the rendered subject graphics data and the rendered GUI graphics data; and
transmitting the composited graphics data using an unreliable transport protocol.
10. A method by which a client communicates with a server, comprising:
transmitting a rendering command to a server using a reliable transport protocol, wherein the server processes the rendering command to render graphical user interface (GUI) graphics data in a first rendering pipeline, to render subject graphics data in a second rendering pipeline, and to composite, using a compositor, the rendered subject graphics data and the rendered GUI graphics data, wherein the first and second rendering pipelines comprise a graphical operation module and a rasterizer; and
receiving the composited graphics data from the server using an unreliable transport protocol.
11. A computer-readable storage device having computer-executable instructions stored thereon that, if executed by a computing device, cause the computing device to perform a method comprising:
receiving a rendering command from a client using a reliable transport protocol;
rendering graphical user interface (GUI) graphics data in a first rendering pipeline, and rendering subject graphics data in a second rendering pipeline, wherein the first and second rendering pipelines comprise a graphical operation module and a rasterizer;
compositing, using a compositor, the rendered subject graphics data and the rendered GUI graphics data; and
transmitting the composited graphics data using an unreliable transport protocol.
12. A computer-readable storage device having computer-executable instructions stored thereon that, if executed by a computing device, cause the computing device to perform a method comprising:
transmitting a rendering command to a server using a reliable transport protocol, wherein the server processes the rendering command to render graphical user interface (GUI) graphics data in a first rendering pipeline, to render subject graphics data in a second rendering pipeline, and to composite, using a compositor, the rendered subject graphics data and the rendered GUI graphics data, wherein the first and second rendering pipelines comprise a graphical operation module and a rasterizer; and
receiving the composited graphics data from the server using an unreliable transport protocol.
13. A system for rendering graphics data, the system comprising:
a processor for executing graphics application;
a rendering control module;
a compositor; and
a plurality of rendering pipelines comprising a graphical operation module and a rasterizer,
wherein the processor receives a rendering command for the application from a client using a reliable transport protocol,
wherein, in response to the rendering command, the application instructs the rendering control module to render graphical user interface (GUI) graphics data and subject graphics data, wherein a first rendering pipeline of the plurality of rendering pipelines renders the GUI graphics data, a second rendering pipeline of the plurality of rendering pipelines renders subject graphics data, and the compositor composites the rendered subject graphics data and the rendered GUI graphics data, and
wherein the rendering control module transmits the composited graphics data using an unreliable transport protocol.
14. A system for rendering graphics data, the system comprising:
a display device; and
a communication module,
wherein the communication module transmits, using a reliable transport protocol, a rendering command to a graphics application on a server for processing the rendering command to render graphical user interface (GUI) graphics data in a first rendering pipeline, to render subject graphics data in a second rendering pipeline, and to composite, using a compositor, the rendered subject graphics data and the rendered GUI graphics data, wherein the first and second rendering pipelines comprise a graphical operation module and a rasterizer,
wherein the communication module receives the rendered subject graphics data using an unreliable transport protocol, and
wherein the display device displays the rendered subject graphics data.
15. The system of claim 1 , the plurality of rendering pipelines further comprising a frame buffer.
16. The system of claim 1 , wherein the graphical operation module comprises a vertex operation module.
17. The system of claim 1 , wherein the graphical operation module comprises a pixel operation module.