Method and system for low-latency transfer protocol
View Patent ↗A method and system for providing computer-generated output and in particular graphical output. An output capturing and encoding engine is configured to intercept graphical output from an application on a server, organize the output into regions having similar motion and/or graphical characteristics, and convert the data from each region into a format suitable to balance transmission efficiencies versus display quality or capability at the receiving end.
1. A system for providing a low-latency transfer protocol, the system comprising:
a server in communication with at least one client over a digital computer network;
an application program instantiated on the server, wherein the application provides computer-generated output;
wherein the server is configured to implement the low-latency transfer protocol including being configured to:
intercept the computer-generated output from the application program;
identify a plurality of regions within the computer generated output from the application program;
for each specific region of the plurality of regions, determine and select a specific format tending to decrease latency in network transmission from a plurality of formats including pass-through, graphical commands, and video, the selected format selected based on criteria balancing network performance against display performance;
for each specific region, converting corresponding computer generated output from the application program into the corresponding selected specific format, thereby generating a converted output of one or more different formats from the computer generated output; and
transmit the converted output over the digital computer network to the at least one client.
2. The system of claim 1 , wherein the server is further configured to execute a congestion control module for minimizing latency and optimizing use of available bandwidth.
3. The system of claim 1 ,
wherein the at least one client is configured to
execute, decode, and render graphics and video;
render the graphical output in response to receiving transmitted converted output;
intercept user inputs; and
transmit the intercepted user inputs to the server over the network.
4. The system of claim 3 , wherein the server is further configured to
receive the intercepted user inputs from the client, and
provide the intercepted user inputs to the application for processing.
5. The system of claim 4 , wherein the server is further configured to smooth border areas between the plurality of regions.
6. The system of claim 1 , wherein the graphical commands are represented by a universal intermediary graphical command language.
7. The system of claim 6 , wherein the server is further configured to convert the graphical output into video codec data utilizing at least one of a localized motion detection routine and a motion estimation routine, wherein the server divides the graphical output into a plurality of region sets, each region set sharing similar motion characteristics.
8. The system of claim 7 , wherein the server is configured to compute a motion score for each region set, wherein the motion scores represent motion characteristics.
9. The system of claim 1 , wherein the server is further configured to pre-process the graphical output before converting by selecting regions with graphical activity for processing.
10. The system of claim 9 , wherein the pre-processing further includes overlaying a mask on the graphical output to obscure static regions.
11. A server computer system implementing a low latency transfer protocol, the server computer system comprising:
a server in communication with at least one client over a digital computer network; an application program instantiated on the server, wherein the application provides computer-generated output; wherein the server is configured to implement the low-latency transfer protocol including being configured to: intercept the computer-generated output from the application program;
identify a plurality of regions within the computer generated output from the application program;
for each specific region of the plurality of regions, determine and select a specific format tending to decrease latency in network transmission from a plurality of formats including pass-through, graphical commands, and video, the selected format selected based on criteria balancing network performance against display performance; for each specific region, converting corresponding computer generated output from the application program into the corresponding selected specific format, thereby generating a converted output of one or more different formats from the computer generated output; and
transmit the converted output over the digital computer network to the at least one client.