Method and system for low-latency transfer protocol
View Patent ↗A method and system for providing computer-generated output and in particular graphical output. The system includes a network configured to carry digital information. The system includes a server in communication with the network, the server configured to execute an application and a cloud engine module. The application provides a graphical output. The output capturing and encoding engine module is further configured to intercept the graphical output from the application on the server. The output capturing and encoding engine module is further configured to convert the graphical output into at least one of: graphical commands and video codec data. The output capturing and encoding engine module is further configured to transmit the converted output over the network. The system includes a client in communication with the server over the network, the client configured to execute a graphics and video decoding and rendering engine module. The graphics and video decoding and rendering engine module is configured to, responsive to receiving the transmitted converted output, rendering the graphical output. The graphics and video decoding and rendering engine module is configured to intercept graphics and video decoding and rendering inputs at the client. The graphics and video decoding and rendering engine module is configured to transmit the intercepted user inputs to the output capturing and encoding engine module.
1. A computer implemented method, comprising:
intercepting graphical output from an executing application;
converting the intercepted output into at least one of graphical commands and video codec data, the converting including:
dividing the graphical output into a plurality of regions;
converting graphical output associated with each region;
smoothing a border area between a plurality of regions;
wherein the intercepted output is further converted into at least one of picture codec data and pass-through data, and the graphical output includes full-motion video, wherein the full-motion video is enclosed within one of the plurality of regions; and
transmitting the converted output over a network to a client, wherein transmitting comprises
parsing the converted output into a reliable queue, an unreliable queue, and an unreliable with forward error correction queue, wherein each queue has a plurality of channels, each channel associated with a priority;
transmitting the reliable queue, wherein each packet in the reliable queue is acknowledged by the client and includes a reliable packet sequence number;
transmitting the unreliable queue, wherein each packet in the unreliable queue includes an unreliable packet sequence number; and
transmitting the unreliable with forward error correction queue, wherein each packet in the unreliable with forward error correction queue includes a transform code containing redundant information to recover lost packets.
2. The computer implemented method of claim 1 , wherein the graphical commands are represented by a universal intermediary graphical command language.
3. The computer implemented method of claim 1 , wherein the reliable queue is used to transmit the graphical commands and the unreliable queue is used to transmit the video codec data.
4. The computer implemented method of claim 1 , wherein transmitting the converted output further includes throttling transmission bandwidth based on a congestion control.
5. The computer implemented method of claim 1 , further comprising:
receiving intercepted user inputs, and
providing the intercepted user inputs to the executing application for processing.
6. A computer implemented method, comprising:
receiving a converted output over a network from a server, wherein the converted output includes at least one of graphical commands. video codec data, picture codec data and pass-through data and wherein the graphical output is divided into a plurality of regions. and the graphical output associated with each region is converted into at least one of: graphical commands, video codec data, picture codec data and pass-through data wherein the graphical output includes full-motion video, wherein the full-motion video is enclosed within one of the plurality of regions, the server executing an application providing graphical output and converting the graphical output into the converted output, wherein the receiving comprises
receiving a reliable queue, wherein each packet in the reliable queue includes a reliable packet sequence number, the receiving a reliable queue further including transmitting an acknowledgement to the server in response to receiving a reliable packet queue;
receiving an unreliable queue, wherein each packet in the unreliable queue includes an unreliable packet sequence number;
receiving an unreliable with forward error correction queue, wherein each packet in the unreliable with forward error correction queue includes a transform code containing redundant information to recover lost packets; and
compiling the reliable queue, unreliable queue, and unreliable with forward error correction queue into the received converted output;
rendering a graphical output from the received converted output, and
transmitting the intercepted user inputs to the server in response to intercepting user inputs.
7. The computer implemented method of claim 6 , wherein the user inputs are transmitted to the server via the reliable queue.
8. The computer implemented method of claim 6 , wherein the receiving further includes processing by a jitter buffer.
9. The computer implemented method of claim 6 , wherein each queue is associated with a plurality of channels, and each channel processes incoming data with a jitter buffer.