Systems and methods for remote computing session display based upon user input event prioritization
A computing system may include a server configured to host virtual computing sessions, and a client device. The client device may be configured to remotely access a virtual computing session from the server, and receive user input data associated with the virtual computing session and classify the data into first (higher priority) and second (lower priority) data packets. The client device may also be configured to send the first data packets to the server via a first virtual channel, and send the second data packets to the server via a second virtual channel having a higher packet loss rate associated therewith than the first virtual channel. The server may be configured to assemble the second data packets to reconstruct and inject the user input data into the virtual computing session based upon the first data packets.
1. A computing system comprising:
a server configured to host virtual computing sessions; and
a client device configured to
remotely access a virtual computing session from the server,
receive user input data associated with the virtual computing session including a group of data packets,
divide the group of data packets into first and second data packets, where the first data packets are critical data packets and the second data packets are non-critical data packets,
send the first data packets to the server via a reliable virtual channel, and
send the second data packets to the server via a lossy virtual channel having a higher packet loss rate associated therewith than the reliable virtual channel;
wherein the server is configured to assemble the first and second data packets to reconstruct and inject the user input data into the virtual computing session with reference to a first packet and a last packet of the first data packets, the first and last packets being indicative of the start and end of the user input data.
2. The computing system of claim 1 wherein the client device has a keyboard associated therewith, and wherein the first data packets correspond to keyboard events input via the keyboard.
3. The computing system of claim 1 wherein the client device has a touch screen associated therewith, and wherein the first data packets correspond to touch-up and touch-down events input via the touch screen; and wherein the second data packets correspond to touch-move events input via the touch screen.
4. The computing system of claim 1 wherein the client device has a mouse associated therewith, and wherein the first data packets correspond to mouse-button-up and mouse-button-down events input via the mouse; and wherein the second data packets correspond to mouse-drag and mouse-wheel events input via the mouse.
5. The computing system of claim 1 wherein the client device has an electronic pen associated therewith, and wherein the first data packets correspond to pen-up and pen-down events input via the electronic pen; and wherein the second data packets correspond to pen-drag events input via the electronic pen.
6. The computing system of claim 1 wherein the client device is further configured to send at least some of the first data packets via the lossy virtual channel.
7. The computing system of claim 1 wherein the client device is further configured to apply forward error correction (FEC) to the first and second data packets responsive to a packet loss greater than a threshold level.
8. The computing system of claim 7 wherein the client device is configured to apply FEC to the first and second data packets after a delay period.
9. The computing system of claim 7 wherein the client device is further configured to selectively change a level of FEC applied to at least one of the first and second data packets based upon changes in data packet loss.
10. A client device comprising:
a memory and a processor cooperating with the memory to
remotely access a virtual computing session from a server,
receive user input data associated with the virtual computing session including a group of data packets,
divide the group of data packets into first and second data packets, where the first data packets are critical data packets and the second data packets are non-critical data packets,
send the first data packets to the server via a reliable virtual channel, and
send the second data packets to the server via a lossy virtual channel having a higher packet loss rate associated therewith than the reliable virtual channel for the server to assemble the first and second data packets to reconstruct and inject the user input data into the virtual computing session with reference to a first packet and a last packet of the first data packets, the first and last packets being indicative of the start and end of the user input data.
11. The client device of claim 10 wherein the first data packets correspond to keyboard events input via a keyboard.
12. The client device of claim 10 wherein the first data packets correspond to touch-up and touch-down events input via a touch screen; and wherein the second data packets correspond to touch-move events input via the touch screen.
13. The client device of claim 10 wherein the first data packets correspond to mouse-button-up and mouse-button-down events input via a mouse; and wherein the second data packets correspond to mouse-drag and mouse-wheel events input via the mouse.
14. The client device of claim 10 wherein the first data packets correspond to pen-up and pen-down events input via an electronic pen; and wherein the second data packets correspond to pen-drag events input via the electronic pen.
15. A method comprising:
remotely accessing a virtual computing session at a client device from a server;
receiving user input data at the client device associated with the virtual computing session including a group of data packets,
dividing the group of data packets into first and second data packets, where the first data packets are critical data packets and the second data packets are non-critical data packets;
sending the first data packets from the client device to the server via a reliable virtual channel, and
sending the second data packets from the client device to the server via a lossy virtual channel having a higher packet loss rate associated therewith than the reliable virtual channel to enable the server to assemble the first and second data packets to reconstruct and inject the user input data into the virtual computing session with reference to a first packet and a last packet of the first data packets, the first and last packets being indicative of the start and end of the user input data.
16. The method of claim 15 wherein the first data packets correspond to keyboard events input via a keyboard.
17. The method of claim 15 wherein the first data packets correspond to touch-up and touch-down events input via a touch screen; and wherein the second data packets correspond to touch-move events input via the touch screen.
18. The method of claim 15 wherein the first data packets correspond to mouse-button-up and mouse-button-down events input via a mouse; and wherein the second data packets correspond to mouse-drag or mouse-wheel events input via the mouse.
19. The method of claim 15 wherein the first data packets correspond to pen-up and pen-down events input via an electronic pen; and wherein the second data packets correspond to pen-drag events input via the electronic pen.
20. A computing device comprising:
a memory and a processor cooperating with the memory to
host virtual computing sessions to be remotely accessed by a client device, the client device being configured to receive user input data associated with the virtual computing session including a group of data packets and divide the group of data packets into first and second data packets, wherein the first data packets are critical data packets and the second data packets are non-critical data packets,
receive the first data packets from the client device via a reliable virtual channel,
receive the second data packets from the client device via a lossy virtual channel having a higher packet loss rate associated therewith than the reliable virtual channel, and
assemble the first and second data packets to reconstruct and inject the user input data into the virtual computing session with reference to a first packet and a last packet of the first data packets, the first and last packets being indicative of the start and end of the user input data.
21. The computing device of claim 20 wherein the processor is further configured to also receive at least some of the first data packets from the client device via the lossy virtual channel.