Method and system for enhancing user experience for remoting technologies

1. A method, comprising:

executing, on a client device in communication with a remoting server, an image enhancement application, the image enhancement application configured to perform the following three image enhancement techniques:

predicting a frame based on previously received frames;

interpolating a frame based on at least two buffered frames; and

reducing appearance of artifacts in a received frame;

the image enhancement application further configured to:

monitor a sequence of frame updates for a display screen received at the client device from the remoting server;

observe the available network bandwidth, frame refresh rate, and image quality corresponding to the received sequence of frame updates; and

select one or more of the three image enhancement techniques to apply on the client device based on the observed network bandwidth, frame refresh rate, and/or image quality being sufficiently low.

2. The method of claim 1 , wherein the image enhancement application is further configured to:

determine a type of content distributed by the remoting server; and

wherein selecting an image enhancement technique is further based on the type of content distributed by the remoting server.

3. The method of claim 1 , wherein the image enhancement application is further configured to:

determine if the received frame updates are associated with a response to a user interaction with a user-interface object and in response select the image enhancement technique that comprises predicting a frame based on previously received frames; and

determine if the received frame updates are associated with a streamed video and in response select the image enhancement technique that comprises interpolating a frame based on at least two buffered frames.

4. The method of claim 1 , wherein predicting the frame based on previously received frames comprises the image enhancement application detecting a user interaction with a user-interface object and using information associated with the user interaction to assist predicting the frame; and

wherein the user interaction with the user-interface object comprises at least one of:

a window-scrolling operation;

a window-resizing operation;

a text entry into a text field;

a mouse-pointer movement;

an button click; and

a drag operation that moves the object to a new screen position.

5. The method of claim 1 , wherein the image enhancement application is further configured to:

determine whether a selected image enhancement technique sufficiently improves user experience; and

in response to the image enhancement technique not sufficiently improving user experience, discontinue application of the image enhancement technique.

6. The method of claim 1 , wherein reducing appearance of artifacts in the received frame comprises the image enhancement application:

detecting an edge in the frame;

determining whether the edge belongs to a user-interface object or is caused by artifacts;

in response to the edge being caused by artifacts, blurring the edge; and

in response to the belonging to a user-interface object, preserving the edge.

7. The method of claim 6 , wherein determining whether the edge belongs to the user-interface object or is caused by artifacts comprises:

determining whether a color variance along the edge is greater than a predetermined threshold; and

in response to the color variance being greater than the threshold, identifying the edge as being caused by artifacts.

8. A non-transitory storage medium storing instructions which when executed by a processor cause the processor to perform a method, the method comprising:

executing, on a client device in communication with a remoting server, an image enhancement application, the image enhancement application configured to perform the following three image enhancement techniques:

predicting a frame based on previously received frames;

interpolating a frame based on at least two buffered frames; and

reducing appearance of artifacts in a received frame;

the image enhancement application further configured to:

monitor a sequence of frame updates for a display screen received at the client device from the remoting server;

observe the available network bandwidth, frame refresh rate, and image quality corresponding to the received sequence of frame updates; and

select one or more of the three image enhancement techniques to apply on the client device based on the observed network bandwidth, frame refresh rate, and/or image quality being sufficiently low.

9. The non-transitory storage medium of claim 8 , wherein the image enhancement application is further configured to:

determine a type of content distributed by the remoting server; and

wherein selecting an image enhancement technique is further based on the type of content distributed by the remoting server.

10. The non-transitory storage medium of claim 8 , wherein the image enhancement application is further configured to:

determine if the received frame updates are associated with a response to a user interaction with a user-interface object and in response select the image enhancement technique that comprises predicting a frame based on previously received frames; and

determine if the received frame updates are associated with a streamed video and in response select the image enhancement technique that comprises interpolating a frame based on at least two buffered frames.

11. The non-transitory storage medium of claim 8 , wherein predicting the frame based on previously received frames comprises the image enhancement application detecting a user interaction with a user-interface object and using information associated with the user interaction to assist predicting the frame; and

wherein the user interaction with the user-interface object comprises at least one of:

a window-scrolling operation;

a window-resizing operation;

a text entry into a text field;

a mouse-pointer movement;

an button click; and

a drag operation that moves the object to a new screen position.

12. The non-transitory storage medium of claim 8 , wherein the image enhancement application is further configured to:

determine whether a selected image enhancement technique sufficiently improves user experience; and

in response to the image enhancement technique not sufficiently improving user experience, discontinue application of the image enhancement technique.

13. The non-transitory storage medium of claim 8 , wherein reducing appearance of artifacts in the received frame comprises the image enhancement application:

detecting an edge in the frame;

determining whether the edge belongs to a user-interface object or is caused by artifacts;

in response to the edge being caused by artifacts, blurring the edge; and

in response to the belonging to a user-interface object, preserving the edge.

14. The non-transitory storage medium of claim 13 , wherein determining whether the edge belongs to the user-interface object or is caused by artifacts comprises:

determining whether a color variance along the edge is greater than a predetermined threshold; and

in response to the color variance being greater than the threshold, identifying the edge as being caused by artifacts.

15. A computer system, comprising:

a processor;

memory coupled to the processor; and

a storage device storing instructions which when loaded into the memory and executed by the processor cause the processor to perform a method, the method comprising:

executing, on a client device in communication with a remoting server, an image enhancement application, the image enhancement application configured to perform the following three image enhancement techniques:

predicting a frame based on previously received frames;

interpolating a frame based on at least two buffered frames; and

reducing appearance of artifacts in a received frame;

the image enhancement application further configured to:

monitor a sequence of frame updates for a display screen received at the client device from the remoting server;

observe the available network bandwidth, frame refresh rate, and image quality corresponding to the received sequence of frame updates; and

select one or more of the three image enhancement techniques to apply on the client device based on the observed network bandwidth, frame refresh rate, and/or image quality being sufficiently low.

16. The computer system of claim 15 , wherein the image enhancement application is further configured to:

determine a type of content distributed by the remoting server; and

wherein selecting an image enhancement technique is further based on the type of content distributed by the remoting server.

17. The computer system of claim 15 , wherein the image enhancement application is further configured to:

determine if the received frame updates are associated with a response to a user interaction with a user-interface object and in response select the image enhancement technique that comprises predicting a frame based on previously received frames; and

determine if the received frame updates are associated with a streamed video and in response select the image enhancement technique that comprises interpolating a frame based on at least two buffered frames.

18. The computer system of claim 15 , wherein predicting the frame based on previously received frames comprises the image enhancement application detecting a user interaction with a user-interface object and using information associated with the user interaction to assist predicting the frame; and

wherein the user interaction with the user-interface object comprises at least one of:

a window-scrolling operation;

a window-resizing operation;

a text entry into a text field;

a mouse-pointer movement;

an button click; and

a drag operation that moves the object to a new screen position.

19. The computer system of claim 15 , wherein the image enhancement application is further configured to:

determine whether a selected image enhancement technique sufficiently improves user experience; and

in response to the image enhancement technique not sufficiently improving user experience, discontinue application of the image enhancement technique.

20. The computer system of claim 15 , wherein reducing appearance of artifacts in the received frame comprises the image enhancement application:

detecting an edge in the frame;

determining whether the edge belongs to a user-interface object or is caused by artifacts;

in response to the edge being caused by artifacts, blurring the edge; and

in response to the belonging to a user-interface object, preserving the edge.