Providing user interface (UI) elements having scrollable content in virtual machine sessions at reduced latency and related methods

1. A computing system operable to run virtual machine sessions from a virtualization server and provide a hosted application, the computing system comprising:

at least one client computing device configured to access the hosted application during a virtual machine session and receive a hierarchical tree of user interface (UI) elements having vertically or horizontally scrollable content that includes text and graphics, and configured to perform the following:

display some of the UI elements from the hierarchical tree as at least a portion of the vertically or horizontally scrollable content as local virtual UI elements and based upon the received hierarchical tree, maintain hidden the other UI elements as scrollable content,

apply user input to scroll a portion of the displayed vertically or horizontally scrollable content,

cache the hierarchical tree of scrollable content, and

generate a local virtual UI element graphics overlay and display the local virtual UI elements to which user input had been selected and scrolled and one or more previously hidden UI elements as scrollable content corresponding to a predicted response to the user input;

send the user input to said virtualization server, and

receive from the virtualization server an updated UI element graphics that includes scrollable content corresponding to an actual response to the user input and replace at least a portion of the displayed local virtual UI element graphics overlay that includes at least a portion of the cached scalable content with the received updated UI element graphics.

2. The computing system of claim 1 further comprising metadata associated with the vertically or horizontally scrollable content.

3. The computing system of claim 2 wherein the vertically or horizontally scrollable content includes at least one of grid item, table item, spreadsheet item, list item, and data item.

4. The computing system of claim 2 wherein the metadata includes encrypted text.

5. The computing system of claim 2 wherein the metadata includes at least one of UI automation properties and events retrieved from the hosted application.

6. The computing system of claim 1 wherein the vertically or horizontally scrollable content includes the text and graphics and a scroll pattern and at least one of a text pattern, grid pattern, table pattern, spreadsheet pattern, and a list pattern.

7. The computing system of claim 1 wherein the local virtual UI element graphics overlay is deleted based on one or more of replacing said local virtual UI element graphics overlay with the received updated UI element graphics, hosted application process terminating, hosted application window closing, hosted application UI element getting out of focus, hosted application UI element becoming hidden, hosted application UI element being deleted, heuristics specific to receiving the hosted application UI element, and policies associated with displaying the received hosted application UI element.

8. The computing device of claim 1 wherein the at least one client computing device is configured to access the hosted application based upon a determined latency by the virtualization server with the at least one client computing device as a latency standard deviation above predetermined levels or an average or median latency.

9. The computing device of claim 1 wherein the at least one client computing device is configured to receive an updated UI element graphics that includes scrollable content corresponding to an actual response to the user input and replace at least a portion of the displayed local virtual UI element graphics overlay that includes at least a portion of the cached scalable content with the received updated UI element graphics.

10. The computing system of claim 9 wherein the replacement of the at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics is performed by hiding the said at least a portion of the local virtual UI element graphics overlay and unveiling the said received updated UI element graphics underneath the said at least a portion of the local virtual UI element graphics overlay.

11. A method for operating virtual machine sessions provided by a virtualization server in communication with at least one client computing device to provide a hosted application that includes user interface (UI) elements having vertically or horizontally scrollable content that includes text and graphics, the method comprising:

accessing the hosted application from at least one client computing device during one of the virtual machines sessions for receiving a hierarchical tree of the UI elements having the vertically or horizontally scrollable content, the client computing device operable to perform the following:

displaying some of the UI elements from the hierarchical tree as at least a portion of the vertically or horizontally scrollable content as local virtual UI elements, and based upon the received hierarchical tree maintaining hidden the other UI elements as scrollable content,

applying user input to scroll a portion of the displayed vertically or horizontally scrollable content,

caching the hierarchical tree of scrollable content, and

generating a local virtual UI element graphics overlay and displaying the local virtual UI elements to which user input had been selected and scrolled and one or more previously hidden UI elements as scrollable content corresponding to a predicted response to the user input;

sending the user input to said virtualization server, and

receiving updated UI element graphics from the virtualization server that includes scrollable content corresponding to an actual response to the user input and replacing at least a portion of the displayed local virtual UI element graphics overlay that includes at least a portion of the cached scrollable content with the received updated UI element graphics.

12. The method of claim 11 further comprising metadata associated with the scrollable content.

13. The method of claim 12 wherein the metadata includes encrypted text.

14. The method of claim 12 wherein the metadata includes at least one of UI automation properties and events retrieved from the hosted application.

15. The method of claim 11 wherein the vertically or horizontally scrollable content includes the text and graphics and a scroll pattern and at least one of a text pattern, grid pattern, table pattern, spreadsheet pattern, and a list pattern.

16. The method of claim 11 wherein the local virtual UI element graphics overlay is deleted based on one or more of replacing the said local virtual UI element graphics overlay with the received updated UI element graphics, hosted application process terminating, hosted application window closing, hosted application UI element getting out of focus, hosted application UI element becoming hidden, hosted application UI element being deleted, heuristics specific to receiving the hosted application UI element, and policies associated with displaying the received hosted application UI element.

17. The method of claim 11 wherein the local virtual UI element graphics overlay is generated based upon a determined latency by the virtualization server with the at least one client computing device as a latency standard deviation above predetermined levels or an average or median latency.

18. The method of claim 11 wherein the at least one client computing device is operative for receiving updated UI element graphics from the virtualization server that includes scrollable content corresponding to the actual response to the user input and replacing at least a portion of the displayed local virtual UI element graphics overlay that includes at least a portion of the cached scrollable content with the received updated UI element graphics.

19. The method of claim 18 wherein the replacement of the at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics is performed by hiding the said at least a portion of the local virtual UI element graphics overlay and unveiling the said received updated UI element graphics underneath the said at least a portion of the local virtual UI element graphics overlay.

20. A non-transitory computer readable medium for operating virtual machine sessions provided by a virtualization server in communication with at least one client computing device and providing a hosted application that includes user interface (UI) elements having vertically or horizontally scrollable content that includes text and graphics that are displayed at the at least one client computing device, and with the non-transitory computer readable medium having a plurality of computer executable instructions for causing the at least one client computing device to perform steps comprising:

accessing the hosted application during one of the virtual machines sessions for receiving a hierarchical tree of the UI elements having the vertically or horizontally scrollable content;

displaying some of the UI elements from the hierarchical tree as at least a portion of the vertically or horizontally scrollable content as local virtual UI elements, and based upon the received hierarchical tree maintaining hidden the other UI elements as scrollable content;

applying user input to scroll a portion of the displayed vertically or horizontally scrollable content;

caching the hierarchical tree of scrollable content, and

generating a local virtual UI element graphics overlay and displaying the local virtual UI elements to which user input had been selected and scrolled and one or more previously hidden UI elements as scrollable content corresponding to a predicted response to the user input;

sending the user input to said virtualization server, and

receiving updated UI element graphics from the virtualization server that includes scrollable content corresponding to an actual response to the user input and replacing at least a portion of the displayed local virtual UI element graphics overlay that includes at least a portion of the cached scrollable content with the received updated UI element graphics.