Providing user interface (UI) elements having an ordered hierarchy of graphical control elements in virtual machine sessions at reduced latency and related methods

1. A computing system operable to run virtual machine sessions from a virtualization server and provide a hosted application, comprising:

at least one client computing device configured to access the hosted application during one of the virtual machine sessions with the virtualization server and receive therefrom a hierarchical tree of user interface (UI) elements that include an ordered hierarchy of graphical control elements when sequenced therethrough and are enumerated at a top-level as a root element in a control view using a tag index of each control element, wherein the at least one client computing device is configured to perform the following:

display some of the UI elements from the hierarchical tree locally as local virtual UI elements and based upon the received hierarchical tree, maintain the other UI elements hidden,

apply user input as a tab or arrow key entry to one of the displayed UI elements as a graphical control element, and

generate a local virtual UI element graphics overlay and display the local virtual UI element to which user input had been applied and one or more previously hidden UI elements as a graphical control element corresponding to a predicted response to the user input of the tab or arrow key entry;

send the user input of the tab or arrow key entry to said virtualization server; and

receive an updated UI element graphics from the virtualization server corresponding to an actual response to the user input of the tab or arrow key entry and replace at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics, and repeat the tab or arrow key entry for a next graphical control element in the ordered hierarchy of graphical control elements.

2. The computing system of claim 1 wherein said at least one client computing device comprises an input device, with the user input being based on at least one of a user selecting tab, button, menu, jump list, toggle and picker control via said input device.

3. The computing system of claim 1 wherein the next graphical control element in the ordered hierarchy after the graphical control element receiving the user input of the tab or arrow key entry is displayed as a highlighted rectangle on said at least one client computing device.

4. The computing system of claim 3 wherein the highlighting is based on at least one of a border and color.

5. The computing system of claim 1 wherein each graphical control element has data associated therewith, with the data being cached by said at least one client computing device.

6. The computing system of claim 5 wherein the caching by said at least one client computing device is performed when the corresponding graphical control element is in focus.

7. The computing system of claim 5 wherein the cached data includes at least one of encrypted text, list item, picker control item, menu item, jump list item, and start menu item.

8. The computing system of claim 1 wherein the local virtual UI element graphics overlay is deleted based on one or more of replacing said local virtual UI element graphics overlay with the received updated UI element graphics, hosted application process terminating, hosted application window closing, hosted application UI element getting out of focus, hosted application UI element becoming hidden, hosted application UI element being deleted, heuristics specific to receiving the hosted application UI element, and policies associated with displaying the received hosted application UI element.

9. The computing system of claim 1 wherein the local virtual UI element graphics overlay is generated based upon a determined latency by the virtualization server with the at least one client computing device as a latency standard deviation above predetermined levels or an average or median latency.

10. The computing system of claim 9 wherein the latency is based on at least one of an average, a median and a standard deviation of latencies between the virtualization server and said at least one client computing device.

11. The computing system of claim 1 wherein the at least one client computing device is configured to:

send the user input of the tab or arrow key entry to said virtualization server; and

receive an updated UI element graphics from the virtualization server corresponding to an actual response to the user input of the tab or arrow key entry and replace at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics, and repeat the tab or arrow key entry for a next graphical control element in the ordered hierarchy of graphical control elements.

12. The computing system of claim 11 wherein the replacement of the local virtual UI element graphics overlay with the received updated UI element graphics is performed by hiding the said local virtual UI element graphics overlay and unveiling the received updated UI element graphics underneath the local virtual UI element graphics overlay.

13. A method for operating virtual machine sessions provided by a virtualization server in communication with at least one client computing device of a computing system to provide a hosted application that includes user interface (UI) elements comprising an ordered hierarchy of graphical control elements when sequenced therethrough, wherein the virtualization server is configured to enumerate UI elements at a top-level as a root element in a control view where the relative tab order of enumerated controls are retrieved using a tag index of each control element in an enumerated order, and insert the enumerated control element and tab order in a hierarchical tree, the method comprising:

accessing the hosted application from at least one client computing device during one of the virtual machine sessions for receiving the hierarchical tree of the UI elements as an ordered hierarchy of graphical control elements, the client computing device operable to perform the following:

displaying some of the UI elements from the hierarchical tree locally as local virtual UI elements and based upon the received hierarchical tree, maintaining the other UI elements hidden,

applying user input as a tab or arrow key entry to one of the displayed UI elements as a graphical control element, and

generating a local virtual UI element graphics overlay and displaying the local virtual UI element to which user input had been applied and one or more previously hidden UI elements as graphical control elements corresponding to a predicted response to the user input of the tab or arrow key entry;

sending the user input of the tab or arrow key entry to the virtualization server, and

receiving updated UI element graphics from the virtualization server corresponding to an actual response to the user input, and replacing at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics, and repeating the tab or arrow key entry for the next graphical control element in the ordered hierarchy of graphical control elements.

14. The method of claim 13 wherein each graphical control element has data associated therewith, with the data being cached by the at least one client computing device.

15. The method of claim 14 wherein the caching by said at least one client computing device is performed when the corresponding graphical control element is in focus.

16. The method of claim 13 wherein the next graphical control element in the ordered hierarchy after the graphical control element receiving the user input of the tab or arrow key entry is displayed as a highlighted rectangle on said at least one client computing device.

17. The method of claim 13 wherein the local virtual UI element graphics overlay is deleted based on one or more of replacing the said local virtual UI element graphics overlay with the received updated UI element graphics, hosted application process terminating, hosted application window closing, hosted application UI element getting out of focus, hosted application UI element becoming hidden, hosted application UI element being deleted, heuristics specific to receiving the hosted application UI element, and policies associated with displaying the received hosted application UI element.

18. The method of claim 13 wherein the local virtual UI element graphics overlay are generated based upon a determined latency by the virtualization server with the at least one client computing device as a latency standard deviation above predetermined levels or an average or median latency.

19. The method of claim 13 wherein the client computing device is operable for:

sending the user input of the tab or arrow key entry to the virtualization server, and

receiving updated UI element graphics from the virtualization server corresponding to an actual response to the user input, and replacing at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics, and repeating the tab or arrow key entry for the next graphical control element in the ordered hierarchy of graphical control elements.

20. A non-transitory computer readable medium for operating virtual machine sessions provided by a virtualization server in communication with at least one client computing device and providing a hosted application that includes user interface (UI) elements comprising an ordered hierarchy of graphical control elements when sequenced therethrough, wherein the virtualization server is configured to enumerate UI elements at a top-level as a root element in a control view where the relative tab order of enumerated controls are retrieved using a tag index of each control element in an enumerated order, and insert the enumerated control element and tab order in a hierarchical tree, and with the non-transitory computer readable medium having a plurality of computer executable instructions for causing the at least one client computing device to perform steps comprising:

accessing the host application during one of the virtual machine sessions for receiving the hierarchical tree of the UI elements as an ordered hierarchy of graphical control elements,

displaying some of the UI elements from the hierarchical tree locally as local virtual UI elements and based upon the received hierarchical tree, maintaining the other UI elements hidden,

applying user input as a tab or arrow key entry to one of the displayed UI elements as a graphical control element, and

generating a local virtual UI element graphics overlay and displaying the local virtual UI element to which user input had been applied and one or more previously hidden UI elements as graphical control elements corresponding to a predicted response;

sending the user input to the virtualization server, and

receiving updated UI element graphics from the virtualization server corresponding to an actual response to the user input, and replacing at least a portion of the local virtual UI element graphics overlay with the received updated UI element graphics, and repeating the tab or arrow key entry for the next graphical control element in the ordered hierarchy of graphical control elements.