System, method, and graphical user interface for controlling an application executing on a server
A system, method, and graphical user interface for playing games and/or executing applications on a tablet-based client. One embodiment of a graphical user interface (GUI) for playing a video game on a tablet-based client device comprises: a virtual controller rendered on a display of the tablet computer, the virtual controller substantially mimicking the control provided by a thumb stick of a physical game controller and providing omnidirectional, free-form movement in a synchronous direction in which a user moves a finger on the display of the tablet-based client.
1. A method, comprising:
receiving selection of an online application at a hosting service, the selection received from a remote client device;
executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating frames of streaming video content for the online application for onward transmission to the remote client device;
identifying graphical user interface elements used for interacting with the streaming video content of the online application, wherein at least a part of the graphical user interface elements are executing on the server of the hosting service; and
encoding the graphical user interface elements for onward transmission to the remote client device for rendering,
wherein the encoded graphical user interface elements are decoded at the remote client device and rendered on a display of the remote client device, the graphical user interface elements map to at least some controls of a physical game controller, and
wherein inputs provided at each of the graphical user interface elements rendered at the remote client device are interpreted by the server of the hosting service to control a state of the online application.
2. The method of claim 1 , wherein the graphical user interface elements are identified in part by retrieving virtual user interface configuration defined for the online application, the virtual user interface configuration including the graphical user interface elements used for interacting with the online application.
3. The method of claim 2 , wherein the virtual user interface configuration is configured to include select ones of the graphical user interface elements based on a state of the online application.
4. The method of claim 2 , wherein the virtual user interface configuration is encoded with the frames of the streaming video content, the virtual user interface configuration includes a user interface defined for the display of the remote client device, the user interface used for rendering the graphical user interface elements.
5. The method of claim 1 , wherein the graphical user interface elements are decoded at the remote client device and rendered as an overlay over the frames of the streaming video content.
6. The method of claim 5 , wherein the graphical user interface elements are configured to be transparent or semi-transparent so as to allow viewing of the streaming video content rendering beneath the overlay.
7. The method of claim 1 , wherein each one of the graphical user interface elements is configured to include one or more distinct settings, the one or more distinct settings being specified by a user viewing the streaming video content, or by a developer of the online application, or by the hosting service.
8. The method of claim 7 , wherein the one or more distinct settings specified for each graphical user interface element includes a first setting for an inactive state and a second setting for an active state, and
wherein the one or more distinct settings include activation setting, or transparency setting, or brightness setting, or sensitivity setting, or color setting, or scale setting, or size setting, or location setting, or operational mode setting, or operational direction setting, or graphical user interface element type setting, or any two or more combinations thereof.
9. The method of claim 7 , wherein the one or more distinct settings are configured for the display of the remote client device on which the streaming video content and the graphical user interface elements are rendered.
10. The method of claim 1 , wherein the graphical user interface elements are defined as a set of widgets, wherein each widget in the set of widgets is associated with a particular graphical user interface element configured to provide particular control action of the physical game controller.
11. The method of claim 10 , wherein a part of each widget executes locally at the remote client and a remaining part executes on the server of the hosting service.
12. The method of claim 1 , wherein the online application is a video game, and a number and types of the graphical user interface elements are based on types of control actions needed for interacting with streaming video content of the video game.
13. The method of claim 1 , wherein the graphical user interface elements are specific for the online application and for the display of the remote client device.
14. The method of claim 1 , wherein the graphical user interface elements include a virtual controller with controls that mimic controls of a physical controller.
15. The method of claim 14 , wherein the virtual controller includes user specific adjustments provided by a user during previous interactive sessions of the online application.
16. The method of claim 15 , wherein the user specific adjustment identifies configuration parameters used for rendering the virtual controller on the user interface rendered at the remote client device.
17. The method of claim 1 , wherein the graphical user interface elements include user specific adjustments provided by a user during previous interactive sessions of the online application, wherein the user specific adjustments identifies configuration parameters used for rendering the graphical user interface elements on the display of the remote client device of the user.
18. The method of claim 1 , further includes,
receiving user specific adjustment to one or more of the graphical user interface elements from a user during a current session, the adjustment to the one or more of the graphical user interface elements updated to a virtual user interface configuration stored in a database of the hosting service and used during subsequent sessions of the online application.
19. A non-transitory computer readable storage medium having program instructions, which when executed by a processor of a computing device of a hosting service performs a method, the program instructions comprising:
program instructions for receiving selection of an online application at a hosting service, the selection received from a remote client device;
program instructions for executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating frames of streaming video content for the online application for onward transmission to the remote client device;
program instructions for identifying graphical user interface elements used for interacting with the streaming video content of the online application, wherein at least a part of the graphical user interface elements are executing on the server of the hosting service; and
program instructions for encoding the graphical user interface elements for onward transmission to the remote client device for rendering,
wherein the encoded graphical user interface elements are decoded at the remote client device and rendered on a display of the remote client device, the graphical user interface elements map to at least some controls of a physical game controller, and
wherein inputs provided at each of the graphical user interface elements rendered at the remote client device are interpreted by the processor of the hosting service to control a state of the online application.