System, method, and graphical user interface for controlling an application executing on a server
A system, method, and graphical user interface for playing games and/or executing applications on a tablet-based client. One embodiment of a graphical user interface (GUI) for playing a video game on a tablet-based client device comprises: a virtual controller rendered on a display of the tablet computer, the virtual controller substantially mimicking the control provided by a thumb stick of a physical game controller and providing omnidirectional, free-form movement in a synchronous direction in which a user moves a finger on the display of the tablet-based client.
1. A method, comprising:
receiving selection of an online application from a client device, the selection received at a hosting service that is located remotely from the client device;
executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating streaming video content for the online application for transmission to the client device;
identifying a configuration of a virtual controller used for interacting with the online application, the identified configuration of the virtual controller includes a user interface defined for a display of the client device and includes user specific adjustments to the configuration provided by a user selecting the online application;
encoding the configuration of the virtual controller with frames of streaming video content for onward transmission to the client device,
wherein a processor of the client device is configured to decompress the frames and render the user interface of the virtual controller with the streaming video content at the display of the client device, the user interface renders interactive controls that mimic controls of a physical game controller, the interactive controls of the virtual controller used to provide inputs to the online application;
receiving inputs provided at the interactive controls of the virtual controller of the client device, the inputs interpreted in accordance to the user specific adjustments and used to control a state of the online application,
wherein operations of the method are executed by a processor of the server at the hosting service.
2. The method of claim 1 , wherein the configuration of the virtual controller defined for the client device is stored in a database of the hosting service and the user specific adjustments included in the configuration are provided by a user during previous interactive sessions of the online application.
3. The method of claim 2 , wherein the configuration of the virtual controller adjusted by the user during a current session, the adjustment to the configuration updated to the configuration of the virtual controller stored in the database for use during subsequent sessions of the online application.
4. The method of claim 1 , wherein the user specific adjustment causes rendering of the interactive image of the virtual controller in a specific region of the display of the client device.
5. The method of claim 1 , wherein the configuration of the virtual controller specifies a size of the virtual controller and a region on the user interface for rendering the virtual controller, wherein the size and the region being configurable by a user.
6. The method of claim 1 , wherein the configuration of the virtual controller specifies a rendering state of the virtual controller when rendered on the user interface, wherein the rendering state of the virtual controller is one of transparent state and semi-transparent state.
7. The method of claim 1 , wherein the configuration of the virtual controller specifies sensitivity index for different controls of the virtual controller.
8. The method of claim 1 , wherein the user specific adjustments include rendering the image of the virtual controller in a relative mode or an absolute mode, wherein in the relative mode the interactive image of the virtual controller is drawn on the user interface at an initial touch point and the user input on the display is determined in relation to the initial touch point, and
wherein in the absolute mode the interactive image of the virtual controller is drawn in a fixed location on the user interface, wherein the fixed location is determined based on the user specific adjustment to the configuration.
9. The method of claim 1 , wherein the interactive controls included in the user interface renders interactive image of the virtual controller that includes graphics for buttons that are configurable on a per user basis.
10. The method of claim 1 , wherein the interactive controls included in the user interface render interactive image of the virtual controller that includes graphics for buttons that are configurable based on content of the online application.
11. A non-transitory computer readable storage medium having program instructions, which when executed by a processor of a computing device of a hosting service performs a method, the program instructions comprising:
program instructions for receiving selection of an online application from a client device, the selection received at the hosting service that is located remotely from the client device;
program instructions for executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating streaming video content for the online application for transmission to the client device;
program instructions for identifying a configuration of a virtual controller used for interacting with the online application, the identified configuration of the virtual controller includes a user interface defined for a display of the client device and includes user specific adjustments to the configuration provided by a user selecting the online application;
program instructions for encoding the configuration of the virtual controller with frames of streaming video content for onward transmission to the client device,
wherein a processor of the client device is configured to decompress the frames and render the user interface of the virtual controller with the streaming video content at the display of the client device, the user interface renders interactive controls that mimic controls of a physical game controller, the interactive controls of the virtual controller used to provide inputs to the online application;
program instructions for receiving inputs provided at the interactive controls of the virtual controller of the client device, the inputs interpreted in accordance to the user specific adjustments and used to control a state of the online application.
12. The computer readable storage medium of claim 11 , further includes program instructions for receiving adjustment to the configuration of the virtual controller provided by a user during a current session, the adjustment to the configuration updated to the configuration of the virtual controller stored in a database at the hosting service for use during subsequent sessions of the online application.
13. The computer readable storage medium of claim 11 , wherein the program instructions for rendering the interactive controls includes program instructions for rendering interactive image of the virtual controller that includes graphics for buttons that are configurable based on content of the online application or on a per user basis.
14. A method, comprising:
communicating, by a client device, with a cloud hosting service a selection of an online application;
receiving, by the client device, streaming video content from execution of the online application on a server of the cloud hosting service, for rendering at a display screen associated with the client device, the streaming video content includes program code for a configuration of a virtual controller defined during a previous configuration session, the virtual controller used for interacting with the streaming video content provided by the cloud hosting service, the streaming video content and the program code being specific for a type of the client device used in interacting with the online application;
executing, by the client device, the program code of the virtual controller, the execution causing a user interface with an interactive image of the virtual controller to be rendered at the display screen of the client device, the interactive image of the virtual controller identifying controls of the virtual controller that are specific for the type of the client device and includes user specific adjustments to the configuration provided by a user selecting the online application;
transmitting, by the client device, user input to the online application executing on the server, the user input provided using the controls of the virtual controller is used to communicate functionality that parallels functionality of buttons of a physical game controller, the user input interpreted in accordance to the user specific adjustments to control a state of the online application executing on the server.
15. The method of claim 14 , wherein the user specific adjustments to the configuration of the virtual controller received during prior interactive sessions of the online application is stored in a database at the cloud hosting service and used during subsequent interactive sessions.
16. The method of claim 14 , wherein the type of client device used in interacting with the online application is determined during initialization of the client device.
17. The method of claim 14 , wherein the streaming video content provided by the server to the client device is selected based on a format that is appropriate for a type of the display screen associated with the client device.
18. The method of claim 14 , wherein the display screen is an external display screen that is communicatively connected to the client device, and the streaming video content provided for rendering at the client device is formatted for the external display screen.
19. The method of claim 14 , wherein the configuration of the virtual controller specifies one or more modes of operation, a location and a mode of operation of the controls in the virtual controller specified based on a type of the online application and on user specific adjustments to the configuration.
20. The method of claim 14 , wherein the configuration of the virtual controller specifies a size of the virtual controller and a region on the user interface of the display screen for rendering the virtual controller, wherein the size and the region being specific for the online application and is configurable using user specific adjustments.