IP Library Granted Patent US 12,064,687
Granted Patent B2
US 12,064,687 · App. 18/298,754 · Granted Aug 20, 2024

System, method, and graphical user interface for controlling an application executing on a server

Inventors: Joshuah Vincent (San Francisco, CA); Nico Benitez (Los Angeles, CA); Emily Adams (Fremont, CA); Jan Chong (San Francisco, CA); Ryan Cook (Mountain View, CA); Paul Marshall (Cupertino, CA); Pierpaolo Baccichet (San Francisco, CA); Matt Piotrowski (San Francisco, CA); Collin Carey (Menlo Park, CA); Kevin Koleckar (Mountain View, CA)
Assignee: Sony Interactive Entertainment LLC
A63F13/42A63F13/2145A63F13/426G06F3/0338A63F13/355A63F2300/1068A63F2300/308A63F2300/6045
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,064,687
App. No.
18/298,754
Granted
Aug 20, 2024
Kind
B2
Abstract

A system, method, and graphical user interface for playing games and/or executing applications on a tablet-based client. One embodiment of a graphical user interface (GUI) for playing a video game on a tablet-based client device comprises: a virtual controller rendered on a display of the tablet computer, the virtual controller substantially mimicking the control provided by a thumb stick of a physical game controller and providing omnidirectional, free-form movement in a synchronous direction in which a user moves a finger on the display of the tablet-based client.

Claims (40)

1. A method, comprising:

receiving selection of an online application at a hosting service, the selection received from a remote client device;

executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating frames of streaming video content for the online application for onward transmission to the remote client device;

identifying a set of widgets corresponding to graphical user interface elements used for interacting with the streaming video content of the online application, each widget in the set is associated with configuration parameters used for rendering; and

encoding at least one component of a widget in the set of widgets for onward transmission to the remote client device for rendering,

wherein the encoded at least one component of the widget in the set of widgets are decoded at the remote client device and rendered on a display of the remote client device in accordance to the configuration parameters associated with each widget in the set, the graphical user interface element of each widget of the set of widgets map to at least some controls of a physical game controller, and

wherein inputs provided at each of the graphical user interface elements rendered at the remote client device are interpreted by the server of the hosting service to control a state of the online application.

2. The method of claim 1 , wherein the at least one component is identified based on a type of the remote client device used for rendering,

wherein the encoding of the at least one component of the widget includes,

identifying the at least one component of the widget defined for the type of remote client device and encoding the identified at least one component.

3. The method of claim 2 , wherein the type of the remote client device is based on an operating system used to run the remote client device, and

wherein the at least one component is identified for the operating system of the remote client device.

4. The method of claim 1 , wherein the configuration parameters are defined for each online application, and the widgets in the set are identified to include configuration parameters that is specific for the online application.

5. The method of claim 4 , wherein the at least one component of the widget in the set of widgets is identified based on a type of online application, and

wherein the encoding of the at least one component of the widget includes,

identifying the at least one component of the widget defined for the type of online application and encoding the identified at least one component.

6. The method of claim 1 , wherein the configuration parameters of each widget in the set include interactive attributes that are applicable for each online application, and

wherein identifying the set of widgets includes activating specific interactive attributes defined for the online application.

7. The method of claim 1 , wherein the set of widgets is encoded with content of the online application forwarded to the remote client device.

8. The method of claim 1 , wherein at least a part of each widget is executing on the remote client device.

9. The method of claim 1 , wherein at least a part of each widget is executing remotely on the server of the hosting service.

10. The method of claim 1 , wherein a part of each widget executes locally at the remote client device and a remaining part executes remotely on the server of the hosting service.

11. The method of claim 1 , wherein the widgets in the set are identified in part by retrieving virtual user interface configuration defined for the online application, the virtual user interface configuration identifying the widgets that correspond with the graphical user interface elements used for interacting with the online application.

12. The method of claim 11 , wherein the virtual user interface configuration is configured to include select ones of the graphical user interface elements based on a state of the online application.

13. The method of claim 1 , wherein the set of widgets associated with the graphical user interface elements is decoded at the remote client device and rendered as an overlay over the frames of the streaming video content.

14. The method of claim 13 , wherein the set of widgets associated with the graphical user interface elements are configured to be transparent or semi-transparent so as to allow viewing of the streaming video content rendering beneath the overlay.

15. The method of claim 1 , wherein the configuration parameters associated with each widget in the set include one or more distinct settings, the one or more distinct settings being specified by a user viewing the streaming video content, or a developer of the online application, or the hosting service.

16. The method of claim 15 , wherein the one or more distinct settings specified for each widget in the set includes a first setting for an inactive state and a second setting for an active state, and

wherein the one or more distinct settings include activation setting, or transparency setting, or brightness setting, or sensitivity setting, or color setting, or scale setting, or size setting, or location setting, or operational mode setting, or operational direction setting, or graphical user interface element type setting, or any two or more combinations thereof.

17. The method of claim 15 , wherein the one or more distinct settings are configured for the display of the remote client device on which the streaming video content and the graphical user interface elements are rendered.

18. The method of claim 1 , wherein each widget in the set of widgets associated with a particular graphical user interface element is configured to provide particular control action of the physical game controller.

19. The method of claim 1 , further includes,

receiving user specific adjustment to a graphical user interface element associated with the widget in the set from a user during a current session, the adjustment to the one or more of the graphical user interface elements updated to corresponding widget stored in a database of the hosting service and used during subsequent sessions of the online application.

20. A non-transitory computer readable storage medium having program instructions, which when executed by a processor of a computing device of a hosting service performs a method, the program instructions comprising:

program instructions for receiving selection of an online application at the hosting service, the selection received from a remote client device;

program instructions for executing an instance of the online application on a server of the hosting service, in response to the selection, the execution generating frames of streaming video content for the online application for onward transmission to the remote client device;

program instructions for identifying a set of widgets corresponding to graphical user interface elements used for interacting with the streaming video content of the online application, each widget in the set is associated with configuration parameters used for rendering; and

program instructions for encoding at least one component of a widget in the set of widgets for onward transmission to the remote client device for rendering,

wherein the encoded at least one component of the widget in the set of widgets are decoded at the remote client device and rendered on a display of the remote client device in accordance to the configuration parameters associated with each widget in the set, the graphical user interface elements of each widget of the set of widgets map to at least some controls of a physical game controller, and

wherein inputs provided at each of the graphical user interface elements rendered at the remote client device are interpreted by the processor of the hosting service to control a state of the online application.

Continuity (7)
Continuation 17405754 · Aug 18, 2021
Continuation 16660762 · Oct 22, 2019
Continuation 16236230 · Dec 28, 2018
Continuation 15797589 · Oct 30, 2017
Continuation 13797039 · Mar 12, 2013
Provisional Application 61610278 · Mar 13, 2012
Related Publication 20230249067A1 · Aug 10, 2023