IP Library Granted Patent US 12,691,373
Granted Patent B2
US 12,691,373 · App. 18/281,278 · Granted Jul 28, 2026

State based action buttons

Inventor: Gregory Peng (Bellevue, WA)
Assignee: SONY INTERACTIVE ENTERTAINMENT LLC
A63F13/2145G06F3/04886
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,691,373
App. No.
18/281,278
Filed
Sep 8, 2023
Granted
Jul 28, 2026
Kind
B2
Art Unit
3715
USPC
463/37
Abstract

Described herein are techniques for implementing a virtualized physical controller. The techniques may comprise receiving. from a user via a touchscreen display. a first touch input corresponding to at least one input mechanism of a set of input mechanisms. determining a first state that is currently active. generating. based on the first state. first activation data that includes an indication of a first action. The techniques may further comprise receiving. from the user via the touchscreen display. a second touch input corresponding to the at least one input mechanism. determining a second state that is currently active, and generating. based on the second state, second activation data that includes an indication of a second action different from the first action.

Claims (61)

1 . A method of state-based activation, the method comprising:

determining a first state within an interactive video game, wherein the first state corresponds to an active state of an object presented within the interactive video game and being controlled by a user using a virtual controller;

rendering at least two input mechanisms of a set of input mechanisms on a graphical user interface on a touchscreen display of a touch-enabled device based on the first state, wherein the at least two input mechanisms comprise a first input mechanism and a second input mechanism;

detecting a first touch input from the user on the touchscreen display of the touch-enabled device during the first state, wherein a location of the first touch input on the touchscreen display corresponds to a location of the first input mechanism of the at least two input mechanisms for the virtual controller presented as the graphical user interface on the touchscreen display and the first touch input from the user is determined to be activation of the first input mechanism based on the location of the first touch input corresponding to the location of the first input mechanism;

generating first activation data that includes an indication of a first action to be executed for the object within the interactive video game in response to the activation of the first input mechanism during the first state, wherein the first activation data is generated based on configuration data for the touch-enabled device that maintains information comprising actions to be initiated upon activation of each input mechanism of the set of input mechanisms in accordance with states within the interactive video game;

executing the first action for the object within the interactive video game based on the first activation data;

determining a second state within the interactive video game, wherein the second state corresponds to a different active state of the object presented within the interactive video game, and wherein the first state changed to the second state in response to executing the first action;

updating the rendering of the second input mechanism of the at least two input mechanisms on the graphical user interface based on the second state;

detecting a second touch input from the user on the touchscreen display during the second state, wherein a location of the second touch input on the touchscreen display corresponds to a location of the second input mechanism for the virtual controller presented as the graphical user interface on the touchscreen display and the second touch input from the user is determined to be activation of the second input mechanism based on the location of the second touch input corresponding to the location of the second input mechanism;

generating second activation data that includes an indication of a second action to be executed for the object within the interactive video game in response to the activation of the second input mechanism during the second state; and

executing the second action for the object within the interactive video game based on the second activation data.

2 . The method of claim 1 , further comprising updating at least one of a size, location, shape, or appearance of the second input mechanism based on the second state.

3 . The method of claim 1 , further comprising updating the set of input mechanisms by adding an input mechanism to the set of input mechanisms presented on the touchscreen display while the second state is active.

4 . The method of claim 1 , further comprising updating the set of input mechanisms by removing an input mechanism from the set of input mechanisms presented on the touchscreen display while the second state is active.

5 . The method of claim 1 , wherein the first state and the second state are both determined to be active at a same time.

6 . The method of claim 1 , wherein the second state replaces the first state, such that no more than one of the first state or the second state are active at a same time.

7 . The method of claim 1 , further comprising initiating the second state upon selection of a different input mechanism of the set of input mechanisms presented on the touchscreen display.

8 . The method of claim 1 , wherein the second state persists for a predetermined amount of time.

9 . The method of claim 1 , further comprising:

detecting a third touch input from the user on the touchscreen display, wherein a location of the third touch input alse corresponds to the location of the first input mechanism;

determining that the second state is no longer active; and

generating a third activation data that includes an indication of the first action based on the determination that the second state is no longer active.

10 . A user device comprising:

a processor; and

a memory including instructions executable by the processor to, at least:

determine a first state within an interactive video game, wherein the first state corresponds to an active state of an object presented within the interactive video game and being controlled by a user using a virtual controller;

render at least two input mechanisms of a set of input mechanisms on a graphical user interface on a touchscreen display of a touch-enabled device based on the first state, wherein the at least two input mechanisms comprise a first input mechanism and a second input mechanism;

detect a first touch input from the user on the touchscreen display of the touch-enabled device during the first state, wherein a location of the first touch input on the touchscreen display corresponds to a location of the first input mechanism of the at least two input mechanisms for the virtual controller presented as the graphical user interface on the touchscreen display and the first touch input from the user is determined to be activation of the first input mechanism based on the location of the first touch input corresponding to the location of the first input mechanism;

generate first activation data that includes an indication of a first action to be executed for the object within the interactive video game in response to the activation of the first input mechanism during the first state, wherein the first activation data is generated based on configuration data for the touch-enabled device that maintains information comprising actions to be initiated upon activation of each input mechanism of the set of input mechanisms in accordance with states within the interactive video game;

execute the first action for the object within the interactive video game based on the first activation data;

determine a second state within the interactive video game, wherein the second state corresponds to a different active state of the object presented within the interactive video game, and wherein the first state changed to the second state in response to executing the first action;

update the rendering of the second input mechanism of the at least two input mechanisms on the graphical user interface based on the second state;

detect a second touch input from the user on the touchscreen display during the second state, wherein a location of the second touch input on the touchscreen display corresponds to a location of the second input mechanism for the virtual controller presented as the graphical user interface on the touchscreen display and the second touch input from the user is determined to be activation of the second input mechanism based on the location of the second touch input corresponding to the location of the second input mechanism;

generate second activation data that includes an indication of a second action to be executed for the object within the interactive video game in response to the activation of the second input mechanism during the second state; and

execute the second action for the object within the interactive video game based on the second activation data.

11 . The user device of claim 10 , further comprising a touchscreen that generates the touchscreen display.

12 . The user device of claim 10 , wherein the user device is one of a smartphone or tablet device.

13 . The user device of claim 10 , wherein the instructions are further executable to:

detect a third touch input from the user at a location corresponding to the location of the first input mechanism;

determine that the second state is no longer active; and

generate a third activation data that includes an indication of the first action based on the determination that the second state is no longer active.

14 . The user device of claim 10 , wherein the instructions are associated with the virtual controller specific to a software application.

15 . The user device of claim 14 , wherein the software application comprises the interactive video game.

16 . The user device of claim 15 , wherein each of the first state and the second state pertain to a status of an avatar in the interactive video game.

17 . The user device of claim 10 , wherein the instructions are further executable to update at least one of a size, location, shape, or appearance of the second input mechanism based on the second state.

18 . A non-transitory computer-readable storage medium, having embodied thereon instructions executable by one or more computing devices to collectively perform acts comprising:

determining a first state within an interactive video game, wherein the first state corresponds to an active state of an object presented within the interactive video game and being controlled by a user using a virtual controller;

rendering at least two input mechanisms of a set of input mechanisms on a graphical user interface on a touchscreen display of a touch-enabled device based on the first state, wherein the at least two input mechanisms comprise a first input mechanism and a second input mechanism;

detecting a first touch input from the user on the touchscreen display of the touch-enabled device during the first state, wherein a location of the first touch input on the touchscreen display corresponds to a location of the first input mechanism of the at least two input mechanisms for the virtual controller presented as the graphical user interface on the touchscreen display and the first touch input from the user is determined to be activation of the first input mechanism based on the location of the first touch input corresponding to the location of the first input mechanism;

generating first activation data that includes an indication of a first action to be executed for the object within the interactive video game in response to the activation of the first input mechanism during the first state, wherein the first activation data is generated based on configuration data for the touch-enabled device that maintains information comprising actions to be initiated upon activation of each input mechanism of the set of input mechanisms in accordance with states within the interactive video game;

executing the first action for the object within the interactive video game based on the first activation data;

determining a second state within the interactive video game, wherein the second state corresponds to a different active state of the object presented within the interactive video game, and wherein the first state changed to the second state in response to executing the first action;

updating the rendering of the second input mechanism of the at least two input mechanisms on the graphical user interface based on the second state;

detecting a second touch input from the user on the touchscreen display during the second state, wherein a location of the second touch input on the touchscreen display corresponds to a location of the second input mechanism for the virtual controller presented as the graphical user interface on the touchscreen display and the second touch input from the user is determined to be activation of the second input mechanism based on the location of the second touch input corresponding to the location of the second input mechanism;

generating second activation data that includes an indication of a second action to be executed for the object within the interactive video game in response to the activation of the second input mechanism during the second state; and

executing the second action for the object within the interactive video game based on the second activation data.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein the acts further comprise updating at least one of a size, location, shape, or appearance of the second input mechanism based on the second state.

20 . The non-transitory computer-readable storage medium of claim 18 , wherein the acts further comprise:

detecting a third touch input from the user on the touchscreen display, wherein a location of the third touch input corresponds to the location of the first input mechanism;

determining that the second state is no longer active; and

generating a third activation data that includes an indication of the first action based on the determination that the second state is no longer active.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2026
From: BUNGIE, INC.
To: SONY INTERACTIVE ENTERTAINMENT LLC
Reel/Frame 074404/0767 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2023
From: PENG, GREGORY
To: BUNGIE, INC.
Reel/Frame 064911/0515 →
Continuity (2)
Provisional Application 63158952 · Mar 10, 2021
Related Publication 20240207720A1 · Jun 27, 2024
References Cited (141)
US 20050176502A1 · Nishimura et al. · 2005 [cited by applicant]
US 20060252542A1 · Nicely · 2006 [cited by examiner]
US 20070060335A1 · Sakaguchi et al. · 2007 [cited by applicant]
US 20090181770A1 · Viner et al. · 2009 [cited by applicant]
US 20090191968A1 · Johnson et al. · 2009 [cited by applicant]
US 20100248843A1 · Karsten · 2010 [cited by applicant]
US 20140187328A1 · Reynolds et al. · 2014 [cited by applicant]
US 20150072784A1 · Lee · 2015 [cited by applicant]
US 20160287990A1 · Garvin et al. · 2016 [cited by applicant]
US 20170340959A1 · Tang et al. · 2017 [cited by applicant]
US 20180043254A1 · Soejima et al. · 2018 [cited by applicant]
US 20200174618A1 · Wang et al. · 2020 [cited by applicant]
US 20200179805A1 · Miao · 2020 [cited by applicant]
US 20200282308A1 · Guo et al. · 2020 [cited by applicant]
US 20200289933A1 · Seropian et al. · 2020 [cited by applicant]
US 20200298104A1 · Wang · 2020 [cited by applicant]
US 20240149146A1 · Timmins et al. · 2024 [cited by applicant]
US 20240149147A1 · Peng · 2024 [cited by applicant]
AU 2022232383 · 2023 [cited by applicant]
AU 2022234308 · 2023 [cited by applicant]
AU 2022234961 · 2023 [cited by applicant]
CA 3212868 · 2022 [cited by applicant]
CA 3212970 · 2022 [cited by applicant]
CA 3212972 · 2022 [cited by applicant]
CN 108635850 · 2018 [cited by applicant]
CN 108635850A · 2018 [cited by applicant]
CN 112169315A · 2021 [cited by applicant]
CN 112261973 · 2021 [cited by applicant]
CN 116964550 · 2023 [cited by applicant]
CN 116964552 · 2023 [cited by applicant]
CN 117043736 · 2023 [cited by applicant]
EP 4288860 · 2023 [cited by applicant]
EP 4291975 · 2023 [cited by applicant]
EP 4291976 · 2023 [cited by applicant]
IL 305742 · 2023 [cited by applicant]
IL 305743 · 2023 [cited by applicant]
IL 305749 · 2023 [cited by applicant]
JP 2005218778 · 2002 [cited by applicant]
JP 4243553B2 · 2009 [cited by applicant]
JP 4317774B2 · 2009 [cited by applicant]
JP 4937421B2 · 2012 [cited by applicant]
JP 2015123244 · 2015 [cited by applicant]
JP 2015231545 · 2015 [cited by applicant]
JP 5879007B1 · 2016 [cited by applicant]
JP 6045837B2 · 2016 [cited by applicant]
JP 2016201028 · 2016 [cited by applicant]
JP 2017012423 · 2017 [cited by applicant]
JP 6143934B1 · 2017 [cited by applicant]
JP 2018086275 · 2018 [cited by applicant]
JP 2018097649 · 2018 [cited by applicant]
JP 2018161446 · 2018 [cited by applicant]
JP 6447853B1 · 2018 [cited by applicant]
JP 6480531B1 · 2019 [cited by applicant]
JP 2019150269 · 2019 [cited by applicant]
JP 2019212119 · 2019 [cited by applicant]
JP 2020028652 · 2020 [cited by applicant]
JP 2020192239 · 2020 [cited by applicant]
JP 2021000192A · 2021 [cited by applicant]
JP 2021029291A · 2021 [cited by applicant]
JP 2024511304 · 2024 [cited by applicant]
JP 2024513669 · 2024 [cited by applicant]
JP 2024513672 · 2024 [cited by applicant]
KR 1020130027621 · 2013 [cited by applicant]
KR 101411684 · 2014 [cited by applicant]
KR 20140077991A · 2014 [cited by applicant]
KR 1020140127931 · 2014 [cited by applicant]
KR 1020200091897 · 2020 [cited by applicant]
MX a2023010550 · 2023 [cited by applicant]
MX a2023010552 · 2023 [cited by applicant]
MX a2023010553 · 2023 [cited by applicant]
WO WO2013173838 · 2013 [cited by applicant]
WO WO2022192468 · 2022 [cited by applicant]
WO WO2022192471 · 2022 [cited by applicant]
WO WO2022192492 · 2022 [cited by applicant]
International Patent Application No. PCT/US2022/019645, International Search Report and Written Opinion mailed Jun. 16, 2022, 9 pages. [cited by applicant]
Canadian Application No. 3,212,972, Examiner's Report dated Dec. 5, 2024. [cited by applicant]
Japanese Application No. 2023-554299, Decision of Refusal dated Dec. 12, 2024. [cited by applicant]
UAE Application No. P6002241/2023, Examination Report and Search Report dated Dec. 13, 2024. [cited by applicant]
UAE Application No. P6002244/2023, Examination Report and Search Report dated Dec. 13, 2024. [cited by applicant]
UAE Application No. P6002248/2023, Examination Report and Search Report dated Dec. 13, 2024. [cited by applicant]
European Application No. 22767958.6, Extended European Search Report dated Jul. 9, 2024. [cited by applicant]
European Application No. 22767945.3, Extended European Search Report dated Jul. 25, 2024. [cited by applicant]
Australian Application No. 2022234961, Second Examination Report dated Aug. 16, 2024. [cited by applicant]
Australian Application No. 2022232383, Second Examination Report dated Sep. 24, 2024. [cited by applicant]
European Application No. 22767943.8, Extended European Search Report dated Aug. 16, 2024. [cited by applicant]
Japanese Application No. 2023-554299, Non-Final Notification of Reasons for Refusal dated Sep. 10, 2024. [cited by applicant]
Japanese Application No. 2023-553638, Non-Final Notification of Reasons for Refusal dated Oct. 1, 2024. [cited by applicant]
PCT Application No. PCT/US2022/019681, International Search Report dated Jun. 21, 2022. [cited by applicant]
PCT Application No. PCT/US2022/019681, International Preliminary Report on Patentability dated Sep. 12, 2023. [cited by applicant]
PCT Application No. PCT/US2022/019645, International Preliminary Report on Patentability dated Sep. 12, 2023. [cited by applicant]
PCT Application No. PCT/US2022/019648, International Search Report and Written Opinion dated Jun. 16, 2022. [cited by applicant]
PCT Application No. PCT/US2022/019648, International Preliminary Report on Patentability dated Sep. 12, 2023. [cited by applicant]
Australian Application No. 2022234308, First Examination Report dated Jan. 11, 2024. [cited by applicant]
Australian Application No. 2022234961, First Examination Report dated Jan. 12, 2024. [cited by applicant]
Australian Application No. 2022232383, First Examination Report dated Jan. 13, 2024. [cited by applicant]
Australian Application No. AU2022234961, “Notice of Acceptance”, mailed Jan. 9, 2025, 3 pages. [cited by applicant]
European Application No. EP22767943.8, Extended European Search Report, mailed Feb. 11, 2025, 11 pages. [cited by applicant]
European Application No. EP22767943.8 , “Supplementary European Search Report”, mailed Feb. 14, 2025, 1 page. [cited by applicant]
U.S. Appl. No. 18/281,274, Luke Timmins, Infinite Drage and Swipe U Turn, filed Sep. 8, 2023. [cited by applicant]
P6002241/2023, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 7, 2023. [cited by applicant]
2022234308, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 18, 2023. [cited by applicant]
U.S. Pat. No. 3,212,868, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 7, 2023. [cited by applicant]
202280020263.1, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 8, 2023. [cited by applicant]
22767958.6, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 6, 2023. [cited by applicant]
305742, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 7, 2023. [cited by applicant]
2023-554299, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 6, 2023. [cited by applicant]
MX/a/2023/010550, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 7, 2023. [cited by applicant]
803788, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 18, 2023. [cited by applicant]
PCT/US2022/019681, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Oct. 10, 2022. [cited by applicant]
2023124994, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 29, 2023. [cited by applicant]
523450614, Luke Timmins, Infinite Drag and Swipe for Virtual Controller, Sep. 10, 2023. [cited by applicant]
P6002244/2023, Gregory Peng, State Based Action Buttons, Sep. 7, 2023. [cited by applicant]
2022234961, Gregory Peng, State Based Action Buttons, Sep. 19, 2023. [cited by applicant]
U.S. Pat. No. 3,212,970, Gregory Peng, State Based Action Buttons, Sep. 7, 2023. [cited by applicant]
202280019768.6, Gregory Peng, State Based Action Buttons, Sep. 8, 2023. [cited by applicant]
22767943.8, Gregory Peng, State Based Action Buttons, Sep. 10, 2023. [cited by applicant]
305743, Gregory Peng, State Based Action Buttons, Sep. 7, 2023. [cited by applicant]
2023-553638, Gregory Peng, State Based Action Buttons, Sep. 4, 2023. [cited by applicant]
MX/a/2023/010552, Gregory Peng, State Based Action Buttons, Sep. 7, 2023. [cited by applicant]
803844, Gregory Peng, State Based Action Buttons, Sep. 19, 2023. [cited by applicant]
PCT/US2022/019645, Gregory Peng, State Based Action Buttons, Mar. 9, 2022. [cited by applicant]
2023124995, Gregory Peng, State Based Action Buttons, Sep. 29, 2023. [cited by applicant]
523450612, Gregory Peng, State Based Action Buttons, Sep. 10, 2023. [cited by applicant]
U.S. Appl. No. 18/281,279, Gregory Peng, Virtual Button Charging, filed Sep. 8, 2023. [cited by applicant]
P6002248/2023, Gregory Peng, Virtual Button Charging, Sep. 7, 2023. [cited by applicant]
2022232383, Gregory Peng, Virtual Button Charging, Sep. 19, 2023. [cited by applicant]
U.S. Pat. No. 3,212,972, Gregory Peng, Virtual Button Charging, Sep. 8, 2023. [cited by applicant]
202280020310.2, Gregory Peng, Virtual Button Charging, Sep. 8, 2023. [cited by applicant]
22767945.3, Gregory Peng, Virtual Button Charging, Sep. 15, 2023. [cited by applicant]
305749, Gregory Peng, Virtual Button Charging, Sep. 7, 2023. [cited by applicant]
2023-553639, Gregory Peng, Virtual Button Charging, Sep. 4, 2023. [cited by applicant]
MX/a/2023/010553, Gregory Peng, Virtual Button Charging, Sep. 7, 2023. [cited by applicant]
803836, Gregory Peng, Virtual Button Charging, Sep. 19, 2023. [cited by applicant]
PCT/US2022/019648, Gregory Peng, Virtual Button Charging, Mar. 9, 2022. [cited by applicant]
2023125045, Gregory Peng, Virtual Button Charging, Sep. 29, 2023. [cited by applicant]
523450616, Gregory Peng, Virtual Button Charging, Sep. 10, 2023. [cited by applicant]
Canadian Application No. 3,212,868, Examiner's Report dated Nov. 12, 2024. [cited by applicant]
Canadian Application No. 3,212,970, Examiner's Report dated Nov. 14, 2024. [cited by applicant]
Japanese Application No. 2023-553639, Non-Final Notification of Reasons for Refusal dated Oct. 8, 2024. [cited by applicant]
Japanese Application No. JP2023-553638 , Notice of Decision to Grant, Mailed On May 7, 2025, 3 pages. [cited by applicant]
New Zealand Application No. NZ803844, “First Examination Report”, mailed Apr. 7, 2025, 4 pages. [cited by applicant]