Timed input/action release
A machine learning-based model or other system is configured to release user commands to a computer game at a time when the command correctly aligns with what is going on in the computer game. As such, the command might be predicted and released before the user input action is completed, or might be delayed after completion and processing until a later time when the command is most appropriate for release.
1 . An apparatus, comprising: at least one processor assembly programmed with instructions to: execute a computer game; receive input from a sensor;
generate, based on the input from the sensor, a predicted user command to the computer game, the generating comprising processing the input from the sensor using a first machine learning model;
determine, based on the predicted user command, a time for controlling the computer game according to the predicted user command, the determining comprising processing the predicted user command generated by the first machine learning model using a second machine learning model that is separate and distinct from the first machine learning model, wherein the second machine learning model receives as input both the predicted user command and game state data of the computer game; and
control the computer game according to the predicted user command at the determined time, the computer game controlled according to the predicted user command by one or more of: controlling the computer game according to the predicted user command in advance of a user completing the predicted user command, delaying control of the computer game according to the predicted user command subsequent to the user completing the predicted user command.
2 . The apparatus of claim 1 , wherein the sensor comprises a camera, and wherein the at least one processor assembly is programmed with instructions to:
receive input from the camera;
generate, based on the input from the camera, the predicted user command; and
control the computer game according to the predicted user command in advance of the user completing the predicted user command.
3 . The apparatus of claim 2 , wherein the at least one processor assembly is programmed with instructions to:
identify a game action as occurring for which the predicted user command is to be executed; and
based on the identification, control the computer game according to the predicted user command in advance of the user completing the predicted user command.
4 . The apparatus of claim 2 , wherein the predicted user command comprises a gesture command.
5 . The apparatus of claim 2 , wherein the predicted user command comprises a computer game controller command.
6 . The apparatus of claim 2 , comprising the camera.
7 . The apparatus of claim 1 , wherein the sensor comprises a controller button, and wherein the at least one processor assembly is programmed with instructions to:
receive input generated based on actuation of the controller button;
generate, based on the input generated based on actuation of the controller button, the predicted user command; and
control the computer game according to the predicted user command at the determined time by delaying control of the computer game according to the predicted user command until a time subsequent to the user completing the predicted user command, wherein the time subsequent to the user completing the predicted user command matches the determined time.
8 . The apparatus of claim 7 , wherein the at least one processor assembly is programmed with instructions to:
delay control of the computer game according to the predicted user command until a game action occurs at the determined time for which the predicted user command is to be executed; and
based on the game action occurring at the determined time, control the computer game according to the predicted user command.
9 . The apparatus of claim 1 , wherein the at least one processor assembly is programmed with instructions to:
use the first machine learning model to infer the predicted user command in advance of the user completing the predicted user command.
10 . The apparatus of claim 1 , comprising the at least one processor assembly.
11 . The apparatus of claim 1 , wherein the first machine learning model and the second machine learning model are elements of a multi-function machine learning model.
12 . A method, comprising: receiving input from a sensor;
generating, based on the input from the sensor, a predicted user command to a first computer game, the generating comprising processing the input from the sensor using a first machine learning model;
determining, based on the predicted user command, a time for controlling the first computer game according to the predicted user command, the determining comprising processing the predicted user command generated by the first machine learning model using a second machine learning model that is separate and distinct from the first machine learning model, wherein the second machine learning model receives as input both the predicted user command and game state data of the computer game; and
controlling the first computer game according to the predicted user command at the determined time, the first computer game controlled according to the predicted user command by one or more of: controlling the first computer game according to the predicted user command in advance of a user completing the predicted user command, delaying control of the first computer game according to the predicted user command subsequent to the user completing the predicted user command.
13 . The method of claim 12 , comprising:
providing, as an input to the first machine learning model, camera input, the camera input indicating the predicted user command;
receiving, as an output from an activation layer of the first machine learning model, an inference of the predicted user command; and
using the inference for generating the predicted user command.
14 . The method of claim 13 , wherein the first machine learning model is trained on at least one dataset of camera inputs and respective ground truth user commands.
15 . The method of claim 14 , wherein at least some of the camera inputs of the at least one dataset indicate user gestures to actuate computer game controllers.
16 . The method of claim 12 , comprising: receiving, as an output from an activation layer of the second machine learning model, an inference based on the game state data, the inference indicating the determined time for controlling the first computer game according to the predicted user command; and controlling, at the determined time indicated by the inference, the first computer game according to the predicted user command.
17 . The method of claim 16 , wherein the second machine learning model is trained on at least one dataset of game state data, user commands, and respective ground truth times for controlling one or more computer games.
18 . The method of claim 12 , wherein determining the time for controlling the first computer game according to the predicted user command comprises:
using a rules-based software algorithm to determine, in part and based on game state data and the predicted user command, the time for controlling the first computer game according to the predicted user command.
19 . A system comprising: at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly to:
generate a predicted user command to a computer game, the generating comprising processing input from a sensor using a first machine learning model;
determine, based on the predicted user command, a time for controlling the computer game according to the predicted user command, the determining comprising processing the predicted user command generated by the first machine learning model using a second machine learning model that is separate and distinct from the first machine learning model, wherein the second machine learning model receives as input both the predicted user command and game state data of the computer game; and
release the predicted user command to the computer game at the determined time, the determined time being a time during which a game action occurs for which the predicted user command is to be executed, the determined time being different from a second time at which the predicted user command is completed.
20 . The system of claim 19 , wherein the instructions are executable to, one or more of:
release the predicted user command in advance of receiving input generated based on actuation of a computer game controller button that is predicted to be used to provide the predicted user command; or
delay release of the predicted user command subsequent to a user completing the predicted user command based on a determination that the game action has not yet occurred for which to apply the predicted user command.