Method, etc. for generating trained model for predicting action to be selected by user
One or more embodiments of the invention is a method for generating a trained model for predicting an action to be selected by a user in a game that proceeds in accordance with actions selected by the user, while updating game states, the method including: determining weights for individual history-data element groups; generating training data from data of game states and actions included in the history-data element groups; and generating a trained model on the basis of the generated training data, wherein the generation of training data includes generating a number of items of game state text as game state text corresponding to one game state, having different orders of a plurality of text elements, the number being based on the determined weight, and generating training data including pairs of the individual generated items of game state text and corresponding action text.
1 . A method comprising:
determining a plurality of weights for a plurality of individual history-data element groups included in history data concerning a computer game based on user information associated with the plurality of individual history-data element groups;
generating game state text data and action text data based on the computer game, wherein the game state text data and the action text data are expressed in a prescribed format corresponding to a plurality of game states and a plurality of actions included in the plurality of individual history-data element groups included in the history data;
generating training data using the game state text data and the action text data,
wherein the training data comprises a plurality of pairs of game state text and action text, and
wherein a respective pair among the plurality of pairs corresponds to a single game state and an action selected in the single game state;
generating a trained transformer neural network using the training data and a pretrained natural language model,
wherein the pretrained natural language model is configured in advance to understand a plurality of grammatical structures and a plurality of text-to-text relationships concerning a natural language,
wherein the training data comprises a number of items of game state text,
wherein the items of game state text have different orders of a plurality of text elements, and
wherein the number of items of game state text is based on a weight among the plurality of weights that is determined for a history-data element group among the plurality of individual history-data element groups;
displaying, on a display device and by a user terminal device comprising an input device and a computer processor, a plurality of game screens for the computer game;
receiving, during the computer game and by the input device, a player input from a player that is operating the computer game in response to displaying at least one game screen among a plurality of game screens;
determining, by the user terminal device, a first game state within the computer game based on the player input and in response to receiving the player input from the input device;
determining, using the trained transformer neural network, a predicted user action in the computer game using the first game state as an input to the trained transformer neural network;
determining, by the user terminal device, a second game state within the computer game using the predicted user action; and
executing, by the user terminal device, the computer game based on the second game state that is determined using the predicted user action.
2 . The method according to claim 1 , wherein the trained transformer neural network is generated by training a deep learning model with the training data, the deep learning model being directed to learning sequential data.
3 . The method according to claim 1 , wherein the plurality of weights are determined so as to have a plurality of magnitudes corresponding to a plurality of levels of user ranks included in the user information.
4 . The method according to claim 1 , wherein:
generating the training data includes generating first pairs of training data and second pairs of training data, the first pairs of training data being pairs of game state text and action text corresponding to pairs of one game state and an action selected in the one game state, generated based on data of game states and actions included in the plurality of individual history-data element groups included in the history data, and the second pairs of training data being pairs of game state text and action text corresponding to actions that are randomly selected from actions selectable by a user and that are not included in the first pairs; and
generating the trained transformer neural network comprises performing training with the first pairs of training data as correct data and performing training with the second pairs of training data as incorrect data.
5 . A non-transitory computer readable medium storing a program that causes a computer to execute the method according to claim 1 .
6 . The method according to claim 1 , wherein the predicted user action is used by a game artificial intelligence (AI) within the computer game.
7 . A system comprising:
a processor; and
a memory connected to the processor, wherein the memory comprises a program configured to perform a method comprising:
determining a plurality of weights for a plurality of individual history-data element groups included in history data concerning a computer game based on user information associated with the plurality of individual history-data element groups;
generating game state text data and action text data based on the computer game,
wherein the game state text data and the action text data are expressed in a prescribed format corresponding to a plurality of game states and a plurality of actions included in the plurality of individual history-data element groups included in the history data;
generating training data using the game state text data and the action text data,
wherein the training data comprises a plurality of pairs of game state text and action text, and
wherein a respective pair among the plurality of pairs corresponds to a single game state and an action selected in the single game state;
generating a trained transformer neural network using the training data and a pretrained natural language model,
wherein the pretrained natural language model is configured in advance to understand a plurality of grammatical structures and a plurality of text-to-text relationships concerning a natural language,
wherein the training data comprises a number of items of game state text,
wherein the items of game state text have different orders of a plurality of text elements, and
wherein the number of items of game state text is based on a weight among the plurality of weights that is determined for a history-data element group among the plurality of individual history-data element groups;
displaying, on a display device, a plurality of game screens for the computer game;
receiving, during the computer game and by an input device, a player input from a player that is operating the computer game in response to displaying at least one game screen among a plurality of game screens;
determining a first game state within the computer game based on the player input and in response to receiving the player input from the input device;
determining, using the trained transformer neural network, a predicted user action in the computer game using the first game state as an input to the trained transformer neural network;
determining a second game state within the computer game using the predicted user action; and
executing the computer game based on the second game state that is determined using the predicted user action.