Learning model applying system, a learning model applying method, and a program
A learning model applying system including at least one processor configured to: calculate a plurality of feature amounts based on data for training used for training a learning model; select at least one of the plurality of feature amounts based on an impact of each feature amount on the learning model; apply a trained learning model that is trained based on the selected feature amount, and perform a setting related to calculation of a feature amount to be entered in the trained learning model based an the selected feature amount.
1 . An applying system for determining fraudulent activity in a website comprising:
a preprocessing server configured to calculate a plurality of feature amounts based on data used for training a first learning model and a plurality of preprocessing components;
wherein the first learning model is a model for detecting fraud in the website;
a background server configured to:
determine an amount of impact of each of the plurality of feature amounts on the first learning model's determination of the fraudulent activity;
select at least one of the plurality of feature amounts based on an amount of impact of each of the plurality of feature amounts on the first learning model's determination of the fraudulent activity;
apply a trained second learning model that is trained based on the at least one selected feature amount, the trained second learning model being different from the first learning model;
wherein a final number of the selected feature amount or amounts, are less than a total number of feature amounts;
a database server configured to store a value in a memory, the value being related to a first calculation of a first feature amount to be entered in the trained second learning model based on the at least one selected feature amount;
a real-time server configured to:
estimate, when a user accesses the website, fraud of the user based on the trained second learning model;
restrict the user's access to the website when the fraud of the user is estimated as fraudulent, and
wherein the background server is further configured to, among the plurality of preprocessing components, set a preprocessing component which is used in calculation of the selected feature amount, to be used in the calculation of the first feature amount to be entered in the trained second learning model.
2 . The applying system according to claim 1 , wherein the preprocessing server is configured to input data in a same format in each of the plurality of preprocessing components, and
each of the plurality of preprocessing components performs calculation processing based on data in a same format as data entered in other preprocessing components.
3 . The applying system according to claim 1 , wherein the preprocessing server is configured to:
add a new preprocessing component for calculating a new feature amount,
calculate the new feature amount based on the new preprocessing component, and
the preprocessing server selects the new feature amount, the preprocessing server sets the new preprocessing component to be used in the calculation of the new feature amount to be entered in the trained second learning model.
4 . The applying system according to claim 1 , wherein the background server is configured to:
obtain a calculation time of each feature amount, and
select at least one of the plurality of feature amounts based on an impact of each feature amount on the first learning model and the calculation time of each feature amount.
5 . The applying system according to claim 1 , wherein the background server is configured to:
specify a combination of feature amounts that are related to each other from the plurality of feature amounts, and
when a combination of feature amounts that are related to each other exists, the background server selects a part of the combination.
6 . The applying system according to claim 1 , wherein the background server is configured to:
obtain a calculation time of each feature amount;
determine whether the calculation time of each feature amount is equal to or longer than a predetermined time, and
when a feature amount having its calculation time equal to or longer than the predetermined time, the background server treats the feature amount as a missing value and evaluates an impact of the feature amount on the first learning model.
7 . The applying system according to claim 1 , wherein the real-time server is configured to:
estimate, when a user accesses the website, fraud of the user based on the trained second learning model and a setting for the calculation of the plurality of feature amounts, and
select, the applying, and the setting is performed as background processing that is different from processing of the estimating.
8 . The applying system according to claim 1 , wherein the background server is configured to notify an administrator of the website of information about a selected feature amount.
9 . An applying method for determining fraudulent activity in a website comprising:
calculating, using a preprocessing server, a plurality of feature amounts based on data used for training a first learning model and a plurality of preprocessing components;
wherein the first learning model is a model for detecting fraud in the website;
a background server configured to:
determining, using the preprocessing server, an amount of impact of each of the plurality of feature amounts on the first learning model's determination of the fraudulent activity;
selecting, using the preprocessing server, at least one of the plurality of feature amounts based on an amount of an impact of each of the plurality of feature amounts on the first learning model's determination of the fraudulent activity;
wherein a final number of the selected feature amount or amounts, are less than a total number of feature amounts;
applying, using the preprocessing server, a trained second learning model that is trained based on the at least one selected feature amount, the trained second learning model being different from the first learning model;
storing, using a database server configured to store, a value in a memory, the value being related to a first calculation of a first feature amount to be entered in the trained second learning model based on the at least one selected feature amount;
estimating, using a real-time server
when a user accesses the website, fraud of the user based on the trained second learning model;
restricting, using a real-time server, the user's access to the website when the fraud of the user is estimated as fraudulent, and
wherein the background server is further configured to, among the plurality of preprocessing components, set a preprocessing component which is used in calculation of the selected feature amount, to be used in the calculation of the first feature amount to be entered in the trained second learning model.
10 . The applying system according to claim 1 , wherein the at least one processor sets a first, of the plurality of preprocessing components based on the value, which is not used in calculation of the selected at least one feature amount out of the plurality of feature amounts, not to be used in the calculation of the first feature amount to be entered in the trained second learning model.
11 . The applying system according to claim 1 , wherein the plurality of feature amounts indicate characteristics of behavior data indicating a behavior of a user on a website.
12 . The applying system according to claim 11 , wherein the system generates the behavior data based on the user's behavior on the website.
13 . The applying system according to claim 1 , wherein the at least one selected feature amounts have a relatively large impact, compared to other feature amounts, with respect the first learning model's determination of the fraudulent activity.
14 . The applying system according to claim 1 , wherein the plurality of feature amounts are at least two selected from the group consisting of: an average distance from the user's access location, an elapsed time from the user's previous access, a frequency of the user's access, a usage of a service by the user, an amount of payment by the user, and a frequency of payment by the user.
15 . The applying system according to claim 1 , wherein the amount of impact of each of the plurality of feature amounts is a numerical value.
16 . The applying system according to claim 15 , wherein the background server is configured to determine the amount of impact of each of the plurality of feature amounts on the first learning model's determination of the fraudulent activity using a Permutation Importance method.
17 . The applying system according to claim 1 , wherein the preprocessing server calculates the plurality of feature amounts.
18 . The applying system according to claim 1 , wherein the preprocessing server is configured to calculate the plurality of feature amounts using a plurality of calculation formulas for calculating a feature amount from a value output from the plurality of preprocessing components.