Classification system and method based on generative adversarial network
A generative adversarial network-based classification system and method that can generate missing data as missing data imputation values similar to real data using a generative adversarial network (GAN) and allowing training with labeled data sets with labels, as well as and irregular data sets such as non-labeled data sets without labels.
1. A classification system based on a generative adversarial network, the system comprising:
a generator for generating a missing imputation value for a missing part among states from a labeled dataset;
a discriminator for discriminating the missing imputation value generated by the generator from original data;
an actor for predicting an action through a policy with the missing imputation value generated by the generator; and
a weighted function unit for generating a weight value of a reward on the basis of a state replaced with the missing imputation value, the predicted action, and a label of the labeled dataset, wherein
the weighted function unit operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold, and
the actor learns the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, wherein the weight value of the reward is defined by the following equation
W
(
s
^
,
a
,
y
)
=
r
(
s
^
)
*
{
ω
y
,
γ
=
a
,
right
prediction
ω
y
+
ω
a
2
,
y
≠
a
,
wrong
prediction
,
wherein
r(ŝ) is a reward that can be taken from state ŝ, a is an action predicted by policy π for a given state, y is a label of a state, and ωy and ωa are weight coefficients based on ω k =1−log b φ k (b is e based on logarithm, 10 . . . ).
2. The system according to claim 1 , wherein the weighted function unit operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold, wherein the label frequency is approximated as shown in the equation below
φ
k
=
n
k
∑
k
=
0
k
-
1
n
k
,
wherein
n k is the number of samples of the k-th label, and φ k is within a range of (0, 1), and the actor learns the policy to optimize the policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit.
3. The system according to claim 1 , wherein the weighted function unit operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold, and the actor learns the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, wherein learning the policy uses the equation shown below
L
L
=
{
-
𝔼
[
[
y
log
π
(
s
^
)
+
(
1
-
y
)
log
(
1
-
π
(
s
^
)
)
]
*
W
(
s
^
,
a
,
y
)
]
,
binomial
-
𝔼
[
∑
y
=
0
,
1
…
y
y
log
π
y
(
s
^
)
*
W
(
s
^
,
a
,
y
)
]
,
categorial
,
wherein Ŝ is a missing imputation value, y is a label of a state, a is an action predicted by policy π for a given state, and W (ŝ, a, y) is a weight value of a reward for the state, action, and label.
4. A generative adversarial network-based classification method using a classification system based on a generative adversarial network (GAN), which is configured of a generator, a discriminator, an actor, and a weighted function unit, the method comprising the steps of:
a) generating a missing imputation value for a missing part among states from a labeled dataset, by the generator;
b) predicting an action through a policy with the missing imputation value generated by the generator, by the actor;
c) generating a weight value of a reward on the basis of a state replaced with the missing imputation value, the predicted action, and a label of the labeled dataset, by the weighted function unit; and
d) learning the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, by the actor, wherein
at step c), the weighted function unit operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold,
wherein the weighted function unit of step c) operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold, wherein
the label frequency is approximated as shown in the equation below
φ
k
=
n
k
∑
k
=
0
k
-
1
n
k
,
wherein
nk is the number of samples of the k-th label, and φ k is within a range of (0, 1), and the weight value of the reward is defined by the following equation
W
(
s
^
,
a
,
y
)
=
r
(
s
^
)
*
{
ω
y
,
γ
=
a
,
right
prediction
ω
y
+
ω
a
2
,
y
≠
a
,
wrong
prediction
,
wherein
r(ŝ) is a reward that can be taken from state {tilde over (s)}, a is an action predicted by policy π for a given state, y is a label of a state, and ωy and ωa are weight coefficients based on ω k =1−log b φ k (b is e based on logarithm, 10 . . . ).
5. The method according to claim 4 , wherein step a) further includes:
i) a step of selecting a state having a missing value from the labeled dataset, and a missing indicator (m) indicating whether a state element corresponding to the state is missing, by the generator; and
ii) a preprocessing step of generating a missing imputation value ŝ using a missing imputation value {tilde over (s)} obtained by replacing the state with random noise randomly selected from a uniform distribution between ‘0’ and ‘1’, and learning the generator and the discriminator using the generated missing imputation value Ŝ.
6. The method according to claim 4 , wherein the weighted function unit of step c) operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold.
7. A generative adversarial network-based classification method using a classification system based on a generative adversarial network (GAN), which is configured of a generator, a discriminator, an actor, and a weighted function unit, the method comprising the steps of:
a) generating a missing imputation value for a missing part among states from a labeled dataset, by the generator;
b) predicting an action through a policy with the missing imputation value generated by the generator, by the actor;
c) generating a weight value of a reward on the basis of a state replaced with the missing imputation value, the predicted action, and a label of the labeled dataset, by the weighted function unit; and
d) learning the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, by the actor, wherein
at step c), the weighted function unit operates to balance labels by increasing the weight value of the reward for a label having a frequency lower than a predetermined threshold and decreasing the weight of the reward for a label having a frequency higher than a predetermined threshold,
wherein at step d), learning the policy is performed using the equation shown below
L
L
=
{
-
𝔼
[
[
y
log
π
(
s
^
)
+
(
1
-
y
)
log
(
1
-
π
(
s
^
)
)
]
*
W
(
s
^
,
a
,
y
)
]
,
binomial
-
𝔼
[
∑
y
=
0
,
1
…
y
y
log
π
y
(
s
^
)
*
W
(
s
^
,
a
,
y
)
]
,
categorial
,
wherein y is a label of a state, a is an action predicted by policy π for a given state, and W (ŝ, a, y) is a weight value of a reward for the state, action, and label.
8. A classification system based on a generative adversarial network, the system comprising:
a generator for generating a missing imputation value for a missing part among states from a labeled dataset S L or an unlabeled dataset S U ;
a discriminator for discriminating the missing imputation value generated by the generator from original data;
an actor for predicting an action through a policy with the missing imputation value generated by the generator;
a weighted function unit for generating a weight value of a reward on the basis of a state replaced with the missing imputation value, the predicted action, and a label of the labeled dataset; and
a reward unit for providing a reward so that the policy of the actor is learned targeting the labeled dataset and the unlabeled dataset, wherein
the actor learns the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, and learns the policy to optimize a semi-supervised policy loss function on the basis of the predicted action and the reward of the reward unit, and the reward of the reward unit is defined as shown in the following equation
r
(
s
^
u
,
a
)
=
{
1
,
R
(
s
^
L
,
a
)
≥
ɛ
0
,
otherwise
,
wherein
R(ŝ L , a) is the probability of whether the labeled dataset (Ŝ, a) pair output from the reward unit is a label of a labeled dataset with labels or a label generated by the actor, and ε∈[0, 1] is a threshold value considering whether a state-action pair is likely to be included in a labeled dataset.
9. A generative adversarial network-based classification method using a generative adversarial network (GAN) configured of a generator, a discriminator, an actor, a weighted function unit, and a reward unit, the method comprising the steps of:
a) generating a missing imputation value for a missing part among states from a labeled dataset S L , by the generator;
b) predicting an action through a policy with the missing imputation value generated by the generator, by the actor;
c) generating a weight value of a reward on the basis of a state replaced with the missing imputation value, the predicted action, and a label of the labeled dataset, by the weighted function unit; and
d) learning the policy to optimize a policy loss function by reflecting the predicted action and the weight value of the reward generated by the weighted function unit, by the actor, wherein
when there is an unlabeled dataset (S U ) at step a), the method further comprises the steps of:
a-1) generating a missing imputation value ŝ U for a missing part among states from an unlabeled dataset S U , by the generator;
b-1) predicting an action through a policy with the generated missing imputation value ŝ U , by the actor; and
c-1) providing a reward so that the policy of the actor is learned targeting the labeled dataset and the unlabeled dataset, by the reward unit; and
d-1) learning the policy to optimize a semi-supervised policy loss function on the basis of the predicted action and the reward of the reward unit, wherein
the reward of the reward unit is defined as shown in the following equation
r
(
s
^
u
,
a
)
=
{
1
,
R
(
s
^
L
,
a
)
≥
ɛ
0
,
otherwise
,
wherein
R(ŝ L , a) is the probability of whether the labeled dataset (Ŝ, a) pair output from the reward unit is a label of a labeled dataset with labels or a label generated by the actor, and ε∈=[0, 1] is a threshold value considering whether a state-action pair is likely to be included in a labeled dataset.