Noise learning-based denoising autoencoder
Methods and apparatuses for noise learning-based denoising of noisy input data Y that is equal to the original data X plus the noise N (i.e., Y=X+N). In contrast with a conventional denoising autoencoder (DAE) method that attempts to learn the original data X directly from noisy input data Y, the noise learning-based denoising learns the noise N in the noisy input data Y and then regenerates the original data X by subtracting the learned noise N from the noisy input data Y. Learning the noise N may include inputting the noisy input data Y into an encoder of a neural network, and the learned noise N may be output from a decoder of the neural network. Training the neural network may include inputting noisy training data into an encoder of the neural network and outputting training noise from a decoder of the neural network.
1 . A denoising method comprising:
inputting noisy input data Y into an encoder of a neural network of a noise learning-based denoising autoencoder (nlDAE);
using the neural network of the nlDAE to learn noise N in the noisy input data Y;
outputting the learned noise N from a decoder of the neural network of the nlDAE; and
using a subtractor of the nlDAE to regenerate original data X by subtracting the learned noise N from the noisy input data Y;
wherein parameters θ and θ′ of the neural network are optimized as follows for all i ∈ {1, . . . , M}:
θ
*
,
θ
′
*
=
arg
min
1
M
∑
i
=
1
M
Loss
(
n
(
i
)
,
g
θ
′
(
f
θ
(
y
(
i
)
)
)
)
where Loss is a loss function, n is a realization vector of the noise N, y is a realization vector of the noisy input data Y, M is a number of training dataset, the parameter θ is {W,b}, W is a weight matrix for encoding, b is a bias vector for encoding, the parameter θ′ is {W,b}, W is a weight matrix for decoding, b′ is a bias vector for decoding, go is a decoding function of the decoder of the neural network, and ƒ θ is an encoding function of the encoder of the neural network.
2 . The method of claim 1 , further comprising training the neural network, wherein training the neural network comprises:
inputting noisy training data into an encoder of the neural network; and
outputting training noise from a decoder of the neural network.
3 . The method of claim 1 , wherein the noisy input data Y are subcarrier signals of an orthogonal frequency-division multiplexing (OFDM) scheme, the regenerated original data X are original subcarrier signals, and the method further comprises demodulating the original subcarrier signals.
4 . The method of claim 1 , wherein the noisy input data Y are estimated distances between a target node and reference nodes, and the method further comprises using the original data X to estimate a position of the target node.
5 . The method of claim 1 , wherein the noisy input data Y are a corrupted image, the noise N is corruptions in the corrupted image, and the original data X is an original image.
6 . The method of claim 1 , wherein ƒ θ (y)=S(Wy+b), g θ′ (ƒ θ (y))=S(W′(ƒ θ (y))+b′), and S is a sigmoid activation function for neural networks.
7 . The method of claim 1 , wherein
x
~
n
l
(
j
)
is a j-th regenerated original data and is represented as follows for all j ∈ {1, . . . , L}:
x
~
n
l
(
j
)
=
y
(
j
)
-
n
~
(
j
)
=
y
(
j
)
-
g
θ
′
*
(
f
θ
*
(
y
(
j
)
)
)
,
where L is a number of test dataset.
8 . The method of claim 1 , wherein the neural network of the nlDAE comprises an input layer, an output layer, and one or more hidden layers connecting the input and output layers.
9 . A denoising method comprising:
determining whether to use a noise learning-based denoising autoencoder (nlDAE) method or a denoising autoencoder (DAE) method that learns an original data X directly, wherein determining whether to use the nlDAE method or the DAE method is based on one or more of (i) a ratio between a standard deviation of a noise N and a standard deviation of the original data X, (ii) mutual information between the original data X and a noisy input data Y, and (iii) mutual information between the noise N and the noisy input data Y; and
in response to determining to use the nlDAE method:
inputting noisy input data Y into an encoder of a neural network of a nlDAE;
using the neural network of the nlDAE to learn noise N in the noisy input data Y;
outputting the learned noise N from a decoder of the neural network of the nlDAE; and
using a subtractor of the nlDAE to regenerate original data X by subtracting the learned noise N from the noisy input data Y.
10 . The method of claim 9 , wherein parameters θ and θ′ of the neural network are optimized as follows for all i ∈ {1, . . . , M}:
θ
*
,
θ
′
*
=
arg
min
1
M
∑
i
=
1
M
Loss
(
n
(
i
)
,
g
θ
′
(
f
θ
(
y
(
i
)
)
)
)
where Loss is a loss function, n is a realization vector of the noise N, y is a realization vector of the noisy input data Y, M is a number of training dataset, the parameter θ is {W,b}, W is a weight matrix for encoding, b is a bias vector for encoding, the parameter θ′ is {W′,b′}, W′ is a weight matrix for decoding, b′ is a bias vector for decoding, g θ′ is a decoding function of the decoder of the neural network, and ƒ θ is an encoding function of the encoder of the neural network.
11 . The method of claim 10 , wherein ƒ θ (y)=S(Wy+b), g θ′ (ƒ θ (y))=S(W′(ƒ θ (y))+b′), and S is a sigmoid activation function for neural networks.
12 . The method of claim 10 , wherein
x
~
n
l
(
j
)
is j-th regenerated original data and is represented as follows for all j ∈ {1, . . . , L}:
x
~
n
l
(
j
)
=
y
(
j
)
-
n
~
(
j
)
=
y
(
j
)
-
g
θ
′
*
(
f
θ
*
(
y
(
j
)
)
)
,
where L is a number of test dataset.
13 . The method of claim 9 , wherein determining whether to use the nlDAE method or the DAE method is based on at least the ratio between the standard deviation of the noise N and the standard deviation of the original data X.
14 . The method of claim 9 , wherein determining whether to use the nlDAE method or the DAE method is based on at least the mutual information between the original data X and the noisy input data Y.
15 . The method of claim 9 , wherein determining whether to use the nlDAE method or the DAE method is based on at least the mutual information between the noise N and the noisy input data Y.
16 . The method of claim 9 , wherein the neural network of the nlDAE comprises an input layer, an output layer, and one or more hidden layers connecting the input and output layers.
17 . A noise learning-based denoising autoencoder (nlDAE) comprising:
a neural network including an encoder and a decoder, wherein the neural network is configured to receive noisy input data Y at inputs to the encoder, learn noise N in the noisy input data Y, and output the learned noise D from the decoder; and
a subtractor configured to regenerate original data X by subtracting the learned noise N from the noisy input data Y;
wherein parameters θ and θ′ of the neural network are optimized as follows for all i ∈ {1, . . . , M}:
θ
*
,
θ
′
*
=
arg
min
1
M
∑
i
=
1
M
Loss
(
n
(
i
)
,
g
θ
′
(
f
θ
(
y
(
i
)
)
)
)
where Loss is a loss function, n is a realization vector of the noise N, y is a realization vector of the noisy input data Y, M is a number of training dataset, the parameter θ is {W,b}, W is a weight matrix for encoding, b is a bias vector for encoding, the parameter θ′ is {W′,b′}, W′ is a weight matrix for decoding, b′ is a bias vector for decoding, g θ′ is a decoding function of the decoder of the neural network, and ƒ θ is an encoding function of the encoder of the neural network.
18 . The nlDAE of claim 17 , wherein ƒ θ (y)=S(Wy+b), g θ′ (ƒ θ (y))=S(W′(ƒ θ (y))+b′), and S is a sigmoid activation function for neural networks.
19 . The nlDAE of claim 17 , wherein
x
~
n
l
(
j
)
is a j-th regenerated original data and is represented as follows for all j ∈ {1, . . . , L}:
x
~
n
l
(
j
)
=
y
(
j
)
-
n
~
(
j
)
=
y
(
j
)
-
g
θ
′
*
(
f
θ
*
(
y
(
j
)
)
)
,
where L is a number of test dataset.
20 . The nlDAE of claim 17 , wherein the neural network of the nlDAE comprises an input layer, an output layer, and one or more hidden layers connecting the input and output layers.