Neural networks for decoding
Methods and apparatus for training a Neural Network to recover a codeword of a Forward Error Correction (FEC) code are provided. Trainable parameters of the Neural Network are optimised to minimise a loss function. The loss function is calculated by representing an estimated value of the message bit output from the Neural Network as a probability of the value of the bit in a predetermined real number domain and multiplying the representation of the estimated value of the message bit by a representation of a target value of the message bit. Training a neural network may be implemented via a loss function.
1. The method for training a Neural Network, NN, to recover a codeword of a Forward Error Correction, FEC, code from a received signal, wherein layers of the NN implement sequential iterations of the Sum Product Algorithm, SPA, and wherein the received signal comprises a transmitted codeword and channel impairments, the method comprising:
inputting to an input layer of the NN a representation of message bits of a transmitted codeword obtained from a received signal;
propagating the representation through the NN;
calculating a loss function; and
optimising trainable parameters of the NN to minimise the loss function;
wherein calculating a loss function comprises, for bits in the transmitted codeword:
representing an estimated value of the message bit output from the NN as a probability of the value of the bit in a predetermined real number domain; and
multiplying the representation of the estimated value of the message bit by a representation of a target value of the message bit.
2. The method as claimed in claim 1 , wherein calculating a loss function further comprises:
averaging over all bits in the transmitted codeword, the values obtained from multiplying, for bits in the transmitted codeword, the representation of the estimated value of the message bit by a representation of a target value of the message bit.
3. The method as claimed in claim 1 , wherein representing an estimated value of the message bit output from the NN as a probability of the value of the bit in a real number domain comprises:
obtaining a probability of the value of the bit from a layer of the NN; and
transforming the obtained probability to a value within the predetermined real number domain.
4. The method as claimed in claim 3 , wherein the predetermined real number domain is [−1, 1] and wherein transforming the obtained probability to a value within the predetermined real number domain comprises performing a linear transformation on the obtained probability.
5. The method as claimed in claim 1 , wherein the representation of the target value of the message bit comprises a value of the message bit after modulation using a modulation technique applied to the transmitted codeword.
6. The method as claimed in claim 1 , wherein calculating a loss function comprises:
calculating the loss function on the basis of an estimated value of the message bit output from an output layer of the NN.
7. The method as claimed in claim 1 , wherein the loss function comprises:
L
f
E
(
p
,
y
)
=
-
1
N
Σ
n
=
1
N
(
(
1
-
2
p
(
n
)
)
(
-
1
)
y
(
n
)
)
wherein:
N is the number of bits in the transmitted codeword;
p(n) is the probability of the value of the n th bit of the transmitted codeword output by the NN being 1; and
y(n) is the target value of the n th bit of the transmitted codeword.
8. The method as claimed in claim 1 , wherein calculating a loss function comprises:
calculating the loss function on the basis of estimated values of the message bit output from even layers of the NN.
9. The method as claimed in claim 8 , wherein the loss function comprises:
L
m
E
(
p
,
y
)
=
-
1
MN
∑
l
=
2
,
4
,
…
2
M
(
∑
n
=
1
N
(
(
1
-
2
p
(
l
,
n
)
)
(
-
1
)
y
(
n
)
)
)
wherein:
N is the number of bits in the transmitted codeword;
2M is the number of hidden layers in the NN;
p(n) is the probability of the value of the nth bit of the transmitted codeword output by the 1-th layer of the NN being 1; and
y(n) is the target value of the nth bit of the transmitted codeword.
10. The method as claimed in claim 1 , wherein the training codeword comprises a codeword of a binary linear block code.
11. The method as claimed in claim 1 , wherein the representation of message bits obtained from a received signal that is input to the input layer of the NN comprises an array of Log-Likelihood Ratios, LLRs, of the individual message bits obtained from the received signal.
12. The method as claimed in claim 1 , wherein the NN comprises a Neural Network Decoder, NND.
13. A computer program comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method according to claim 1 .
14. A controller for training a Neural Network, NN, to recover a codeword of a Forward Error Correction, FEC, code from a received signal, wherein layers of the NN implement sequential iterations of the Sum Product Algorithm, SPA, and wherein the received signal comprises a transmitted codeword and channel impairments, the controller comprising a processor and a memory, the memory containing instructions executable by the processor such that the controller is operable to:
input to an input layer of the NN a representation of message bits of a transmitted codeword obtained from a received signal;
propagate the representation through the NN;
calculate a loss function; and
optimise trainable parameters of the NN to minimise the loss function;
wherein calculating a loss function comprises, for bits in the transmitted codeword:
representing an estimated value of the message bit output from the NN as a probability of the value of the bit in a predetermined real number domain; and
multiplying the representation of the estimated value of the message bit by a representation of a target value of the message bit.
15. The controller as claimed in claim 14 , wherein the controller is further operable to perform operations of:
inputting to an input layer of the NN a representation of message bits of a transmitted codeword obtained from a received signal;
propagating the representation through the NN;
calculating a loss function; and
optimising trainable parameters of the NN to minimise the loss function;
wherein calculating a loss function comprises, for bits in the transmitted codeword:
representing an estimated value of the message bit output from the NN as a probability of the value of the bit in a predetermined real number domain; and
multiplying the representation of the estimated value of the message bit by a representation of a target value of the message bit.
16. The controller as claimed in claim 14 , wherein the controller comprises a virtualized network function.
17. A base station comprising the controller as claimed in claim 14 .
18. A controller for training a Neural Network, NN, to recover a codeword of a Forward Error Correction, FEC, code from a received signal, wherein layers of the NN implement sequential iterations of the Sum Product Algorithm, SPA, and wherein the received signal comprises a transmitted codeword and channel impairments, the controller adapted to:
input to an input layer of the NN a representation of message bits of a transmitted codeword obtained from a received signal;
propagate the representation through the NN;
calculate a loss function; and
optimise trainable parameters of the NN to minimise the loss function;
wherein calculating a loss function comprises, for bits in the transmitted codeword:
representing an estimated value of the message bit output from the NN as a probability of the value of the bit in a predetermined real number domain; and
multiplying the representation of the estimated value of the message bit by a representation of a target value of the message bit.