Method of encoding high band of audio and method of decoding high band of audio, and encoder and decoder for performing the methods
Disclosed are a method of encoding a high band of an audio, a method of decoding a high band of an audio, and an encoder and a decoder for performing the methods. The method of decoding a high band of an audio, the method performed by a decoder, includes identifying a parameter extracted through a first neural network, identifying side information extracted through a second neural network, and restoring a high band of an audio by applying the parameter and the side information to a third neural network.
1. A method of decoding a high band of an audio, the method performed by a decoder, the method comprising:
identifying a parameter extracted through a first neural network;
identifying side information extracted through a second neural network; and
restoring a high band of an audio by applying the parameter and the side information to a third neural network,
wherein, when a decoding frame of the audio is a current frame and there is at least one previous frame with respect to the current frame, a first input applied to the first neural network includes a high-band coefficient of the current frame, and a second input applied to the second neural network includes a high-band coefficient and a low-band coefficient of the previous frame, and a low-band coefficient of the current frame.
2. The method of claim 1 , wherein the first neural network is configured to extract the parameter of the high band from a first input based on a per-frame spectrum of the audio.
3. The method of claim 2 , wherein the first input is determined to be a subset of a spectrum, and
the spectrum includes a high-band coefficient and a low-band coefficient of a previous frame, and a high-band coefficient and a low-band coefficient of a current frame.
4. The method of claim 1 , wherein the second neural network is configured to extract the side information to restore the high band from a second input based on a per-frame spectrum of the audio.
5. The method of claim 4 , wherein the second input is determined to be a subset of a spectrum, and
the spectrum includes a high-band coefficient and a low-band coefficient of a previous frame, and a low-band coefficient of a current frame.
6. The method of claim 1 , wherein, when a decoding frame of the audio is a current frame, a first input applied to the first neural network includes a high-band coefficient of the current frame, and a second input applied to the second neural network includes a low-band coefficient of the current frame.
7. The method of claim 1 , wherein, when a decoding frame of the audio is a previous frame and there is at least one previous frame with respect to a current frame, a first input applied to the first neural network includes a high-band coefficient of the current frame and a high-band coefficient of the previous frame, and a second input applied to the second neural network includes a low-band coefficient of the previous frame.
8. The method of claim 1 , wherein the identifying of the parameter comprises identifying the parameter by dequantizing a quantized parameter received from an encoder.
9. The method of claim 1 , wherein the identifying of the parameter comprises identifying the extracted parameter by randomly sampling an output of the first neural network.
10. The method of claim 1 , wherein the restoring comprises using a high band of the current frame and a high band of at least one previous frame of the current frame.
11. A decoder, comprising:
a processor,
wherein the processor is configured to identify a parameter extracted through a first neural network, identify side information extracted through a second neural network, and restore a high band of an audio by applying the parameter and the side information to a third neural network, and
wherein, when a decoding frame of the audio is a current frame and there is at least one previous frame with respect to the current frame, a first input applied to the first neural network includes a high-band coefficient of the current frame, and a second input applied to the second neural network includes a high-band coefficient and a low-band coefficient of the previous frame, and a low-band coefficient of the current frame.
12. The decoder of claim 11 , wherein the first neural network is configured to extract the parameter of the high band from a first input based on a per-frame spectrum of the audio.
13. The decoder of claim 12 , wherein the first input is determined to be a subset of a spectrum, and
the spectrum includes a high-band coefficient and a low-band coefficient of a previous frame, and a high-band coefficient and a low-band coefficient of a current frame.
14. The decoder of claim 11 , wherein the second neural network is configured to extract the side information to restore the high band from a second input based on a per-frame spectrum of the audio.