Content-adaptive online training method and apparatus for deblocking in block-wise image compression
Aspects of the disclosure provide a method, an apparatus, and non-transitory computer-readable storage medium for video decoding. The apparatus includes processing circuitry that reconstructs blocks of an image that is to be reconstructed from a coded video bitstream. The processing circuitry decodes first deblocking information in the coded video bitstream including a first deblocking parameter of a deep neural network (DNN) in a video decoder. The first deblocking parameter of the DNN is an updated parameter that has been previously determined by a content adaptive training process. The processing circuitry determines the DNN for a first boundary region comprising a subset of samples in the reconstructed blocks based on the first deblocking parameter included in the first deblocking information. The processing circuitry deblocks the first boundary region comprising the subset of samples in the reconstructed blocks based on the determined DNN corresponding to the first deblocking parameter.
1. A method for video decoding in a video decoder, comprising:
reconstructing blocks of an image that is to be reconstructed from a coded video bitstream;
decoding first deblocking information in the coded video bitstream including a first deblocking parameter of a deep neural network (DNN) in the video decoder, wherein the first deblocking parameter of the DNN is an updated parameter that has been previously determined by a content adaptive training process;
determining the DNN in the video decoder for a first boundary region comprising a subset of samples in the reconstructed blocks based on the first deblocking parameter included in the first deblocking information; and
deblocking the first boundary region comprising the subset of samples in the reconstructed blocks based on the determined DNN corresponding to the first deblocking parameter.
2. The method of claim 1 , wherein
the reconstructed blocks include first neighboring reconstructed blocks that have a first shared boundary and include the first boundary region of samples on both sides of the first shared boundary;
the first neighboring reconstructed blocks further include non-boundary regions that are outside the first boundary region; and
the first boundary region in the first neighboring reconstructed blocks is replaced with the deblocked first boundary region.
3. The method of claim 2 , wherein
the reconstructed blocks include second neighboring reconstructed blocks that have a second shared boundary and include a second boundary region of samples on both sides of the second shared boundary; and
the method further includes:
decoding second deblocking information in the coded video bitstream corresponding to the second boundary region, the second deblocking information indicating a second deblocking parameter that has been previously determined by a content adaptive training process, the second boundary region being different from the first boundary region;
updating the DNN based on the first deblocking parameter and the second deblocking parameter, the updated DNN corresponding to the second boundary region and being configured with the first deblocking parameter and the second deblocking parameter; and
deblocking the second boundary region based on the updated DNN corresponding to the second boundary region.
4. The method of claim 2 , wherein
the reconstructed blocks include second neighboring reconstructed blocks of the reconstructed blocks that have a second shared boundary and include a second boundary region having samples on both sides of the second shared boundary; and
the method further includes deblocking the second boundary region based on the determined DNN corresponding to the first boundary region.
5. The method of claim 2 , wherein
the first boundary region further includes samples on both sides of a third shared boundary between third two neighboring reconstructed blocks included in the reconstructed blocks, and
the first two neighboring reconstructed blocks are different from the third two neighboring reconstructed blocks.
6. The method of claim 1 , wherein the first deblocking parameter is a bias term or a weight coefficient in the DNN.
7. The method of claim 1 , wherein
the DNN is configured with initial parameters, and
the determining the DNN includes updating one of the initial parameters based on the first deblocking parameter.
8. The method of claim 7 , wherein
the first deblocking information indicates a difference between the first deblocking parameter and the one of the initial parameters, and
the method further includes determining the first deblocking parameter according to a sum of the difference and the one of the initial parameters.
9. The method of claim 1 , wherein a number of layers of the DNN is dependent on a size of the first boundary region.
10. An apparatus for video decoding, comprising:
processing circuitry configured to:
reconstruct blocks of an image that is to be reconstructed from a coded video bitstream;
decode first deblocking information in the coded video bitstream including a first deblocking parameter of a deep neural network (DNN) in the video decoder, wherein the first deblocking parameter of the DNN is an updated parameter that has been previously determined by a content adaptive training process;
determine the DNN in the video decoder for a first boundary region comprising a subset of samples in the reconstructed blocks based on the first deblocking parameter included in the first deblocking information; and
deblock the first boundary region comprising the subset of samples in the reconstructed blocks based on the determined DNN corresponding to the first deblocking parameter.
11. The apparatus of claim 10 , wherein
the reconstructed blocks include first neighboring reconstructed blocks that have a first shared boundary and include the first boundary region of samples on both sides of the first shared boundary;
the first neighboring reconstructed blocks further include non-boundary regions that are outside the first boundary region; and
the first boundary region in the first neighboring reconstructed blocks is replaced with the deblocked first boundary region.
12. The apparatus of claim 11 , wherein
the reconstructed blocks include second neighboring reconstructed blocks that have a second shared boundary and include a second boundary region of samples on both sides of the second shared boundary; and
the processing circuitry is configured to:
decode second deblocking information in the coded video bitstream corresponding to the second boundary region, the second deblocking information indicating a second deblocking parameter that has been previously determined by a content adaptive training process, the second boundary region being different from the first boundary region;
update the DNN based on the first deblocking parameter and the second deblocking parameter, the updated DNN corresponding to the second boundary region and being configured with the first deblocking parameter and the second deblocking parameter; and
deblock the second boundary region based on the updated DNN corresponding to the second boundary region.
13. The apparatus of claim 11 , wherein
the reconstructed blocks include second neighboring reconstructed blocks of the reconstructed blocks that have a second shared boundary and include a second boundary region having samples on both sides of the second shared boundary; and
the processing circuitry is configured to deblock the second boundary region based on the determined DNN corresponding to the first boundary region.
14. The apparatus of claim 11 , wherein
the first boundary region further includes samples on both sides of a third shared boundary between third two neighboring reconstructed blocks included in the reconstructed blocks, and
the first two neighboring reconstructed blocks are different from the third two neighboring reconstructed blocks.
15. The apparatus of claim 10 , wherein the first deblocking parameter is a bias term or a weight coefficient in the DNN.
16. The apparatus of claim 10 , wherein
the DNN is configured with initial parameters, and
the processing circuitry is configured to update one of the initial parameters based on the first deblocking parameter.
17. The apparatus of claim 16 , wherein
the first deblocking information indicates a difference between the first deblocking parameter and the one of the initial parameters, and
the processing circuitry is configured to determine the first deblocking parameter according to a sum of the difference and the one of the initial parameters.
18. The apparatus of claim 10 , wherein a number of layers of the DNN is dependent on a size of the first boundary region.
19. A non-transitory computer-readable storage medium storing a program executable by at least one processor to perform:
reconstructing blocks of an image that is to be reconstructed from a coded video bitstream;
decoding first deblocking information in the coded video bitstream including a first deblocking parameter of a deep neural network (DNN) in the video decoder, wherein the first deblocking parameter of the DNN is an updated parameter that has been previously determined by a content adaptive training process;
determining the DNN in the video decoder for a first boundary region comprising a subset of samples in the reconstructed blocks based on the first deblocking parameter included in the first deblocking information; and
deblocking the first boundary region comprising the subset of samples in the reconstructed blocks based on the determined DNN corresponding to the first deblocking parameter.
20. The non-transitory computer-readable storage medium of claim 19 , wherein
the reconstructed blocks include first neighboring reconstructed blocks that have a first shared boundary and include the first boundary region of samples on both sides of the first shared boundary;
the first neighboring reconstructed blocks further include non-boundary regions that are outside the first boundary region; and
the first boundary region in the first neighboring reconstructed blocks is replaced with the deblocked first boundary region.