Deriving in-loop filter parameters for video coding
Deriving in-loop filter parameters via training for video encoding is provided. A video encoder performs inter prediction for a frame in a set of frames of the video to generate prediction residuals for the frame. The inter prediction for the frame is performed based on a reconstructed frame in the set of frames filtered using an in-loop filter. The value of a parameter of the in-loop filter is determined by determining, for each candidate in-loop filter parameter value, a visual quality metric for a set of training frames in training video sequences filtered by the in-loop filter. The candidate in-loop filter parameter value that corresponds the highest visual quality metric can be selected as the value of the parameter of the in-loop filter. The video encoder further encodes the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.
1. A method for encoding a video, the method comprising:
accessing a plurality of frames of the video;
performing inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein the inter prediction for the frame is performed based on a reconstructed frame in the plurality of frames filtered using an in-loop filter, wherein determining a value of a parameter of the in-loop filter comprises:
for each candidate in-loop filter parameter value among a plurality of candidate in-loop filter parameter values, determining a visual quality metric for a plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value, and
selecting a candidate in-loop filter parameter value among the plurality of candidate in-loop filter parameter values that corresponds a visual quality metric higher than another visual quality metric as the value of the parameter of the in-loop filter; and
encoding the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.
2. The method of claim 1 , wherein determining the visual quality metric for the plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value comprises:
for each training frame in the plurality of training frames, determining a frame-level visual quality metric for the training frame filtered by the in-loop filter with the value of the candidate in-loop filter parameter; and
combining the frame-level visual quality metrics for the respective training frames to generate the visual quality metric.
3. The method of claim 1 , wherein the value of the parameter of the in-loop filter is determined for a corresponding quantization parameter used in encoding the frame.
4. The method of claim 1 , wherein the reconstructed frame is an intra-coded frame (I-frame) and the plurality of training frames comprise I-frames.
5. The method of claim 1 , wherein the reconstructed frame filtered using the in-loop filter is a predicted frame (P-frame), and wherein determining the value of the parameter of the in-loop filter further comprises determining a value of the parameter of the in-loop filter for P-frames based on the selected candidate in-loop filter parameter value.
6. The method of claim 1 , wherein the value of the parameter of the in-loop filter for a Y component of the video is determined separately from the value of the parameter for a U component of the video or a V component of the video.
7. The method of claim 1 , wherein the visual quality metric comprises a peak signal to noise ratio (PSNR).
8. A system comprising:
a processor; and
at least one memory device including instructions that are executable by the processor to cause the processor to:
access a plurality of frames of a video;
perform inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein the inter prediction for the frame is performed based on a reconstructed frame in the plurality of frames filtered using an in-loop filter, wherein determining a value of a parameter of the in-loop filter comprises:
for each candidate in-loop filter parameter value among a plurality of candidate in-loop filter parameter values, determine a visual quality metric for a plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value, and
select a candidate in-loop filter parameter value among the plurality of candidate in-loop filter parameter values that corresponds a visual quality metric higher than another visual quality metric as the value of the parameter of the in-loop filter; and
encode the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.
9. The system of claim 8 , wherein determining the visual quality metric for the plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value comprises:
for each training frame in the plurality of training frames, determining a frame-level visual quality metric for the training frame filtered by the in-loop filter with the value of the candidate in-loop filter parameter; and
combining the frame-level visual quality metrics for the respective training frames to generate the visual quality metric.
10. The system of claim 8 , wherein the value of the parameter of the in-loop filter is determined for a corresponding quantization parameter used in encoding the frame.
11. The system of claim 8 , wherein the reconstructed frame is an intra-coded frame (I-frame) and the plurality of training frames comprise I-frames.
12. The system of claim 8 , wherein the reconstructed frame filtered using the in-loop filter is a predicted frame (P-frame), and wherein determining the value of the parameter of the in-loop filter further comprises determining a value of the parameter of the in-loop filter for P-frames based on the selected candidate in-loop filter parameter value.
13. The system of claim 8 , wherein the value of the parameter of the in-loop filter for a Y component of the video is determined separately from the value of the parameter for a U component of the video or a V component of the video.
14. The system of claim 8 , wherein the visual quality metric comprises a peak signal to noise ratio (PSNR).
15. A non-transitory computer-readable medium comprising program code that is executable by one or more processors to cause the one or more processors to:
access a plurality of frames of a video;
perform inter prediction for a frame in the plurality of frames to generate prediction residuals for the frame, wherein the inter prediction for the frame is performed based on a reconstructed frame in the plurality of frames filtered using an in-loop filter, wherein determining a value of a parameter of the in-loop filter comprises:
for each candidate in-loop filter parameter value among a plurality of candidate in-loop filter parameter values, determine a visual quality metric for a plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value, and
select a candidate in-loop filter parameter value among the plurality of candidate in-loop filter parameter values that corresponds a visual quality metric higher than another visual quality metric as the value of the parameter of the in-loop filter; and
encode the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.
16. The non-transitory computer-readable medium of claim 15 , wherein determining the visual quality metric for the plurality of training frames in one or more training video sequences filtered by the in-loop filter with the candidate in-loop filter parameter value comprises:
for each training frame in the plurality of training frames, determining a frame-level visual quality metric for the training frame filtered by the in-loop filter with the value of the candidate in-loop filter parameter; and
combining the frame-level visual quality metrics for the respective training frames to generate the visual quality metric.
17. The non-transitory computer-readable medium of claim 15 , wherein the value of the parameter of the in-loop filter is determined for a corresponding quantization parameter used in encoding the frame.
18. The non-transitory computer-readable medium of claim 15 , wherein the reconstructed frame is an intra-coded frame (I-frame) and the plurality of training frames comprise I-frames.
19. The non-transitory computer-readable medium of claim 15 , wherein the reconstructed frame filtered using the in-loop filter is a predicted frame (P-frame), and wherein determining the value of the parameter of the in-loop filter further comprises determining a value of the parameter of the in-loop filter for P-frames based on the selected candidate in-loop filter parameter value.
20. The non-transitory computer-readable medium of claim 15 , wherein the value of the parameter of the in-loop filter for a Y component of the video is determined separately from the value of the parameter for a U component of the video or a V component of the video.