Adaptive wiener filter shape for video and image compression

1 . A method of video decoding, comprising:

receiving a bitstream that comprises coded information of one or more pictures;

generating, according to the bitstream, at least a to-be-filtered sample, and input samples for filtering the to-be-filtered sample in an in-loop filter, the in-loop filter being based on a Wiener-based filter with a predetermined filter shape;

identifying one or more outliers in the input samples when a difference between each of the one or more outliers and the to-be-filtered sample is outside of a shared range or a range associated with the respective outlier;

modifying the predetermined filter shape of the Wiener-based filter to obtain a modified filter with a modified filter shape based on exclusion of the one or more outliers from the modified filter; and

applying the modified filter with the modified filter shape on the input samples to generate a filtered output of the to-be-filtered sample.

2 . The method of claim 1 , wherein the identifying the one or more outliers comprises:

comparing a difference of a first input sample to the to-be-filtered sample with a threshold value, the first input sample being associated with one of filter taps of the Wiener-based filter, the threshold value being used for each filter tap of the filter taps of the Wiener-based filter; and

identifying the first input sample to be an outlier when the difference is out of a range defined according to the threshold value.

3 . The method of claim 1 , wherein the identifying the one or more outliers comprises:

selecting a threshold value from a plurality of predefined threshold candidates, the threshold value being used for each filter tap of filter taps of the Wiener-based filter;

comparing a difference of a first input sample to the to-be-filtered sample with the threshold value; and

identifying the first input sample to be an outlier when the difference is out of a range defined according to the threshold value.

4 . The method of claim 1 , wherein the identifying the one or more outliers comprises:

selecting respective threshold values for filter taps of the Wiener-based filter from a plurality of predefined threshold candidates;

comparing a difference of a first input sample to the to-be-filtered sample with a first respective threshold value selected for a first filter tap of the Wiener-based filter associated with the first input sample; and

identifying the first input sample to be an outlier when the difference is out of a range defined according to the respective threshold value.

5 . The method of claim 1 , wherein the identifying the one or more outliers comprises:

identifying a first input sample to be an outlier when a position of the first input sample is out of a boundary, the boundary being at least one of a boundary of a picture, a boundary of a slice, a boundary of a tile.

6 . The method of claim 1 , wherein the identifying the one or more outliers comprises:

identifying one or more input samples in the input samples to be outliers when the one or more input samples are located farther from the to-be-filtered sample than other input samples in the input samples.

7 . The method of claim 1 , wherein the modifying the predetermined filter shape comprises:

starting a pruning process of the input samples in a predetermined area from an immediate neighborhood of the to-be-filtered sample in an outwards order, the pruning process excluding the one or more outliers; and

determining, by the pruning process, a subset of the input samples having a same number of filter taps the predetermined filter shape.

8 . The method of claim 1 , wherein modifying the predetermined filter shape comprises:

calculating delta values of the input samples to the to-be-filtered sample; and

adjusting filter coefficients of the Wiener-based filter based on the delta values.

9 . The method of claim 1 , wherein the to-be-filtered sample and the input samples are of a first color component, the filtered output is of the first color component or is an offset of a second color component that is different from the first color component.

10 . The method of claim 1 , wherein the applying the modified filter comprises:

generating a weighted sum of the input samples excluding the one or more outliers, a non-outlier input sample in the input samples being weighted based on a filter coefficient of the Wiener-based filter associated with the non-outlier input sample.

11 . The method of claim 1 , wherein the applying the modified filter comprises:

calculating a scaling factor for filter coefficients associated with non-outlier input samples in the input samples; and

generating a weighted sum of the input samples excluding the one or more outliers, a non-outlier input sample in the input samples being weighted based on a filter coefficient of the Wiener-based filter associated with the non-outlier input sample and the scaling factor.

12 . The method of claim 1 , wherein the applying the modified filter comprises:

selecting a Wiener-based filter set from a plurality of Wiener-based filter sets according to the modified filter shape, each of the plurality of Wiener-based filter sets including a set of filter coefficients having a same number of filter taps.

13 . The method of claim 1 , wherein the applying the modified filter comprises:

applying the Wiener-based filter when no outlier is identified in the input samples; or

disabling the Wiener-based filter when more than a reference number of outliners are marked identified in the input samples.

14 . The method of claim 1 , further comprising:

determining a class of the to-be-filtered sample for applying the Wiener-based filter at least partially according to the modified filter shape;

deriving filter coefficients of the Wiener-based filter based on the class; and

calculating the filtered output of the to-be-filtered sample according to the filter coefficients.

15 . The method of claim 1 , further comprising:

adapting the modified filter shape according to the one or more outliers when a number of the one or more outliers is larger than a threshold.

16 . The method of claim 1 , further comprising:

decoding a signal from the coded information, the signal indicating that the predetermined filter shape of the Wiener-based filter is modifiable based on presence of the one or more outliers.

17 . A method of video encoding, comprising:

generating, during an encoding process of one or more pictures into a bitstream, at least a to-be-filtered sample and input samples for filtering the to-be-filtered sample in an in-loop filter, the in-loop filter being based on a Wiener-based filter with a predetermined filter shape;

identifying one or more outliers in the input samples when a difference between each of the one or more outliers and the to-be-filtered sample is outside of a shared range or a range associated with the respective outlier;

modifying the predetermined filter shape of the Wiener-based filter to obtain a modified filter with a modified filter shape based on exclusion of the one or more outliers from the modified filter;

applying the modified filter with the modified filter shape on the input samples to generate a filtered output of the to-be-filtered sample; and

encoding the one or more pictures into the bitstream based on the filtered output.

18 . The method of claim 17 , further comprising:

encoding a signal into the bitstream, the signal indicating that the predetermined filter shape of the Wiener-based filter is modifiable based on presence of the one or more outliers.

19 . The method of claim 17 , wherein the to-be-filtered sample and the input samples are of a first color component, the filtered output is of the first color component or is an offset of a second color component that is different from the first color component.

20 . A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform a method of encoding a bitstream, the method comprising:

generating, during an encoding process of one or more pictures into the bitstream, at least a to-be-filtered sample and input samples for filtering the to-be-filtered sample in an in-loop filter, the in-loop filter being based on a Wiener-based filter with a predetermined filter shape;

identifying one or more outliers in the input samples when a difference between each of the one or more outliers and the to-be-filtered sample is outside of a shared range or a range associated with the respective outlier;

modifying the predetermined filter shape of the Wiener-based filter to obtain a modified filter with a modified filter shape based on exclusion of the one or more outliers from the modified filter;

applying the modified filter with the modified filter shape on the input samples to generate a filtered output of the to-be-filtered sample;

encoding the one or more pictures into the bitstream based on the filtered output; and

transmitting the bitstream.