Cross-component filter
A video coding device generates first and second components of a current picture. Additionally, the video coding device determines a first parameter and a second parameter. The first and second parameters are each based on a value of a current sample in the first component. The video coding device applies a cross-component filter to the current sample, thereby determining a filtered value of the current sample based on the first parameter, the second parameter, and one or more cross-component samples. Each of the one or more cross-component samples is in the second component.
1. A method of coding video data performed by a video coding device, the method comprising:
generating a first component of a current picture of the video data, the first component comprising a first array of samples;
generating a second component of the current picture, the second component comprising a second array of samples separate from the first array of samples;
determining a first parameter, wherein the first parameter is based on a value of a current sample in the first component of the current picture;
determining a second parameter, wherein the second parameter is based on the value of the current sample;
applying a cross-component filter to the current sample as an in-loop filter after applying a deblocking filter, Sample Adaptive Offset (SAO) filter, or Adaptive Loop Filter (ALF) to a block of the current picture containing the current sample, thereby determining a filtered value of the current sample, wherein the cross-component filter is based on the first parameter, the second parameter, and one or more cross-component samples, each of the one or more cross-component samples being in the second component of the current picture; and
performing one or more actions in a group consisting of:
outputting a final version of the current picture, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample; and
using the final version of the current picture as a reference picture in encoding a later picture of the video data, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample.
2. The method of claim 1 , wherein applying the cross-component filter comprises determining the filtered value of the current sample according to a formula:
p′ c =αP cc +β,
where P′ c is the filtered value of the current sample, a is the first parameter, P cc is a cross-component sample of the one or more cross-component samples, and β is the second parameter.
3. The method of claim 1 , wherein:
the one or more cross-component samples include a plurality of cross-component samples,
the first parameter is a first instance of the first parameter,
the method comprises determining a plurality of first parameters that includes the first instance of the first parameter and one or more additional instances of the first parameter, each respective additional instance of the first parameter of the one or more additional instances of the first parameter being based on a value of a different corresponding sample in the first component, and
applying the cross-component filter comprises determining the filtered value of the current sample according to a formula:
P′ c =Σ i α i p i cc +β,
where P′ c is the filtered value of the current sample, i is a pixel index, α i is an i-th first parameter in the plurality of first parameters, P i cc is an i-th cross-component sample of cross-component samples, and β is the second parameter.
4. The method of claim 1 , wherein:
the one or more cross-component samples include a plurality of cross-component samples,
the first parameter is a first instance of the first parameter,
the method comprises determining a plurality of first parameters that includes the first instance of the first parameter and one or more additional instances of the first parameter, each respective additional instance of the first parameter of the one or more additional instances of the first parameter being based on a value of a different corresponding sample in the first component, and
applying the cross-component filter comprises determining the filtered value of the current sample according to a formula:
P′ c =Σ i α i p i cc +Σ j γ j p j n +β,
where P′ c is the filtered value of the current sample, i is a first pixel index, α i , is an i-th first parameter in the plurality of first parameters, p i cc is an i-th cross-component sample of the plurality of cross-component samples, j is a second pixel index, γ j is a j-th third parameter in a plurality of third parameters, p j n is a j-th sample that spatially neighbors the current sample, and β is the second parameter.
5. The method of claim 1 , further comprising determining a modified filtered value of the current sample as a weighted sum of the value of the current sample and the filtered value of the current sample, wherein the value of the pixel in the final version of the current picture is based on the modified filtered value of the current sample.
6. The method of claim 1 , wherein the current picture has a 4:4:4 color format or a 4:2:2 color format and the one or more cross-component samples include a sample in the second component that is collocated with the current sample.
7. The method of claim 1 , wherein:
the current picture has a 4:4:4 color format or a 4:2:2 color format,
the second component includes a set of neighboring samples that spatially neighbor a collocated sample in the second component,
the collocated sample in the second component is collocated with the current sample,
the method further comprises applying a spatial noise-reduction filter to the collocated sample and the set of neighboring samples, thereby deriving a filtered collocated sample, and
the cross-component filter is based on the first parameter, the second parameter, and the filtered collocated sample.
8. The method of claim 1 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between an upper luma sample above the chroma sample and a lower luma sample below the chroma sample, the upper luma sample and the lower luma sample being luma samples in the second component, and
the cross component filter is based on the first parameter, the second parameter, and either the upper luma sample or the lower luma sample.
9. The method of claim 1 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between an upper luma sample above the chroma sample and a lower luma sample below the chroma sample, the upper luma sample and the lower luma sample being luma samples in the second component,
the method further comprises using a weighted average of the upper luma sample and the lower luma sample to determine a virtual luma sample, and
the cross-component filter is based on the first parameter, the second parameter, and the virtual luma sample.
10. The method of claim 1 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between six luma samples in the second component,
the method further comprises using a weighted average of the six luma samples to determine a virtual luma sample, and
the cross-component filter is based on the first parameter, the second parameter, and the virtual luma sample.
11. The method of claim 1 , wherein:
the first component comprises a current set of samples, the current set of samples including the current sample and a plurality of neighbor samples that spatially neighbor the current sample in the first component,
the one or more cross-component samples includes a plurality of cross-component samples, wherein, for each respective neighbor sample in the plurality of neighbor samples, the plurality of cross-component samples includes a sample in the second component corresponding to the respective neighbor sample, the plurality of cross-component samples further including a sample of the second component corresponding to the current sample,
the first parameter is determined through minimization of a mean square error between the current set of samples and the plurality of cross-component samples given a predefined relation between the current set of samples and the plurality of cross-component samples, and
the second parameter is determined through minimization of a mean square error between the current set of samples and the plurality of cross-component samples given the predefined relation between the current set of samples and the plurality of cross-component samples.
12. The method of claim 11 , wherein in determining the first parameter and the second parameter more weight is placed on the current sample and a corresponding cross-component sample of the plurality of cross-component samples than the plurality of neighbor samples that spatially neighbor the current sample and the plurality of cross-component samples that correspond to the neighbor samples.
13. The method of claim 1 , wherein at least one of:
determining the first parameter comprises using a Gaussian filter or an edge-preserved filter; or
determining the second parameter comprises using the Gaussian filter or the edge-preserved filter.
14. The method of claim 1 , wherein the method further comprises:
including the filtered value of the current sample in a predictive block;
reconstructing the block by adding samples of the predictive block to corresponding samples of a residual block, wherein the block is in the first component; and
generating the final version of the current picture based on the reconstructed block.
15. A device for coding video data, the device comprising:
a memory configured to store the video data; and
one or more processors configured to:
generate a first component of a current picture of the video data, the first component comprising a first array of samples;
generate a second component of the current picture, the second component comprising a second array of samples separate from the first array of samples;
determine a first parameter, wherein the first parameter is based on a value of a current sample in the first component of the current picture;
determine a second parameter, wherein the second parameter is based on the value of the current sample;
apply a cross-component filter to the current sample as an in-loop filter after applying a deblocking filter, Sample Adaptive Offset (SAO) filter, or Adaptive Loop Filter (ALF) to a block of the current picture containing the current sample, thereby determining a filtered value of the current sample, wherein the cross-component filter is based on the first parameter, the second parameter, and one or more cross-component samples, each of the one or more cross-component samples being in the second component of the current picture; and
perform one or more actions in a group consisting of:
outputting a final version of the current picture, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample; and
using the final version of the current picture as a reference picture in encoding a later picture of the video data, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample.
16. The device of claim 15 , wherein the one or more processors are configured such that, as part of applying the cross-component filter, the one or more processors determine the filtered value of the current sample according to a formula:
P′ c =αP cc +β,
where P′ c is the filtered value of the current sample, α is the first parameter, P cc is a cross-component sample of the one or more cross-component samples, and β is the second parameter.
17. The device of claim 15 , wherein:
the one or more cross-component samples include a plurality of cross-component samples,
the first parameter is a first instance of the first parameter,
the one or more processors are configured to determine a plurality of first parameters that includes the first instance of the first parameter and one or more additional instances of the first parameter, each respective additional instance of the first parameter of the one or more additional instances of the first parameter being based on a value of a different corresponding sample in the first component, and
the one or more processors are configured such that, as part of applying the cross-component filter, the one or more processors determine the filtered value of the current sample according to a formula:
P′ c =Σ i α i p i cc +β,
where P′ c is the filtered value of the current sample, i is a pixel index, α i is an i-th first parameter in the plurality of first parameters, P i cc is an i-th cross-component sample of the plurality of cross-component samples, and β is the second parameter.
18. The device of claim 15 , wherein:
the one or more cross-component samples include a plurality of cross-component samples,
the first parameter is a first instance of the first parameter,
the one or more processors are configured to determine a plurality of first parameters that includes the first instance of the first parameter and one or more additional instances of the first parameter, each respective additional instance of the first parameter of the one or more additional instances of the first parameter being based on a value of a different corresponding sample in the first component, and
the one or more processors are configured such that, as part of applying the cross-component filter, the one or more processors determine the filtered value of the current sample according to a formula:
P′ c =Σ i α i p i cc +Σ j γ j p j n +β,
where P′ c is the filtered value of the current sample, i is a first pixel index, α i is an i-th first parameter in the plurality of first parameters, p i cc is an i-th cross-component sample of the plurality of cross-component samples, j is a second pixel index, γ j is an j-th third parameter in a plurality of third parameters, p j n is a j-th sample that spatially neighbors the current sample, and β is the second parameter.
19. The device of claim 15 , wherein the one or more processors are further configured to determine a modified filtered value of the current sample as a weighted sum of the value of the current sample and the filtered value of the current sample, wherein the value of the pixel in the final version of the current picture is based on the modified filtered value of the current sample.
20. The device of claim 15 , wherein the current picture has a 4:4:4 color format or a 4:2:2 color format and the one or more cross-component samples include a sample in the second component that is collocated with the current sample.
21. The device of claim 15 , wherein:
the current picture has a 4:4:4 color format or a 4:2:2 color format,
the second component includes a set of neighboring samples that spatially neighbor a collocated sample in the second component,
the collocated sample in the second component is collocated with the current sample,
the one or more processors are further configured to apply a spatial noise-reduction filter to the collocated sample and the set of neighboring samples, thereby deriving a filtered collocated sample, and
the cross-component filter is based on the first parameter, the second parameter, and the filtered collocated sample.
22. The device of claim 15 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between an upper luma sample above the chroma sample and a lower luma sample below the chroma sample, the upper luma sample and the lower luma sample being luma samples in the second component, and
the cross component filter is based on the first parameter, the second parameter, and either the upper luma sample or the lower luma sample.
23. The device of claim 15 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between an upper luma sample above the chroma sample and a lower luma sample below the chroma sample, the upper luma sample and the lower luma sample being luma samples in the second component,
the one or more processors are further configured to use a weighted average of the upper luma sample and the lower luma sample to determine a virtual luma sample, and
the cross-component filter is based on the first parameter, the second parameter, and the virtual luma sample.
24. The device of claim 15 , wherein:
the current picture has a 4:2:0 color format,
the current sample is a chroma sample at a position between six luma samples in the second component,
the one or more processors are further configured to use a weighted average of the six luma samples to determine a virtual luma sample, and
the cross-component filter is based on the first parameter, the second parameter, and the virtual luma sample.
25. The device of claim 15 , wherein:
the first component comprises a current set of samples, the current set of samples including the current sample and a plurality of neighbor samples that spatially neighbor the current sample in the first component,
the one or more cross-component samples includes a plurality of cross-component samples, wherein, for each respective neighbor sample in the plurality of neighbor samples, the plurality of cross-component samples includes a sample in the second component corresponding to the respective neighbor sample, the plurality of cross-component samples further including a sample of the second component corresponding to the current sample,
the first parameter is determined through minimization of a mean square error between the current set of samples and the plurality of cross-component samples given a predefined relation between the current set of samples and the plurality of cross-component samples, and
the second parameter is determined through minimization of a mean square error between the current set of samples and the plurality of cross-component samples given the predefined relation between the current set of samples and the plurality of cross-component samples.
26. The device of claim 15 , wherein the one or more processors are further configured to:
include the filtered value of the current sample in a predictive block;
reconstruct the block by adding samples of the predictive block to corresponding samples of a residual block, wherein the block is in the first component; and
generate the final version of the current picture based on the reconstructed block.
27. The device of claim 15 , wherein the one or more processors are further configured to:
receive the video data at a receiver of a wireless communication device;
store the video data in a memory of the wireless communication device; and
process the video data on one or more processors of the wireless communication device.
28. The device of claim 15 , wherein the device comprises a wireless communication device, further comprising a transmitter configured to transmit encoded video data.
29. A device for coding video data, the device comprising:
means for generating a first component of a current picture of the video data, the first component comprising a first array of samples;
means for generating a second component of the current picture, the second component comprising a second array of samples separate from the first array of samples;
means for determining a first parameter, wherein the first parameter is based on a value of a current sample in the first component of the current picture;
means for determining a second parameter, wherein the second parameter is based on the value of the current sample;
means for applying a cross-component filter to the current sample as an in-loop filter after applying a deblocking filter, Sample Adaptive Offset (SAO) filter, or Adaptive Loop Filter (ALF) to a block of the current picture containing the current sample, thereby determining a filtered value of the current sample, wherein the cross-component filter is based on the first parameter, the second parameter, and one or more cross-component samples, each of the one or more cross-component samples being in the second component of the current picture; and
means for performing one or more actions in a group consisting of:
outputting a final version of the current picture, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample; and
using the final version of the current picture as a reference picture in encoding a later picture of the video data, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample.
30. A computer-readable storage medium having instructions stored thereon, wherein execution of the instructions causes a video coding device to:
generate a first component of a current picture of the video data, the first component comprising a first array of samples;
generate a second component of the current picture, the second component comprising a second array of samples separate from the first array of samples;
determine a first parameter, wherein the first parameter is based on a value of a current sample in the first component of the current picture;
determine a second parameter, wherein the second parameter is based on the value of the current sample;
apply a cross-component filter to the current sample as an in-loop filter after applying a deblocking filter, Sample Adaptive Offset (SAO) filter, or Adaptive Loop Filter (ALF) to a block of the current picture containing the current sample, thereby determining a filtered value of the current sample, wherein the cross-component filter is based on the first parameter, the second parameter, and one or more cross-component samples, each of the one or more cross-component samples being in the second component of the current picture; and
perform one or more actions in a group consisting of:
outputting a final version of the current picture, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample; and
using the final version of the current picture as a reference picture in encoding a later picture of the video data, wherein a value of a pixel in the final version of the current picture is based on the filtered value of the current sample.