CCSO with downsampling filters methods
Various implementations described herein include methods and systems for coding video. In one aspect, a video bitstream includes a current image frame and a first syntax element for a cross-component sample offset (CCSO) mode. The first syntax element indicates whether to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples. An electronic device identifies a first luma sample that is collocated with the first color sample and neighboring luma sample(s) s of the first luma sample. The first luma sample and the one neighboring luma sample(s) are downsampled to generate downsampled luma sample(s) for the CCSO mode. The electronic device determines the first sample offset of the first color sample based on the downsampled luma sample(s), and reconstructs the current image frame at least by adjusting the first color sample based on the first sample offset.
1 . A method for decoding video data, comprising:
receiving a video bitstream including a current image frame, wherein the video bitstream includes a first syntax element indicating a downsampling index for a cross-component sample offset (CCSO) mode in which a first sample offset of a first color sample of the current image frame is generated based on one or more luma samples;
when the downsampling index is zero, reconstructing the current image frame without applying the CCSO mode; and
when the downsampling index is non-zero:
identifying a first luma sample that is collocated with the first color sample and one or more neighboring luma samples of the first luma sample;
downsampling, using a filter indicated by the downsampling index, the first luma sample and the one or more neighboring luma samples to generate one or more downsampled luma samples for the CCSO mode;
determining the first sample offset of the first color sample based on the one or more downsampled luma samples; and
reconstructing the current image frame at least by adjusting the first color sample based on the first sample offset.
2 . The method of claim 1 , wherein determining the first sample offset of the first color sample further comprises:
generating one or more quantized values based on the one or more downsampled luma samples; and
classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.
3 . The method of claim 2 , wherein the CCSO mode corresponds to at least an edge offset classifier, and generating one or more quantized values based on the one or more downsampled luma samples further comprises, based on the edge offset classifier:
determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples; and
determining one or more downsampled difference values between the one or more neighboring downsampled luma samples and the first downsampled luma sample;
wherein the one or more quantized values are generated based on the one or more downsampled difference values and applied by the edge offset classifier to classify the first color sample.
4 . The method of claim 2 , wherein:
the CCSO mode corresponds to an edge offset classifier and an alternative classifier;
the one or more quantized values include a first quantized value that is generated, and further applied by the edge offset classifier to classify the first color sample, based on the one or more downsampled luma samples; and
the one or more quantized values further include a second quantized value that is generated, and further applied by the alternative classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and
the first sample offset is determined based on an edge classification result and an alternative classification result.
5 . The method of claim 2 , wherein the CCSO mode corresponds to at least a band offset classifier, and generating one or more quantized values based on the one or more downsampled luma samples further comprises, based on the band offset classifier:
determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples;
wherein the one or more quantized values are generated based on the one or more downsampled luma samples, and applied by the band offset classifier, to classify the first color sample.
6 . The method of claim 2 , wherein:
the CCSO mode corresponds to a band offset classifier and an alternative classifier;
the one or more quantized values include a first quantized value that is generated, and further applied by the band offset classifier to classify the first color sample, based on the one or more downsampled luma samples; and
the one or more quantized values further include a second quantized value that is generated, and further applied by the alternative classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and
the first sample offset is determined based on an edge classification result and an alternative classification result.
7 . The method of claim 2 , wherein the CCSO mode corresponds to a plurality of offset classifiers, and generating one or more quantized values based on the one or more downsampled luma samples further comprises:
determining that the one or more downsampled luma samples include a first downsampled luma sample and one or more neighboring downsampled luma samples;
wherein the one or more quantized values are generated based on the one or more downsampled luma samples, and applied by at least one of the plurality of offset classifiers, to classify the first color sample and generate a respective classification result;
wherein the first sample offset is determined based on a plurality of classification results including the respective classification result of at least one of the plurality of offset classifiers.
8 . The method of claim 7 , wherein the plurality of offset classifiers include a band offset classifier and an edge offset classifier.
9 . The method of claim 1 , further comprising:
applying a chroma-from-luma (CfL) filter in a cross-component intra prediction (CCIP) mode to determine a first chroma sample, which is collocated with the first luma sample, based on a subset of the first luma sample and the one or more neighboring luma samples.
10 . The method of claim 1 , wherein the first luma sample and the one or more neighboring luma samples are downsampled using a first downsampling filter for the CCSO mode, the method further comprising:
applying a second downsampling filter to the first luma sample or the first color sample in a coding module distinct from the CCSO mode, wherein a type of the first downsampling filter matches a type of the second downsampling filter.
11 . The method of claim 1 , wherein the video bitstream further includes a second high-level syntax element selecting one or more classifiers applied in the CCSO mode to generate the first sample offset of the first color sample based on one or more luma samples, the second high-level syntax element having a first predefined value indicating that the one or more classifiers includes only a band offset classifier and a second predefined value indicating that the one or more classifiers includes both the band offset classifier and an edge offset classifier.
12 . The method of claim 11 , wherein a first quantized value is generated, and further applied by the band offset classifier to classify the first color sample, based on (1) the one or more downsampled luma samples or (2) a subset of the first luma sample and the one or more neighboring luma samples, the method further comprising:
classifying the first color sample based on at least the first quantized value to determine the first sample offset of the first color sample.
13 . The method of claim 1 , wherein:
the video bitstream further includes a third high-level syntax element for a downsampling mode when only a band offset classifier is applied in the CCSO mode;
the third high-level syntax element has a plurality of predefined values including a first predefined value and one or more second predefined values;
the first predefined value indicates that one or more quantized values are generated, and further applied only by the band offset classifier to classify the first color sample, based on a subset of the first luma sample and the one or more neighboring luma samples; and
the one or more second predefined values correspond to one or more types of a downsampling filter, and each second predefined value indicates that the one or more quantized values are generated, and further applied to classify the first color sample, based on the one or more downsampled luma samples generated by a respective type of the downsampling filter.
14 . The method of claim 1 , wherein the video bitstream further includes a fourth high-level syntax element for a downsampling filter mode indicating application conditions of the one or more downsampled luma samples to a band offset classifier and an edge offset classifier, and the fourth high-level syntax element corresponds to four predefined values that are uniquely associated with four application conditions including: the band offset classifier alone, a combination of the band offset classifier and the edge offset classifier, the edge offset classifier alone, and no application of either the band offset classifier or an edge offset classifier.
15 . The method of claim 1 , further comprising:
determining, on one of a high level and a block level, application conditions of the one or more downsampled luma samples to a band offset classifier and an edge offset classifier based on coded information, wherein the coded information includes one or more of: a distinct coded syntax value, a prediction mode, a block size, a quantization parameter, whether the current image frame is intra or inter coded, whether the first color sample is coded using intra block coding mode, and a palette coding mode; and
wherein based on the application conditions, one or more quantized values are generated, and further applied to classify the first color sample, based on the one or more downsampled luma samples.
16 . The method of claim 1 , wherein the first color sample is one of the first luma sample and a first chroma sample that is collocated with the first luma sample, and wherein each of the first luma sample and the one or more neighboring luma samples includes a respective reconstructed luma sample, and the first chroma sample includes a first reconstructed chroma sample.
17 . A computing system, comprising:
control circuitry; and
memory storing one or more programs configured to be executed by the control circuitry, the one or more programs further comprising instructions for:
receiving video data comprising a current image frame;
when a cross-component sample offset (CCSO) mode is not to be used for the current image frame, signaling, via a video bitstream, a syntax element with a value of zero;
when the CCSO mode is to be used for the current image frame:
encoding the current image frame using the CCSO mode to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples, wherein:
in the CCSO mode, the first sample offset of the first color sample is determined based on one or more downsampled luma samples that are generated by downsampling a first luma sample and one or more neighboring luma samples, and the first luma sample is collocated with the first color sample;
transmitting the encoded current image frame via the video bitstream; and
signaling, via the video bitstream, the syntax element with a value to indicate a filter to be used to perform the downsampling.
18 . The computing system of claim 17 , wherein determining the first sample offset of the first color sample comprises:
generating one or more quantized values based on the one or more downsampled luma samples; and
classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.
19 . A method of generating a video bitstream, the method comprising:
when a cross-component sample offset (CCSO) mode is not to be used for a current image frame of video data, setting a syntax element with a value of zero; and
when the CCSO mode is to be used for the current image frame:
encoding the current image frame using the CCSO mode to generate a first sample offset of a first color sample of the current image frame based on one or more luma samples, wherein, in the CCSO mode, the first sample offset of the first color sample is determined based on one or more downsampled luma samples that are generated by downsampling a first luma sample and one or more neighboring luma samples, and the first luma sample is collocated with the first color sample; and
setting the syntax element with a non-zero value that indicates a filter to be used to perform the downsampling; and
transmitting the video bitstream, wherein the video bitstream comprises the encoded current image frame and the syntax element.
20 . The method of claim 19 , wherein determining the first sample offset of the first color sample comprises:
generating one or more quantized values based on the one or more downsampled luma samples; and
classifying the first color sample based on the one or more quantized values to determine the first sample offset of the first color sample.
21 . The method of claim 19 , wherein the video bitstream further includes a second high-level syntax element selecting one or more classifiers applied in the CCSO mode to generate the first sample offset of the first color sample based on one or more luma samples, the second high-level syntax element having a first predefined value indicating that the one or more classifiers includes only a band offset classifier and a second predefined value indicating that the one or more classifiers includes both the band offset classifier and an edge offset classifier.