IP Library Granted Patent US 12,010,333
Granted Patent B2
US 12,010,333 · App. 18/083,956 · Granted Jun 11, 2024

Parameter derivation in cross component mode

Inventors: Zhipin Deng (Beijing, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Hongbin Liu (Beijing, CN); Jizheng Xu (San Diego, CA)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/30H04N19/105H04N19/132H04N19/146H04N19/176H04N19/186H04N19/196H04N19/70H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,010,333
App. No.
18/083,956
Granted
Jun 11, 2024
Kind
B2
Abstract

A method for visual media processing, including performing a conversion between a current chroma video block of visual media data and a bitstream representation of the current chroma video block. During the conversion, a chroma residual of the current chroma video block is scaled based on a scaling coefficient. The scaling coefficient is derived at least based on luma samples located in predefined positions.

Claims (75)

1. A method of processing video data, comprising:

determining, during a conversion between a first chroma video block of a video and a bitstream of the video, that a scaling process is applied on chroma residual samples of the first chroma video block; and

performing the conversion by applying the scaling process on the chroma residual samples, wherein in the scaling process, the chroma residual samples are scaled based on at least one scaling factor before being used to reconstruct the first chroma video block,

wherein the at least one scaling factor is derived based on an averaged luma variable computed based on neighboring luma samples of a video unit of a video region of the video which is determined based on a luma sample corresponding to a top-left sample of the first chroma video block,

wherein the determining that the scaling process is applied on the chroma residual samples of the first chroma video block is based on a syntax element indicative of selectively enabling or disabling the scaling process in the bitstream of the video when included, wherein in response to the scaling process being enabled, the scaling process is applied on the chroma residual samples based on the at least one scaling factor,

wherein in response to the scaling process being disabled, the scaling process refrains from being applied on the chroma residual samples, and

wherein the syntax element, when included in the bitstream, is included in a picture level associated with a current video block.

2. The method of claim 1 , wherein the neighboring luma samples are located in predefined positions neighboring to the video unit.

3. The method of claim 2 , wherein the neighboring luma samples located in the predefined positions neighboring to the video unit include reconstructed luma samples external to the video unit.

4. The method of claim 2 , wherein the neighboring luma samples located in the predefined positions neighboring to the video unit include reconstructed luma samples adjacent to the video unit,

wherein the reconstructed luma samples adjacent to the video unit include at least one of one or more left neighboring sample columns or one or more above neighboring sample rows of the video unit,

wherein in response to the one or more left neighboring sample columns and the one or more above neighboring sample rows of the video unit being determined to be unavailable, the at least one scaling factor is derived by setting the averaged luma variable equal to a default value, and

wherein the default value equals to 1<<(bitDepth−1), where bitDepth is a bit depth of the video.

5. The method of claim 1 , wherein a location of the top-left luma sample of the video unit is derived by using a size information of a virtual pipeline data unit, and

wherein a total number of the neighboring luma samples of the video unit is N, where N is an integer greater than 1, and a range of N depends on a size information of the video unit.

6. The method of claim 1 , wherein the at least one scaling factor is derived by:

checking an availability of each of one or more neighboring luma blocks of the video unit, wherein the each of the one or more neighboring luma blocks comprise at least one sample of the neighboring luma samples;

determining, based on the availability of the each of the one or more neighboring luma blocks, whether to retrieve the neighboring luma samples of the video unit; and

deriving the at least one scaling factor based on the averaged luma variable computed using the neighboring luma samples by a rounding-based average operation in case that the neighboring luma samples are available,

wherein the checking the availability of each of one or more neighboring luma blocks comprises: checking the availability of each of one or more neighboring luma blocks based on at least one of a width and a height of the video region, and

wherein in response to a top-left sample of the one neighboring block locating outside the video region, the one neighboring block is treated as unavailable.

7. The method of claim 1 , wherein, for a luma video block of the video region, at least one of the following is performed:

1) A forward mapping process for the luma video block, in which prediction samples of the luma video block are converted from an original domain to a reshaped domain; or

2) An inverse mapping process, which is an inverse operation of the forward mapping process, in which reconstructed samples of the luma video block in the reshaped domain are converted to the original domain, and

wherein the neighboring luma samples include reconstructed samples in the reshaped domain,

wherein the video region is a picture, and

wherein the scaling process is based on a piecewise linear model, and wherein an index identifying a piece to which the averaged luma variable belongs, and the at least one scaling factor is derived based on the index.

8. The method of claim 1 , wherein the syntax element is included in a picture header,

wherein in response to a value of the syntax element being equal to one, the application of the scaling process below the picture level is enabled, and in response to the value of the syntax element being equal to zero, the application of the scaling process below the picture level is disabled, and

wherein in response to the syntax element being not included in the bitstream, it is inferred that the application of the scaling process below the picture level is disabled, and a value of the syntax element is inferred to be zero.

9. The method of claim 1 , further comprising:

determining, for a conversion between a second chroma video block of the video and the bitstream of the video, whether to enable or disable a cross-component linear model intra prediction on the second chroma video block, based on whether one or more luma coding units which cover at least one sample of a corresponding luma region of the second chroma video block are coded with a subblock-based prediction mode, wherein the second chroma video block is in a dual tree structure in which a luma component and at least one chroma component have separate partition tree structures;

in response to the cross-component linear model intra prediction being enabled, determining parameters of the cross-component linear model at least based on neighboring chroma samples of the second chroma video block;

applying the cross-component linear model to derive prediction values of the second chroma video block based on the parameters; and

performing the conversion based on the prediction values.

10. The method of claim 9 , wherein the subblock-based prediction mode corresponds to an intra sub-partitions mode, and

wherein the cross-component linear model intra prediction is disabled in response to the one or more luma coding units being coded with the subblock-based prediction mode.

11. The method of claim 9 , wherein whether to enable or disable the cross-component linear model intra prediction on the second chroma video block is further based on partition information of the one or more luma coding units,

wherein the partition information includes a partition tree depth of the one or more luma coding units, and

wherein the partition tree depth is a quadtree depth.

12. The method of claim 9 , wherein whether to enable or disable the cross-component linear model intra prediction on the second chroma video block is further based on a block dimension of one or more chroma coding units which cover at least one sample of a corresponding chroma region of the second chroma video block, or

wherein whether to enable or disable the cross-component linear model intra prediction on the second chroma video block is further based on whether the one or more luma coding units are split from multi-type partition tree operation different from a quadtree partition operation.

13. The method of claim 9 , wherein the parameters of the cross-component linear model are derived by following:

checking an availability of one or more neighboring chroma blocks of the second chroma video block;

finding R chroma samples from neighboring chroma samples of the one or more neighboring chroma blocks to determine a set of values for the parameters of the cross-component linear model, wherein R is an integer greater than 2,

wherein at least one neighboring chroma sample does not belong to the R chroma samples based on a size of the second chroma video block.

14. The method of claim 13 , wherein in response to the one or more neighboring chroma blocks being unavailable, the prediction values of the second chroma video block are set to 1<<(bitDepth−1), where bitDepth is a bit depth of the video.

15. The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.

16. The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.

17. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, during a conversion between a first chroma video block of a video and a bitstream of the video, that a scaling process is applied on chroma residual samples of the first chroma video block; and

perform the conversion by applying the scaling process on the chroma residual samples,

wherein in the scaling process, the chroma residual samples are scaled based on at least one scaling factor before being used to reconstruct the first chroma video block,

wherein the at least one scaling factor is derived based on an averaged luma variable computed based on neighboring luma samples of a video unit of a video region of the video which is determined based on a luma sample corresponding to a top-left sample of the first chroma video block,

wherein the determining that the scaling process is applied on the chroma residual samples of the first chroma video block is based on a syntax element indicative of selectively enabling or disabling the scaling process in the bitstream of the video when included, wherein in response to the scaling process being enabled, the scaling process is applied on the chroma residual samples based on the at least one scaling factor,

wherein in response to the scaling process being disabled, the scaling process refrains from being applied on the chroma residual samples, and

wherein the syntax element, when included in the bitstream, is included in a picture level associated with a current video block.

18. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, during a conversion between a first chroma video block of a video and a bitstream of the video, that a scaling process is applied on chroma residual samples of the first chroma video block; and

perform the conversion by applying the scaling process on the chroma residual samples, wherein in the scaling process, the chroma residual samples are scaled based on at least one scaling factor before being used to reconstruct the first chroma video block,

wherein the at least one scaling factor is derived based on an averaged luma variable computed based on neighboring luma samples of a video unit of a video region of the video which is determined based on a luma sample corresponding to a top-left sample of the first chroma video block,

wherein the determining that the scaling process is applied on the chroma residual samples of the first chroma video block is based on a syntax element indicative of selectively enabling or disabling the scaling process in the bitstream of the video when included, wherein in response to the scaling process being enabled, the scaling process is applied on the chroma residual samples based on the at least one scaling factor,

wherein in response to the scaling process being disabled, the scaling process refrains from being applied on the chroma residual samples, and

wherein the syntax element, when included in the bitstream, is included in a picture level associated with a current video block.

19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus,

wherein the method comprises:

determining that a scaling process is applied on chroma residual samples of a first chroma video block; and

generating the bitstream by applying the scaling process on the chroma residual samples, wherein in the scaling process, the chroma residual samples are scaled based on at least one scaling factor before being used to reconstruct the first chroma video block,

wherein the at least one scaling factor is derived based on an averaged luma variable computed based on neighboring luma samples of a video unit of a video region of the video which is determined based on a luma sample corresponding to a top-left sample of the first chroma video block,

wherein the determining that the scaling process is applied on the chroma residual samples of the first chroma video block is based on a syntax element indicative of selectively enabling or disabling the scaling process in the bitstream of the video when included, wherein in response to the scaling process being enabled, the scaling process is applied on the chroma residual samples based on the at least one scaling factor,

wherein in response to the scaling process being disabled, the scaling process refrains from being applied on the chroma residual samples, and

wherein the syntax element, when included in the bitstream, is included in a picture level associated with a current video block.

20. The apparatus of claim 17 , wherein the syntax element is included in a picture header,

wherein in response to a value of the syntax element being equal to one, the application of the scaling process below the picture level is enabled, and in response to the value of the syntax element being equal to zero, the application of the scaling process below the picture level is disabled, and

wherein in response to the syntax element being not included in the bitstream, it is inferred that the application of the scaling process below the picture level is disabled, and a value of the syntax element is inferred to be zero.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2023
From: ZHANG, LI; ZHANG, KAI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 063825/0218 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2023
From: DENG, ZHIPIN; LIU, HONGBIN
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063825/0246 →
Priority Claims (1)
WO PCT/CN2019/083320 · Apr 18, 2019 · international
Continuity (3)
Continuation 17405212 · Aug 18, 2021
Continuation PCTCN2020085674 · Apr 20, 2020
Related Publication 20230117813A1 · Apr 20, 2023