IP Library Granted Patent US 11,570,464
Granted Patent B2
US 11,570,464 · App. 17/330,945 · Granted Jan 31, 2023

Interactions between in-loop reshaping and palette mode

Inventors: Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Hongbin Liu (Beijing, CN); Jizheng Xu (San Diego, CA); Yue Wang (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/52H04N19/105H04N19/117H04N19/132H04N19/159H04N19/176H04N19/184H04N19/186H04N19/30H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,570,464
App. No.
17/330,945
Granted
Jan 31, 2023
Kind
B2
Abstract

A method for video processing is provided to include: making a determination that a coding mode is enabled for a conversion between a current video block of a video and a coded representation of the video; and performing the conversion using a palette prediction mode wherein at least a palette of representative sample values is used for the current video block, and wherein, in the coding mode, the current video block is constructed based on samples in a first domain and a second domain and/or chroma residue is scaled in a luma-dependent manner.

Claims (54)

1. A method of processing video data, comprising:

making a determination that a palette mode is used for a first conversion between a first luma video block of a first video region of a video and a bitstream of the video;

performing the first conversion by using the palette mode in a reshaped domain for the first luma video block to generate intermediate luma reconstruction samples of the first luma video block; and

performing an inverse mapping process for the intermediate luma reconstruction samples of the first luma video block to generate final luma reconstruction samples of the first luma video block in an original domain;

wherein for a chroma video block of the first video region, whether to apply a luma-dependent chroma residue scaling process on chroma residual samples of the chroma video block is determined, and

wherein in the luma-dependent chroma residue scaling process, the chroma residual samples are scaled before being used to reconstruct the chroma video block based on a luma component of the first video region, and

wherein in the palette mode, the intermediate luma reconstruction samples of the first luma video block are derived from at least one set of representative sample values that represent values in the reshaped domain.

2. The method of claim 1 , wherein the at least one set of representative sample values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream.

3. The method of claim 1 , further comprising:

making a determination that an intra block copy mode is used for a second conversion between a second luma video block of the first video region of the video and the bitstream of the video;

performing the second conversion by using the intra block copy mode in a reshaped domain for the second luma video block to generate intermediate luma reconstruction samples of the second luma video block; and

performing an inverse mapping process for the intermediate luma reconstruction samples of the second luma video block to generate final luma reconstruction samples of the second luma video block in an original domain,

wherein in the intra block copy mode, a luma prediction block of the second luma video block is generated using at least a block vector pointing to a second video region that includes the second luma video block and wherein the intermediate luma reconstruction samples of the second luma video block are generated based on the luma prediction block.

4. The method of claim 3 , wherein the luma prediction block is generated in the reshaped domain, and a residual block is represented in the bitstream in the reshaped domain.

5. The method of claim 4 , wherein the intermediate luma reconstruction samples of the second luma video block are obtained based on a sum of samples of the residual block and samples of the luma prediction block.

6. The method of claim 1 , further comprising:

making a determination that an block-based delta pulse code modulation mode is used for a third conversion between a third luma video block of the first video region of the video and the bitstream of the video;

performing the third conversion by using the block-based delta pulse code modulation mode in a reshaped domain for the third luma video block to generate intermediate luma reconstruction samples of the third luma video block; and

performing an inverse mapping process for the intermediate luma reconstruction samples of the third luma video block to generate final luma reconstruction samples of the third luma video block in an original domain.

7. The method of claim 6 , wherein the block-based delta pulse code modulation mode is a vertical or horizontal block-based delta pulse code modulation mode for quantized residual samples represented in the bitstream in the reshaped domain.

8. The method of claim 7 , wherein the intermediate luma reconstruction samples of the third luma video block are obtained based on a sum of the quantized residual samples and samples of a luma prediction block of the third luma video block.

9. The method of claim 1 , wherein the intermediate luma reconstruction samples are utilized for the conversions between a subsequent non-inter video block and the bitstream of the video.

10. The method of claim 1 , wherein the first conversion includes encoding the first luma video block into the bitstream.

11. The method of claim 1 , wherein the first conversion includes decoding the first luma video block from the bitstream.

12. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

make a determination that a palette mode is used for a first conversion between a first luma video block of a first video region of a video and a bitstream of the video;

perform the first conversion by using the palette mode in a reshaped domain for the first luma video block to generate intermediate luma reconstruction samples of the first luma video block; and

perform an inverse mapping process for the intermediate luma reconstruction samples of the first luma video block to generate final luma reconstruction samples of the first luma video block in an original domain;

wherein for a chroma video block of the first video region, whether to apply a luma-dependent chroma residue scaling process on chroma residual samples of the chroma video block is determined, and

wherein in the luma-dependent chroma residue scaling process, the chroma residual samples are scaled before being used to reconstruct the chroma video block based on a luma component of the first video region, and

wherein in the palette mode, the intermediate luma reconstruction samples of the first luma video block are derived from at least one set of representative sample values that represent values in the reshaped domain.

13. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

make a determination that a palette mode is used for a first conversion between a first luma video block of a first video region of a video and a bitstream of the video;

perform the first conversion by using the palette mode in a reshaped domain for the first luma video block to generate intermediate luma reconstruction samples of the first luma video block; and

perform an inverse mapping process for the intermediate luma reconstruction samples of the first luma video block to generate final luma reconstruction samples of the first luma video block in an original domain;

wherein for a chroma video block of the first video region, whether to apply a luma-dependent chroma residue scaling process on chroma residual samples of the chroma video block is determined, and

wherein in the luma-dependent chroma residue scaling process, the chroma residual samples are scaled before being used to reconstruct the chroma video block based on a luma component of the first video region, and

wherein in the palette mode, the intermediate luma reconstruction samples of the first luma video block are derived from at least one set of representative sample values that represent values in the reshaped domain.

14. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

making a determination that a palette mode is used for a first conversion between a first luma video block of a first video region of a video and a bitstream of the video;

generating a bitstream of the video by using the palette mode in a reshaped domain for the first luma video block to generate intermediate luma reconstruction samples of the first luma video block; and

performing an inverse mapping process for the intermediate luma reconstruction samples of the first luma video block to generate final luma reconstruction samples of the first luma video block in an original domain;

wherein for a chroma video block of the first video region, whether to apply a luma-dependent chroma residue scaling process on chroma residual samples of the chroma video block is determined, and

wherein in the luma-dependent chroma residue scaling process, the chroma residual samples are scaled before being used to reconstruct the chroma video block based on a luma component of the first video region, and

wherein in the palette mode, the intermediate luma reconstruction samples of the first luma video block are derived from at least one set of representative sample values that represent values in the reshaped domain.

15. The apparatus of claim 12 , wherein the method further comprises:

performing the second conversion by using an intra block copy mode in a reshaped domain for the second luma video block to generate intermediate luma reconstruction samples of the second luma video block; and

performing an inverse mapping process for the intermediate luma reconstruction samples of the second luma video block to generate final luma reconstruction samples of the second luma video block in an original domain,

wherein in the intra block copy mode, a luma prediction block of the second luma video block is generated using at least a block vector pointing to a second video region that includes the second luma video block and wherein the intermediate luma reconstruction samples of the second luma video block are generated based on the luma prediction block.

16. The apparatus of claim 15 , wherein the luma prediction block is generated in the reshaped domain, and a residual block is represented in the bitstream in the reshaped domain.

17. The method of claim 3 , wherein the second conversion includes encoding the second luma video block into the bitstream.

18. The method of claim 3 , wherein the first conversion includes decoding the second luma video block from the bitstream.

19. The method of claim 6 , wherein the third conversion includes encoding the third luma video block into the bitstream.

20. The method of claim 6 , wherein the third conversion includes decoding the third luma video block from the bitstream.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED AT REEL: 056363 FRAME: 0136. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 24, 2021
From: LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 057278/0699 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2021
From: ZHANG, LI; ZHANG, KAI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 057722/0563 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 26, 2021
From: ZHANG, LI; ZHANG, KAI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 056363/0107 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 26, 2021
From: LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 056363/0136 →
Priority Claims (1)
WO PCT/CN2019/074437 · Feb 1, 2019 · international
Continuity (2)
Continuation PCTCN2020074144 · Feb 1, 2020
Related Publication 20210297669A1 · Sep 23, 2021