IP Library Granted Patent US 11,284,084
Granted Patent B1
US 11,284,084 · App. 17/357,093 · Granted Mar 22, 2022

Constraints on model-based reshaping in video processing

Inventors: Kai Zhang (San Diego, CA); Li Zhang (San Diego, CA); Hongbin Liu (Beijing, CN); Yue Wang (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/132H04N19/172H04N19/186H04N19/30H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,284,084
App. No.
17/357,093
Granted
Mar 22, 2022
Kind
B1
Abstract

A video processing method is provided, including: performing a conversion between a coded representation of a video including one or more video regions and the video, wherein the coded representation includes reshaping model information applicable for in loop reshaping (ILR) of some of the one or more video regions, wherein the reshaping model information provides information for a reconstruction of a video unit of a video region based on a representation in a first domain and a second domain and/or scaling chroma residue of a chroma video unit, and wherein the reshaping model information includes a parameter set that includes a syntax element specifying a difference between an allowed maximum bin index and a maximum bin index to be used in the reconstruction, and wherein the parameter is in a range.

Claims (47)

1. A method of processing video data, comprising:

determining, during a conversion between a current video unit of a video region of a video and a bitstream of the video, that a coding tool is used for the current video unit;

performing the conversion by using the coding tool during the conversion, in which the current video unit is constructed based on at least one of:

1) a forward mapping process for a luma component of the current video unit, in which prediction samples of the luma component are converted from an original domain to a reshaped domain;

2) an inverse mapping process, which is an inverse operation of the forward mapping process, in which reconstructed samples of the luma component of the current video unit in the reshaped domain are converted to the original domain; or

3) a scaling process, in which residual samples of a chroma component of the current video unit are scaled before being used to reconstruct the chroma component, and

wherein model information of the coding tool for the current video unit is included in the bitstream, and

wherein in response to the video region comprising one or more video units including the current video unit, each of the one or more video units share the same model information of the coding tool for the current video unit.

2. The method of claim 1 , wherein the one or more video units correspond to one or more slices, one or more tile groups, one or more coding tree units, or one or more coding units.

3. The method of claim 1 , wherein the video region is a picture.

4. The method of claim 1 , wherein the model of the coding tool includes a piecewise linear model which is used to map the prediction samples of the luma component of the current video unit into particular values during the forward mapping process.

5. The method of claim 4 , wherein a value of at least one variable of the piecewise linear model is dependent on a bit depth of the video.

6. The method of claim 4 , wherein a range of a value of a code word for an i-th piecewise of the piecewise linear model is dependent on a bit depth of the video, and the code word for the i-th piecewise is associated with the particular values.

7. The method of claim 1 , wherein the model information of the coding tool for the current video unit comprises a first syntax element and a second syntax element, the first syntax element specifying a difference between a maximum allowed bin index and a maximum bin index to be used in the forward mapping process, and the second syntax element specifying a minimum bin index to be used in the forward mapping process.

8. The method of claim 7 , wherein a value of the first syntax element is in a range from 0 to the maximum allowed bin index.

9. The method of claim 7 , wherein a value of the second syntax element is smaller than or equal to the maximum bin index to be used in the forward mapping process.

10. The method of claim 7 , wherein a value of the second syntax element is in a range from 0 to the maximum bin index to be used in the forward mapping process.

11. The method of claim 7 , wherein the maximum allowed bin index is equal to 15.

12. The method of claim 1 , wherein a filtering process is applied on the converted reconstructed samples in the original domain generated in the inverse mapping process.

13. The method of claim 6 , wherein the value of the code word for the i-th piecewise is determined based on the model information of the coding tool for the current video unit.

14. The method of claim 1 , wherein a usage flag of the coding tool is included in a sequence parameter set.

15. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.

16. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.

17. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, during a conversion between a current video unit of a video region of a video and a bitstream of the video, that a coding tool is used for the current video unit;

perform the conversion by using the coding tool during the conversion, in which the current video unit is constructed based on at least one of:

1) a forward mapping process for a luma component of the current video unit, in which prediction samples of the luma component are converted from an original domain to a reshaped domain;

2) an inverse mapping process, which is an inverse operation of the forward mapping process, in which reconstructed samples of the luma component of the current video unit in the reshaped domain are converted to the original domain; or

3) a scaling process, in which residual samples of a chroma component of the current video unit are scaled before being used to reconstruct the chroma component, and

wherein model information of the coding tool for the current video unit is included in the bitstream, and

wherein in response to the video region comprising one or more video units including the current video unit, each of the one or more video units share the same model information of the coding tool for the current video unit.

18. The apparatus of claim 17 , wherein the model information of the coding tool for the current video unit comprises a first syntax element and a second syntax element, the first syntax element specifying a difference between a maximum allowed bin index and a maximum bin index to be used in the forward mapping process, and the second syntax element specifying a minimum bin index to be used in the forward mapping process.

19. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, during a conversion between a current video unit of a video region of a video and a bitstream of the video, that a coding tool is used for the current video unit;

perform the conversion by using the coding tool during the conversion, in which the current video unit is constructed based on at least one of:

1) a forward mapping process for a luma component of the current video unit, in which prediction samples of the luma component are converted from an original domain to a reshaped domain;

2) an inverse mapping process, which is an inverse operation of the forward mapping process, in which reconstructed samples of the luma component of the current video unit in the reshaped domain are converted to the original domain; or

3) a scaling process, in which residual samples of a chroma component of the current video unit are scaled before being used to reconstruct the chroma component, and wherein model information of the coding tool for the current video unit is included in the bitstream, and

wherein in response to the video region comprising one or more video units including the current video unit, each of the one or more video units share the same model information of the coding tool for the current video unit.

20. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining, that a coding tool is used for a current video unit of a video region of a video;

generating the bitstream by using the coding tool, in which the current video unit is constructed based on at least one of:

1) a forward mapping process for a luma component of the current video unit, in which prediction samples of the luma component are converted from an original domain to a reshaped domain;

2) an inverse mapping process, which is an inverse operation of the forward mapping process, in which reconstructed samples of the luma component of the current video unit in the reshaped domain are converted to the original domain; or

3) a scaling process, in which residual samples of a chroma component of the current video unit are scaled before being used to reconstruct the chroma component, and

wherein model information of the coding tool for the current video unit is included in the bitstream, and

wherein in response to the video region comprising one or more video units including the current video unit, each of the one or more video units share the same model information of the coding tool for the current video unit.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2021
From: ZHANG, KAI; ZHANG, LI
To: BYTEDANCE INC.
Reel/Frame 056658/0843 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2021
From: LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 056658/0914 →
Continuity (1)
Continuation PCTCN2020078388 · Mar 9, 2020
Cited By (4)
US 12,301,845 US 12,368,841 US 12,445,653 US 12,483,700