IP Library › Granted Patent US 12,231,693
Granted Patent B2
US 12,231,693 · App. 18/485,088 · Granted Feb 18, 2025

Entropy coding for palette escape symbol

Inventors: Weijia Zhu (San Diego, CA); Jizheng Xu (San Diego, CA); Li Zhang (San Diego, CA); Hsiao Chiang Chuang (San Diego, CA)
Assignee: Bytedance Inc.
H04N19/70H04N19/132H04N19/176H04N19/186
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,231,693
App. No.
18/485,088
Granted
Feb 18, 2025
Kind
B2
Abstract

Methods, systems, and devices for performing entropy coding for a palette escape symbol in palette mode coding and decoding are described. An example method for video processing includes performing a conversion between a video having one or more video regions including a current video block and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule that the current video block is coded using a palette mode coding tool, wherein a binarization of an escape symbol for the current video block uses an exponential-Golomb (EG) code of order K, wherein K is a non-negative integer that is unequal to three, and wherein the palette mode coding tool represents the current video block using a palette of representative color values and wherein the escape symbol is used for a sample of the current video block coded without using the representative color values.

Claims (61)

1. A method for processing video data, comprising:

determining, for a conversion between a current video block of a video and a bitstream of the video, that a prediction mode is applied to the current video block, wherein in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream; and

performing the conversion at least based on the prediction mode,

wherein a first syntax element specifying a quantized value of the escaped sample is included in the bitstream, and wherein a binarization of the first syntax element uses an exponential-Golomb (EG) code of order K, K a non-negative integer that is unequal to three,

wherein a reconstructed escaped sample is generated based on a Clip function and the quantized value of the escaped sample,

wherein the reconstructed escaped sample is further determined based on qP which specifies a quantization parameter, and

wherein the qP is determined based on Max (QpPrimeTsMin, Qp′Y), where QpPrimeTsMin denotes a minimum allowed quantization parameter for a transform skip mode, and Qp′Y denotes a luma quantization parameter.

2. The method of claim 1 , wherein K=5.

3. The method of claim 1 , wherein the reconstructed escape sample is determined based on Clip3 (0, (1<<BitDepth)−1, tmpVal),

wherein tmpVal is determined based on m<< (qP/6)+32)>>6, and

wherein m is determined based on the quantized value of the escaped sample.

4. The method of claim 1 , wherein QpPrimeTsMin is defined as 6*n+4, wherein n is a value of a second syntax element included in the bitstream.

5. The method of claim 4 , wherein the second syntax element is included in a sequence level of the bitstream.

6. The method of claim 1 , wherein the current video block is a luma block,

wherein a current palette comprising one or more palette predictors derived from a palette prediction table is constructed for the current video block, wherein the conversion is performed further based on the current palette,

wherein a determination that a local dual tree coding is applied to the current video block is made, and

wherein based on the determination, the palette prediction table is modified according to the current palette, wherein modifying the palette prediction table comprises a reset process.

7. The method of claim 6 , wherein the local dual tree coding disables partitioning of a chroma parent block and enables partitioning of a corresponding luma parent block based on a size of the chroma parent block,

wherein the luma parent block is generated from a luma coding tree block based on a luma partition scheme including recursive partition operations, and the chroma parent block is generated from a chroma coding tree block based on a chroma partition scheme having same recursive partition operations with the luma partition scheme.

8. The method of claim 6 , wherein the reset process includes resetting an entry of the palette prediction table to a fixed value.

9. The method of claim 1 ,

wherein when the current video block is a chroma block, the quantization parameter is derived from a variable obtained by performing a mapping operation on a luma quantization parameter of a luma video block corresponding to the current video block.

10. The method of claim 9 , wherein the quantization parameter is derived based on the variable and a clip function.

11. The method of claim 1 , wherein the minimum allowed quantization parameter is indicated in a sequence parameter set in the bitstream.

12. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.

13. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.

14. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a current video block of a video and a bitstream of the video, that a prediction mode is applied to the current video block, wherein in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream; and

perform the conversion at least based on the prediction mode,

wherein a first syntax element specifying a quantized value of the escaped sample is included in the bitstream, and wherein a binarization of the first syntax element uses an exponential-Golomb (EG) code of order K, K a non-negative integer that is unequal to three,

wherein a reconstructed escaped sample is generated based on a Clip function and the quantized value of the escaped sample,

wherein the reconstructed escape sample is further determined based on qP which specifies a quantization parameter, and

wherein the qP is determined based on Max (QpPrimeTsMin, Qp′Y), where QpPrimeTsMin denotes a minimum allowed quantization parameter for a transform skip mode, and Qp′Y denotes a luma quantization parameter.

15. The apparatus of claim 14 , wherein the current video block is a luma block,

wherein a current palette comprising one or more palette predictors derived from a palette prediction table is constructed for the current video block, wherein the conversion is performed further based on the current palette,

wherein a determination that a local dual tree coding is applied to the current video block is made, and

wherein based on the determination, the palette prediction table is modified according to the current palette, wherein modifying the palette prediction table comprises a reset process.

16. The apparatus of claim 14 ,

wherein when the current video block is a chroma block, the quantization parameter is derived from a variable obtained by performing a mapping operation on a luma quantization parameter of a luma video block corresponding to the current video block.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a current video block of a video and a bitstream of the video, that a prediction mode is applied to the current video block, wherein in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream; and

perform the conversion at least based on the prediction mode,

wherein a first syntax element specifying a quantized value of the escaped sample is included in the bitstream, and wherein a binarization of the first syntax element uses an exponential-Golomb (EG) code of order K, K a non-negative integer that is unequal to three,

wherein a reconstructed escaped sample is generated based on a Clip function and the quantized value of the escaped sample,

wherein the reconstructed escape sample is further determined based on qP which specifies a quantization parameter, and

wherein the qP is determined based on Max (QpPrimeTsMin, Qp′Y), where QpPrimeTsMin denotes a minimum allowed quantization parameter for a transform skip mode, and Qp′Y denotes a luma quantization parameter.

18. A non-transitory computer-readable recording medium storing a bitstream which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining, for a current video block of a video, that a prediction mode is applied to the current video block, wherein in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values comprises at least one of 1) palette predictors, 2) escaped samples, or 3) palette information included in the bitstream; and

generating the bitstream at least based on the prediction mode,

wherein a first syntax element specifying a quantized value of the escaped sample is included in the bitstream, and wherein a binarization of the first syntax element uses an exponential-Golomb (EG) code of order K, K a non-negative integer that is unequal to three,

wherein a reconstructed escaped sample is generated based on a Clip function and the quantized value of the escaped sample,

wherein the reconstructed escape sample is further determined based on qP which specifies a quantization parameter, and

wherein the qP is determined based on Max (QpPrimeTsMin, Qp′Y), where QpPrimeTsMin denotes a minimum allowed quantization parameter for a transform skip mode, and Qp′Y denotes a luma quantization parameter.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the current video block is a luma block,

wherein a current palette comprising one or more palette predictors derived from a palette prediction table is constructed for the current video block, wherein the conversion is performed further based on the current palette,

wherein a determination that a local dual tree coding is applied to the current video block is made, and

wherein based on the determination, the palette prediction table is modified according to the current palette, wherein modifying the palette prediction table comprises a reset process.

20. The non-transitory computer-readable recording medium of claim 18 , wherein the current video block is a luma block,

wherein a current palette comprising one or more palette predictors derived from a palette prediction table is constructed for the current video block, wherein the generating is performed further based on the current palette,

wherein a determination that a local dual tree coding is applied to the current video block is made, and

wherein based on the determination, the palette prediction table is modified according to the current palette, wherein modifying the palette prediction table comprises a reset process.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2023
From: ZHU, WEIJIA; XU, JIZHENG; ZHANG, LI; CHIANG, HSIAO
To: BYTEDANCE INC.
Reel/Frame 065406/0326 →
Priority Claims (7)
WO PCT/CN2019/100850 · Aug 15, 2019 · international
WO PCT/CN2019/106700 · Sep 19, 2019 · international
WO PCT/CN2019/107494 · Sep 24, 2019 · international
WO PCT/CN2019/108736 · Sep 27, 2019 · international
WO PCT/CN2019/109793 · Oct 1, 2019 · international
WO PCT/CN2019/113931 · Oct 29, 2019 · international
WO PCT/CN2020/071221 · Jan 9, 2020 · international
Continuity (4)
Continuation 17965325 · Oct 13, 2022
Continuation 17671422 · Feb 14, 2022
Continuation PCTUS2020046581 · Aug 15, 2020
Related Publication 20240056609A1 · Feb 15, 2024
References Cited (82)
US 9426462B2 · Wang · 2016 [cited by applicant]
US 9426466B2 · Van der Auwera · 2016 [cited by applicant]
US 9900617B2 · Karczewicz · 2018 [cited by applicant]
US 10055189B2 · Tsai · 2018 [cited by applicant]
US 10148981B2 · Zhu · 2018 [cited by applicant]
US 10237575B2 · Tsai · 2019 [cited by applicant]
US 10277910B2 · Xiu · 2019 [cited by applicant]
US 10484686B2 · Xiu · 2019 [cited by applicant]
US 10659783B2 · Li · 2020 [cited by applicant]
US 10681383B2 · Ye · 2020 [cited by applicant]
US 10735764B2 · Tsai · 2020 [cited by applicant]
US 10812817B2 · Li · 2020 [cited by applicant]
US 11943481B2 · Zhu · 2024 [cited by applicant]
US 20100150242A1 · Ozawa · 2010 [cited by applicant]
US 20130051458A1 · Kitahara · 2013 [cited by applicant]
US 20130064293A1 · Song · 2013 [cited by applicant]
US 20140160139A1 · MacInnis · 2014 [cited by applicant]
US 20150078447A1 · Gamei · 2015 [cited by applicant]
US 20150098510A1 · Ye · 2015 [cited by applicant]
US 20150341643A1 · Xu · 2015 [cited by applicant]
US 20150365671A1 · Pu · 2015 [cited by applicant]
US 20160100177A1 · Pu · 2016 [cited by applicant]
US 20160100179A1 · He · 2016 [cited by applicant]
US 20160234498A1 · Misra · 2016 [cited by examiner]
US 20160286215A1 · Gamei · 2016 [cited by applicant]
US 20160373745A1 · Joshi · 2016 [cited by applicant]
US 20170026641A1 · Lai · 2017 [cited by applicant]
US 20170085891A1 · Seregin · 2017 [cited by applicant]
US 20170127077A1 · Chuang · 2017 [cited by applicant]
US 20170318301A1 · Li · 2017 [cited by applicant]
US 20170374372A1 · Liu · 2017 [cited by applicant]
US 20170374384A1 · Xiu · 2017 [cited by applicant]
US 20180288415A1 · Li · 2018 [cited by applicant]
US 20180307457A1 · Tsai · 2018 [cited by applicant]
US 20190158854A1 · He · 2019 [cited by applicant]
US 20190238845A1 · Zhang · 2019 [cited by applicant]
US 20190246122A1 · Zhang · 2019 [cited by applicant]
US 20200092546A1 · Ye · 2020 [cited by examiner]
US 20200244962A1 · Li · 2020 [cited by applicant]
US 20200267392A1 · Lu · 2020 [cited by applicant]
US 20200322630A1 · Tsai · 2020 [cited by applicant]
US 20210029358A1 · Chao · 2021 [cited by applicant]
US 20210051336A1 · Chao · 2021 [cited by examiner]
US 20210092372A1 · Misra · 2021 [cited by applicant]
US 20210297669A1 · Zhang · 2021 [cited by applicant]
US 20210321098A1 · Chuang · 2021 [cited by applicant]
US 20220182655A1 · Zhu · 2022 [cited by applicant]
US 20220210419A1 · Zhu · 2022 [cited by applicant]
CN 114424545B · 2024 [cited by applicant]
EP 3308542B1 · 2019 [cited by applicant]
IN 546343 · 2024 [cited by applicant]
JP 7494289B2 · 2024 [cited by applicant]
WO 2013158656A2 · 2013 [cited by applicant]
WO 2017206805A1 · 2017 [cited by applicant]
WO 2019026807A1 · 2019 [cited by applicant]
“http://phenix.it-sudparis.eu/jvet/doc_end_user/documents/15_Gothenburg/wg11/JVET-O2001-v14.zip”. [cited by applicant]
Bossen, F., Retrieved From the Internet: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-5.0, Jan. 9, 2023, 2 pages. [cited by applicant]
Document: JVET-O2011, Segall, A., et al., “JVET common test conditions and evaluation procedures for HDR/WCG video,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Go… [cited by applicant]
Document: JVET-N0258-v1, Zhu, W., et al., “CE8-related: Palette Mode Coding,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 6 pages. [cited by applicant]
Document: JVET-O2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 455 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” Recommendation ITU-T H.2651 ISO/IEC 23008-2, Feb. 2018, 692 pages. [cited by applicant]
Document: JVET-N0309-v3, Henkel, A., et al., “Non-CE4: Switched half-pel interpolation filter,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 201… [cited by applicant]
Document: JVET-P0077, Chao, Y-H., et al., “E8-1.3: Line-based CG Palette Mode,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 6 pages. [cited by applicant]
Document: JVET-P2001-v4, Bross, B., et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WVG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 463 pages. [cited by applicant]
Document: JVET-P2001-vE, Bross, B., et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: Geneva, CH, Oct. 1-11, 2019, 491 pages. [cited by applicant]
Document JVET-O0071-V1, Zhu, W., et al., “CE8-2.3: Compound Palette Mode,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IECJTC 1/SC 29/WG 11 15th Meeting, Gothenburg, SE Jul. 3-12, 2019, 17 pages. [cited by applicant]
Document JVET-L0308, Ye, J., et al., “CE15-Related: Palette Mode When Dual-Tree is Enabled,” Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting, Macao, CN Oct. 3-12, 2018, 3 pa… [cited by applicant]
Document: JCTVC-T0112r1, Zou, F., et al. “CE1 Related: On Escape Pixel Coding for Palette Mode,” Joint Collaborative Team on Video Coding (JCTVC)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 20th Meeting, Geneva, CH… [cited by applicant]
Document: JVET-O0398, Choi, J., et al., “Chroma Block Size Restriction in Dual Tree Intra Coding,” Joint Video Experts Team (JVET) of ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG 1115th Meeting, Gothenburg, SE Jul. 3-12, … [cited by applicant]
Xu, J., et al., “Overview of the Emerging HEVC Screen Content Coding Extension”, IEEE Transactions on Circuits and Systems for Video Technology, Sep. 14, 2015, IEEE, vol. 26, No. 1, pp. 50-62, ISSN: 1051-8215, DOI: 10.1… [cited by applicant]
Document: JVET-O0259-v1, Zhu, W., et al., “CE8-related: Palette mode with 8 bits entries,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, Jul. 3-12, 2019, 19 … [cited by applicant]
Document: JVET-O2001-vD, Bross, B., et al., “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, 474 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/US2020/046574, Intemational Search Report dated Jan. 19, 2021, 12 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/US2020/046581, Intemational Search Report dated Jan. 12, 2021, 12 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, PCT Application No. PCT/US2020/051540, Intemational Search Report dated Dec. 31, 2020, 10 pages. [cited by applicant]
Non-Final Office Action dated Jun. 16, 2022, 8 pages, U.S. Appl. No. 17/698,968, filed Mar. 18, 2022. [cited by applicant]
Non-Final Office Action dated Apr. 28, 2022, 9 pages, U.S. Appl. No. 17/671,486, filed Feb. 14, 2022. [cited by applicant]
Foreign Communication From A Related Counterpart Application, Indian Application No. 202247015459, Indian Office Action dated Aug. 22, 2022, 6 pages. [cited by applicant]
Foreign Communication From A Related Counterpart Application, European Application No. 20851940.5, Extended European Search Report dated Aug. 5, 2022, 14 pages. [cited by applicant]
Notice of Allowance dated Oct. 12, 2022, 17 pages, U.S. Appl. No. 17/698,968, filed Mar. 18, 2022. [cited by applicant]
Document: JVET-O2001-v4, Bross, B., et al., “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothernburg, SE, Jul. 3-12, 2019, 422 pages. [cited by applicant]
Korean Notice of Allowance from Korean Application No. 10-2022-7004036 dated Jun. 5, 2024, 8 pages. [cited by applicant]