IP Library Granted Patent US 12,294,739
Granted Patent B2
US 12,294,739 · App. 17/651,975 · Granted May 6, 2025

Updating for counter-based intra prediction mode

Inventors: Junru Li (Hong Kong, CN); Meng Wang (Hong Kong, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Hongbin Liu (Beijing, CN); Yue Wang (Beijing, CN); Shiqi Wang (Hong Kong, CN)
Assignees: Beijing Bytedance Network Technology Co., Ltd.; Bytedance Inc.; Bytedance (HK) Limited
H04N19/593H04N19/105H04N19/132H04N19/159H04N19/167H04N19/176H04N19/186H04N19/1883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,294,739
App. No.
17/651,975
Granted
May 6, 2025
Kind
B2
Abstract

A video processing method includes performing a conversion between a video including a video unit and a coded representation of the video, where, after the video unit is encoded or decoded with an intra prediction mode, one or more frequence tables and/or one or more sorted intra prediction mode (IPM) tables are selectively updated according to a rule, where the one or more frequence tables include information about frequence of the intra prediction mode used for processing the video unit in the conversion, where the frequence indicates an occurrence of the intra prediction mode used for the conversion, and where the one or more sorted IPM tables indicate the intra prediction mode used in the processing.

Claims (49)

1. A method of processing video data, comprising:

determining, for a conversion between a current video block of a video and a bitstream of the video, that a first prediction mode is applied to derive prediction samples of the current video block,

maintaining, a frequency table recording frequencies associated with one or more intra prediction modes, and

performing the conversion based on the first prediction mode,

wherein the frequencies indicate occurrences associated with the one or more intra prediction modes applied for one or more video blocks processed before the current video block,

wherein in a case that the first prediction mode is an intra prediction mode of the one or more intra prediction modes, the frequency table is updated based on a first intra prediction mode, and

wherein in response to the first prediction mode being not one intra prediction mode among the one or more intra prediction modes, the frequency table is disallowed from being updated.

2. The method of claim 1 , wherein in a case that the first prediction mode is a matrix-based intra prediction (MIP) mode, an intra block copy (IBC) mode, or a palette mode, the frequency table is disallowed from being updated.

3. The method of claim 1 , wherein in the case that the first prediction mode is the intra prediction mode of the one or more intra prediction modes, a frequency associated with the first prediction mode in the frequency table is updated by adding an integer K.

4. The method of claim 3 , wherein K is set to 1.

5. The method of claim 3 , wherein frequencies of remaining intra prediction modes of the one or more intra prediction modes are kept unchanged.

6. The method of claim 1 , wherein the one or more intra prediction modes exclude a wide-angular intra prediction mode.

7. The method of claim 1 , wherein the one or more intra prediction modes include at least one of a direct current (DC) mode, a horizontal mode, a vertical mode, or a bilinear intra prediction mode.

8. The method of claim 1 , wherein an intra prediction mode (IPM) table is further maintained for the conversion based on the frequency table, and the IPM table is initialized when an initialization process is invoked for the frequency table,

wherein the IPM table comprises N entries, N is an integer which is not smaller than two, and the N entries represents N modes with highest frequency in the frequency table, and

wherein the IPM table is updated when the frequency table is updated.

9. The method of claim 8 , wherein N is equal to 2.

10. The method of claim 8 , wherein a sorting process is applied to the IPM table according to the frequencies in the updated frequency table in response to that the first intra prediction mode satisfying

F n+l ( M l )>= F n+l ( M i )>= F n+l ( M 0 ),

where M l denotes the first intra prediction mode, M 0 and M l denotes two modes in the IPM table before being updated,

where F n+l (M) denotes the frequency associated with the mode M in the updated frequency table.

11. The method of claim 1 , wherein the current video block comprises a luma coding block.

12. The method of claim 1 , wherein indications of whether a frequency-based intra mode in which the frequency table is applied is enabled are included in a sequence parameter set and a picture header.

13. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.

14. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.

15. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a current video block of a video and a bitstream of the video, that a first prediction mode is applied to derive prediction samples of the current video block,

maintain, a frequency table recording frequencies associated with one or more intra prediction modes, and

perform the conversion based on the first prediction mode,

wherein the frequencies indicate occurrences associated with the one or more intra prediction modes applied for one or more video blocks processed before the current video block,

wherein in a case that the first prediction mode is an intra prediction mode of the one or more intra prediction modes, the frequency table is updated based on a first intra prediction mode, and

wherein in response to the first prediction mode being not one intra prediction mode among the one or more intra prediction modes, the frequency table is disallowed from being updated.

16. The apparatus of claim 15 , wherein in a case that the first prediction mode is a matrix-based intra prediction (MIP) mode, an intra block copy (IBC) mode, or a palette mode, the frequency table is disallowed from being updated.

17. The apparatus of claim 15 , wherein in the case that the first prediction mode is the intra prediction mode of the one or more intra prediction modes, a frequency associated with the first prediction mode in the frequency table is updated by adding an integer K.

18. The apparatus of claim 15 , wherein the one or more intra prediction modes include at least one of a direct current (DC) mode, a horizontal mode, a vertical mode, or a bilinear intra prediction mode.

19. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a current video block of a video and a bitstream of the video, that a first prediction mode is applied to derive prediction samples of the current video block,

maintain, a frequency table recording frequencies associated with one or more intra prediction modes, and

perform the conversion based on the first prediction mode,

wherein the frequencies indicate occurrences associated with the one or more intra prediction modes applied for one or more video blocks processed before the current video block,

wherein in a case that the first prediction mode is an intra prediction mode of the one or more intra prediction modes, a frequency table is updated based on the first prediction mode, and

wherein in response to the first prediction mode being not one intra prediction mode among the one or more intra prediction modes, the frequency table is disallowed from being updated.

20. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining, for a current video block of a video, that a first prediction mode is applied to derive prediction samples of the current video block,

maintaining, a frequency table recording frequencies associated with one or more intra prediction modes, and

generating the bitstream based on the first prediction mode,

wherein the frequencies indicate occurrences associated with the one or more intra prediction modes applied for one or more video blocks processed before the current video block,

wherein in a case that the first prediction mode is an intra prediction mode of the one or more intra prediction modes, a frequency table is updated based on the first prediction mode, and

wherein in response to the first prediction mode being not one intra prediction mode among the one or more intra prediction modes, the frequency table is disallowed from being updated.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2022
From: LI, JUNRU; WANG, MENG; LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 059069/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2022
From: ZHANG, LI; ZHANG, KAI
To: BYTEDANCE INC.
Reel/Frame 059069/0844 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2022
From: WANG, SHIQI
To: BYTEDANCE (HK) LIMITED
Reel/Frame 059069/0888 →
Continuity (2)
Continuation PCTCN2020109971 · Aug 19, 2020
Related Publication 20220182667A1 · Jun 9, 2022
References Cited (76)
US 9031129B2 · Ku et al. · 2015 [cited by applicant]
US 9571809B2 · Deng et al. · 2017 [cited by applicant]
US 9674533B2 · Wang · 2017 [cited by applicant]
US 10142627B2 · Zhao et al. · 2018 [cited by applicant]
US 10306240B2 · Xiu et al. · 2019 [cited by applicant]
US 10321140B2 · Zhang et al. · 2019 [cited by applicant]
US 10404999B2 · Liu et al. · 2019 [cited by applicant]
US 10582213B2 · Li et al. · 2020 [cited by applicant]
US 10750172B2 · Vanam et al. · 2020 [cited by applicant]
US 10764587B2 · Zhang et al. · 2020 [cited by applicant]
US 10812806B2 · Zhang et al. · 2020 [cited by applicant]
US 10820015B2 · Zhang et al. · 2020 [cited by applicant]
US 10841593B2 · Zhao et al. · 2020 [cited by applicant]
US 10965941B2 · Zhao et al. · 2021 [cited by applicant]
US 20090003443A1 · Guo et al. · 2009 [cited by applicant]
US 20120128064A1 · Sato · 2012 [cited by applicant]
US 20130077672A1 · Sato · 2013 [cited by applicant]
US 20130114700A1 · Moriya et al. · 2013 [cited by applicant]
US 20130202036A1 · Nagori et al. · 2013 [cited by applicant]
US 20140126629A1 · Park · 2014 [cited by examiner]
US 20140226912A1 · Lee et al. · 2014 [cited by applicant]
US 20160373742A1 · Zhao et al. · 2016 [cited by applicant]
US 20160373743A1 · Zhao et al. · 2016 [cited by applicant]
US 20160373770A1 · Zhao et al. · 2016 [cited by applicant]
US 20160373782A1 · Zhao et al. · 2016 [cited by applicant]
US 20170347103A1 · Yu et al. · 2017 [cited by applicant]
US 20180176587A1 · Panusopone · 2018 [cited by applicant]
US 20180199061A1 · Zhang et al. · 2018 [cited by applicant]
US 20180316913A1 · Jun · 2018 [cited by examiner]
US 20190045184A1 · Zhang et al. · 2019 [cited by applicant]
US 20190104303A1 · Xiu et al. · 2019 [cited by applicant]
US 20190166370A1 · Xiu et al. · 2019 [cited by applicant]
US 20190174145A1 · Zhang et al. · 2019 [cited by applicant]
US 20190238864A1 · Xiu et al. · 2019 [cited by applicant]
US 20190373285A1 · Vanam et al. · 2019 [cited by applicant]
US 20200177910A1 · Li et al. · 2020 [cited by applicant]
US 20200275124A1 · Ko · 2020 [cited by applicant]
US 20200314418A1 · Wang et al. · 2020 [cited by applicant]
US 20200314432A1 · Wang et al. · 2020 [cited by applicant]
US 20200413045A1 · Zhang et al. · 2020 [cited by applicant]
US 20210029352A1 · Zhang et al. · 2021 [cited by applicant]
US 20210051336A1 · Chao · 2021 [cited by examiner]
US 20210092362A1 · Lee · 2021 [cited by examiner]
US 20210092395A1 · Zhang et al. · 2021 [cited by applicant]
CN 102939759A · 2013 [cited by applicant]
CN 103140877A · 2013 [cited by applicant]
CN 104053007A · 2014 [cited by applicant]
CN 104378643A · 2015 [cited by applicant]
CN 104937936A · 2015 [cited by applicant]
CN 107743705A · 2018 [cited by applicant]
CN 108366256A · 2018 [cited by applicant]
CN 109891883A · 2019 [cited by applicant]
CN 110062227A · 2019 [cited by applicant]
CN 114270825B · 2024 [cited by applicant]
KR 20180075422A · 2018 [cited by applicant]
WO 2008123254A1 · 2008 [cited by applicant]
WO 2012173315A1 · 2012 [cited by applicant]
WO 2016205154A1 · 2016 [cited by applicant]
WO 2019083284A1 · 2019 [cited by applicant]
WO WO2019137732A1 · 2019 [cited by examiner]
WO 2020030002A1 · 2020 [cited by applicant]
WO 2020058893A1 · 2020 [cited by applicant]
Bross et al. “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC ITC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, document JVET-O2001, 2019. [cited by applicant]
De-Luxán-Hernández et al. “CE3: Intra Sub-Partitions Coding Mode (Tests 1.1.1 and 1.1.2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18, 2019. … [cited by applicant]
Pfaff et al. “CE3: Affine Linear Weighted Intra Prediction (CE3-4.1, CE3-4.2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019, document JVET… [cited by applicant]
Rath et al. “CE3-Related: Further Simplifications of the Unified MPM List for Intra Mode Coding,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gothenburg, SE, Jul. 3-12… [cited by applicant]
Zhang et al. “Non-CE3: History-Based Intra MPM Default Angular Modes Derivation,” Joint Video Experts Team (JVET) of ITU-T Sg 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019, document JV… [cited by applicant]
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-5.0. [cited by applicant]
International Search Report and Written Opinion from International Patent Application No. PCT/CN2020/109957 dated Nov. 25, 2020 (11 pages). [cited by applicant]
International Search Report and Written Opinion from International Patent Application No. PCT/CN2020/109961 dated Nov. 20, 2020 (9 pages). [cited by applicant]
International Search Report and Written Opinion from International Patent Application No. PCT/CN2020/109971 dated Nov. 25, 2020 (10 pages). [cited by applicant]
Xu et al. “CE4-related: CTU-level Initialization of History-based Motion Vector Prediction,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, do… [cited by applicant]
Notification to Grant Patent Right for Invention for Chinese Patent Application No. 202080058247.2 dated Apr. 10, 2024. [cited by applicant]
Chinese Office Action from Chinese Patent Application No. 202080057707.X dated Aug. 21, 2024, 15 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 17/651,956 dated Aug. 15, 2024, 17 pages. [cited by applicant]
Notification to Grant Patent Right for Invention for Chinese Application No. 202080057707.X, mailed Mar. 7, 2025, 10 pages. [cited by applicant]