IP Library Granted Patent US 12,519,967
Granted Patent B2
US 12,519,967 · App. 18/152,525 · Granted Jan 6, 2026

Spatial displacement vector prediction for intra picture block and string copying

Inventors: Xiaozhong Xu (State College, PA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/50H04N19/105H04N19/159H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,519,967
App. No.
18/152,525
Granted
Jan 6, 2026
Kind
B2
Abstract

A method, computer program, and computer system is provided for coding video data. Video data including one or more blocks is received. A current block coded in intra block copy mode or string matching mode is predicted from among the one or more blocks based on a coded block vector or a string offset vector corresponding to one or more spatial neighboring or non-neighboring blocks from among the one or more blocks. The video data is decoded based on the predicted current block.

Claims (47)

1 . A method of video coding, executable by a processor, comprising:

receiving video bitstream comprising one or more blocks;

obtaining a displacement vector of a current block coded in an intra block copy (IBC) mode based on string offset vectors associated with one or more spatial non-neighboring blocks of the one or more blocks;

obtaining a string offset vector of the current block coded in a string matching mode based on block vectors associated with the one or more spatial non-neighboring blocks of the one or more blocks, wherein spatial candidates for obtaining the string offset vector are based on a history-based string offset vector candidates in a string offset vector predictor list, and a spatial neighboring block or a spatial non-neighboring block, location information, and size information are added to the string offset vector predictor list based on a class-based prediction being used, the class-based prediction comprising classification of a respective block vector associated with a respective history-based string offset vector candidate into at least one of:

a first class when an area of a coded block is greater than or equal to a threshold;

a second class when a frequency of a block vector is greater than or equal to 2;

a third class when the coded clock is located left of the current block;

a fourth class when the coded clock is located above of the current block;

a fifth class when the coded clock is located towards a top right of the current block; or

a sixth class when the coded clock is located towards a bottom left of the current block; and

decoding the video data based on the displacement vector of the current block or the string offset vector of the current block.

2 . The method of claim 1 , wherein only the one or more spatial non-neighboring blocks coded in the IBC mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

3 . The method of claim 1 , wherein the method further comprises:

one or more spatial neighboring blocks and the one or more spatial non-neighboring blocks being placed at a front of a string vector predictor list.

4 . The method of claim 1 , wherein the method further comprises:

one or more spatial neighboring blocks and the one or more spatial non-neighboring blocks being placed at an end of a string vector predictor list.

5 . The method of claim 1 , wherein only the one or more spatial non-neighboring blocks coded in the string matching mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

6 . The method of claim 1 , wherein only the one or more spatial neighboring blocks coded in the IBC mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

7 . A computer system for video coding, the computer system comprising:

one or more computer-readable non-transitory storage media configured to store computer program code; and

one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:

first predicting code configured to cause the one or more computer processors to determine a displacement vector of a current block coded in an intra block copy (IBC) mode based on string offset vectors associated with one or more spatial non-neighboring blocks oft one or more blocks,

second predicting code configured to cause the one or more computer processors to determine a string offset vector of the current block coded in a string matching mode based on block vectors associated with the one or more spatial non-neighboring blocks of the one or more blocks, wherein spatial candidates for obtaining the string offset vector are based on a history-based string offset vector candidates in a string offset vector predictor list, and a spatial neighboring block or a spatial non-neighboring block, location information, and size information are added to the string offset vector predictor list based on a class-based prediction being used, the class-based prediction comprising classification of a respective block vector associated with a respective history-based string offset vector candidate into at least one of:

a first class when an area of a coded block is greater than or equal to a threshold;

a second class when a frequency of a block vector is greater than or equal to 2;

a third class when the coded clock is located left of the current block;

a fourth class when the coded clock is located above of the current block;

a fifth class when the coded clock is located towards a top right of the current block; or

a sixth class when the coded clock is located towards a bottom left of the current block; and

encoding code configured to cause the one or more computer processors to encode the video data based on the displacement vector of the current block or the string offset vector of the current block.

8 . The computer system of claim 7 , wherein only the one or more spatial non-neighboring blocks coded in the IBC mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

9 . The computer system of claim 7 , wherein one or more spatial neighboring blocks and the one or more spatial non-neighboring blocks are placed at a front of a string vector predictor list.

10 . The computer system of claim 7 , wherein one or more spatial neighboring blocks and the one or more spatial non-neighboring blocks are placed at an end of a string vector predictor list.

11 . The computer system of claim 7 , wherein only the one or more spatial non-neighboring blocks coded in the string matching mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

12 . The computer system of claim 7 , wherein the one or more spatial neighboring blocks coded in the IBC mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

13 . A non-transitory computer readable medium having stored thereon a video bitstream that is generated by a video encoding method, the video encoding method comprising:

determining a displacement vector of a current block coded in an intra block copy (IBC) mode based on string offset vectors associated with one or more spatial non-neighboring blocks of one or more blocks;

determining a string offset vector of the current block coded in a string matching mode based on block vectors associated with the one or more spatial non-neighboring blocks of the one or more blocks, wherein spatial candidates for obtaining the string offset vector are based on a history-based string offset vector candidates in a string offset vector predictor list, and a spatial neighboring block or a spatial non-neighboring block, location information, and size information are added to the string offset vector predictor list based on a class-based prediction being used, the class-based prediction comprising classification of a respective block vector associated with a respective history-based string offset vector candidate into at least one of:

a first class when an area of a coded block is greater than or equal to a threshold;

a second class when a frequency of a block vector is greater than or equal to 2;

a third class when the coded clock is located left of the current block;

a fourth class when the coded clock is located above of the current block;

a fifth class when the coded clock is located towards a top right of the current block; or

a sixth class when the coded clock is located towards a bottom left of the current block; and

encoding the video bitstream comprising the current block based on the string offset vector.

14 . The non-transitory computer readable medium of claim 13 , wherein only the one or more spatial non-neighboring blocks coded in the IBC mode are considered as candidates to predict the string offset vector of the current block coded in the string matching mode.

15 . The non-transitory computer readable medium of claim 13 , wherein the bitstream further comprises the one or more spatial neighboring blocks and the one or more spatial non-neighboring blocks being placed at a front of or at an end of a string vector predictor list.

Continuity (3)
Continuation 17240567 · Apr 26, 2021
Provisional Application 63038020 · Jun 11, 2020
Related Publication 20230156213A1 · May 18, 2023
References Cited (48)
US 10516882B2 · He · 2019 [cited by applicant]
US 20110194609A1 · Rusert et al. · 2011 [cited by applicant]
US 20150264348A1 · Zou · 2015 [cited by examiner]
US 20150373366A1 · He · 2015 [cited by applicant]
US 20170280159A1 · Xu · 2017 [cited by examiner]
US 20170302935A1 · Li · 2017 [cited by examiner]
US 20190182490A1 · Lee · 2019 [cited by applicant]
US 20190200038A1 · He · 2019 [cited by applicant]
US 20200195960A1 · Zhang · 2020 [cited by examiner]
CN 107079161A · 2017 [cited by applicant]
CN 113709458A · 2021 [cited by applicant]
EP 4030761A1 · 2022 [cited by applicant]
WO 2015052273A1 · 2015 [cited by applicant]
WO WO2016034058A1 · 2016 [cited by examiner]
WO WO2020058893A1 · 2020 [cited by examiner]
WO 2022086600A1 · 2022 [cited by applicant]
Min Gao et at., (hereinafter Gao) “CE4.4.2: Long distance merge candidates” JVET-L0323-v1, Macao, CN Oct. 2018 (Year: 2018). [cited by examiner]
Shan Liu et al., (hereinafter Liu) “Overview of HEVC extensions on screen content coding”, 2015 (Year: 2015). [cited by examiner]
Chun-Chi Chen et al., “Intra Line Copy for HEVC Screen Content Intra-Picture Prediction”, 1051-8215 © 2016 IEEE, vol. 27, No. Jul. 7, 2017 (Year: 2017). [cited by examiner]
Benjamin Bross, et al., “General Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression with Capability beyond HEVC”, IEEE Transactions on Circuits and Systems for Video Technology, 20… [cited by applicant]
Benjamin Bross, et al., “Versatile Video Coding (Draft 2)”, Joint Video Experts Team (JVET), of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-K1001, 11th Meeting, Jul. 10-18, 2018, 141 pages, Ljubljana, SI. [cited by applicant]
Chun-Chi Chen, et al., “Screen Content Coding Using Non-Square Intra Block Copy for HEVC”, 6 pages, Department of Computer Science, National Chiao Tung University, Hsinchu, Taiwan, Media Tek USA Inc., San Jose, CA, U.S.… [cited by applicant]
Dong Liu, et al., “Deep Learning-Based Technology in Responses to the Joint Call for Proposals on Video Compression with Capability beyond HEVC”, IEEE Transactions on Circuits and Systems for Video Technology, 2019, IEE… [cited by applicant]
Han Zhu, et al., “Residual Convolutional Neural Network Based In-Loop Filter With Intra and Inter Frames Processed Respectively for AVS3”, 6 pages, Tencent Media Lab, Shenzhen, China, Tencent Media Lab, Palo Alto, CA, U… [cited by applicant]
International Search Report dated Aug. 9, 2021 in Application No. PCT/US21/29648. [cited by applicant]
Rajan Joshi, et al., “High Efficiency Video Coding (HEVC) Screen Content Coding: Draft 4”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 21st Meeting, ISO/IEC 23008… [cited by applicant]
Rajan Joshi, et al., “Screen content coding test model 1 (SCM 1)”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 17th Meeting, JCTVC-Q1014, Mar. 27-Apr. 4, 2014, pp… [cited by applicant]
Shan Liu, et al., “Bit-depth Scalable Coding for High Dynamic Range Video”, Visual Communications and Image Processing, Proc. of SPIE-IS&T Electronic Imaging, 2008, 10 pages, vol. 6822. [cited by applicant]
Shan Liu, et al., “Hybrid global-local motion compensated frame interpolation for low bit rate video coding”, Journal of Visual Communication & Image Representation, 2003, pp. 61-79, vol. 14. [cited by applicant]
Shan Liu, et al., “Joint Temporal-Spatial Bit Allocation for Video Coding With Dependency”, IEEE Transactions on Circuits and Systems for Video Technology, Jan. 2005, 13 pages, vol. 15, No. 1. [cited by applicant]
Shan Liu, et al., “Non-linear Motion-compensated Interpolation for Low Bit Rate Video”, Proceedings of SPIE, Applications of Digital Image Processing XX111, Dec. 28, 2000, pp. 203-213, vol. 4115. [cited by applicant]
Shan Liu, et al., “Overview of HEVC extensions on screen content coding”, Industrial Technology Advances, Overview of HEVC Extensions, SIP, 2015, pp. 1-12, vol. 4, No. e10. [cited by applicant]
Written Opinion of the International Searching Authority dated Aug. 9, 2021 in Application No. PCT/US21/29648. [cited by applicant]
Xiaozhong Xu, et al., “Block Vector Prediction in Intra Block Copy for HEVC Screen Content Coding”, Data Compression Conference, IEEE, 2015, pp. 273-282. [cited by applicant]
Xiaozhong Xu, et al., “Improvements on Fast Motion Estimation Strategy for H.264/AVC”, IEEE Transactions on Circuits and Systems for Video Technology, Mar. 2008, pp. 285-293, vol. 18, No. 3. [cited by applicant]
Xiaozhong Xu, et al., “Intra Block Copy in HEVC Screen Content Coding Extensions”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2016, IEEE, pp. 1-11. [cited by applicant]
Xiaozhong Xu, et al., “Intra block copy in Versatile Video Coding with Reference Sample Memory Reuse”, IEEE, 5 pages. [cited by applicant]
Ximin Zhang, et al., “Intra Mode Coding in HEVC Standard”, Media Tek USA Inc, 6 pages. [cited by applicant]
Yingbin Wang, et al., “Intra Block Copy in AVS3 Video Coding Standard”, IEEE, 2020, pp. 1-6. [cited by applicant]
Liping Zhao, et al., “Mapping of Intra String Copy Parameters for HEVC Screen Content Coding,” 2014 APSIPA, 10 pages. [cited by applicant]
Extended European Search Report dated Nov. 2, 2022 in Application No. 21822937.5. [cited by applicant]
Xiaozhong Xu et al., “CE8-related: CPR mode with merge mode improvements”, Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting, Macao, CN, Oct. 3-12, 2018, pp. 1-7 (7 total pages). [cited by applicant]
Min Gao et al., “CE4.4.2: Long distance merge candidates”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 12th Meeting, Macao, CN, Oct. 3-12, 2018, pp. 1-4 (4 total pages). [cited by applicant]
Pang et al., “Non-CE2: Block vector prediction method for intra block copy”, JCT-VC Meeting; Feb. 11, 2015, (5 total pages), retrieved from the Internet: URL:http://phenix.int-evry.fr/jct/doc_end_user/documents/20_Genev… [cited by applicant]
Zhao Liping et al., “A Universal String Matching Approach to Screen Content Coding”, IEEE Transactions on Multimedia, Apr. 2018, vol. 20, No. 4, pp. 796-809 (14 total pages). [cited by applicant]
Xiaozhong Xu et al., “Overview of Screen Content Coding in Recently Developed Video Coding Standards”, arxiv.org, Nov. 28, 2020, (11 total pages). [cited by applicant]
Office Action issued Dec. 27, 2023 in Chinese Application No. 202180005509.3. [cited by applicant]
Ru-Ling Liao, et al., “CE3: Results of Test A on Intra Line Copy”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11, JCTVC-T0105, Feb. 10-18, 2015, 20th Meeting, Geneva,… [cited by applicant]