IP Library › Granted Patent US 12,407,830
Granted Patent B2
US 12,407,830 · App. 18/303,993 · Granted Sep 2, 2025

Video coding using sample string vector

Inventors: Hongbin Liu (Beijing, CN); Kai Zhang (San Diego, CA); Li Zhang (San Diego, CA); Yue Wang (Beijing, CN); Junru Li (Beijing, CN); Siwei Ma (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/132H04N19/159H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,407,830
App. No.
18/303,993
Granted
Sep 2, 2025
Kind
B2
Abstract

Methods, systems and devices for video coding using sample string vectors are described. An example method of video processing includes determining, for a conversion between a video including a current block and a bitstream of the video, an availability of a block vector of one or more non-adjacent neighboring blocks of the current block for a prediction of a block vector of the current block according to a rule, and performing, based on the determining, the conversion, wherein the rule specifies a pre-defined order for checking the one or more non-adjacent neighboring blocks.

Claims (56)

1. A method of processing video data, comprising:

determining, for a conversion between a video comprising a first block and a bitstream of the video, an availability of a block vector of one or more neighboring blocks of the first block for a prediction of a block vector of the first block according to a rule, wherein the neighboring blocks are allowed to include at least one non-adjacent neighboring block; and

performing, based on the determining, the conversion,

wherein the rule specifies a pre-defined order for checking the one or more neighboring blocks for the availability of the block vector;

wherein the video further comprises a second block and the method further comprises:

determining whether and how to store a sample string vector associated with a first sample string of the second block; and

performing, based on the determining, the conversion,

wherein the determining is based on at least one of a length, a position, or a shape of the first sample string;

wherein the stored sample string vector is allowed to be inserted into a list which is an intra block copy (IBC) merge list, an IBC advanced motion vector prediction (AMVP) list, a sample string vector merge list, or a sample string vector AMVP list.

2. The method of claim 1 , wherein the availability is based on coding information for each of the one or more neighboring blocks.

3. The method of claim 2 , wherein the coding information includes whether each of the one or more neighboring blocks is coded with an intra block copy prediction mode or an intra string copy prediction mode.

4. The method of claim 1 , wherein the stored sample string vector is used to code a subsequent block.

5. The method of claim 1 , wherein the second block is coded with an intra string copy prediction mode in which the first sample string is reconstructed directly by copying a group of pixels in a current picture including the second block based on the sample string vector.

6. The method of claim 1 , wherein the conversion comprises decoding the video from the bitstream.

7. The method of claim 1 , wherein the conversion comprises encoding the video into the bitstream.

8. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a video comprising a first block and a bitstream of the video, an availability of a block vector of one or more neighboring blocks of the first block for a prediction of a block vector of the first block according to a rule, wherein the neighboring blocks are allowed to include at least one non-adjacent neighboring block; and

perform, based on the determining, the conversion,

wherein the rule specifies a pre-defined order for checking the one or more neighboring blocks for the availability of the block vector;

wherein the video further comprises a second block and the instructions further cause the processor to:

determine whether and how to store a sample string vector associated with a first sample string of the second block; and

perform, based on the determining, the conversion,

wherein the determining is based on at least one of a length, a position, or a shape of the first sample string;

wherein the stored sample string vector is allowed to be inserted into a list which is an intra block copy (IBC) merge list, an IBC advanced motion vector prediction (AMVP) list, a sample string vector merge list, or a sample string vector AMVP list.

9. The apparatus of claim 8 , wherein the availability is based on coding information for each of the one or more neighboring blocks.

10. The apparatus of claim 8 , wherein the coding information includes whether each of the one or more neighboring blocks is coded with an intra block copy prediction mode or an intra string copy prediction mode.

11. The apparatus of claim 8 , wherein the stored sample string vector is used to code a subsequent block;

wherein the second block is coded with an intra string copy prediction mode in which the first sample string is reconstructed directly by copying a group of pixels in a current picture including the second block based on the sample string vector.

12. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a video comprising a first block and a bitstream of the video, an availability of a block vector of one or more neighboring blocks of the first block for a prediction of a block vector of the first block according to a rule, wherein the neighboring blocks are allowed to include at least one non-adjacent neighboring block; and

perform, based on the determining, the conversion,

wherein the rule specifies a pre-defined order for checking the one or more neighboring blocks for the availability of the block vector;

the video further comprises a second block and the instructions further cause the processor to:

determine whether and how to store a sample string vector associated with a first sample string of the second block; and

perform, based on the determining, the conversion,

wherein the determining is based on at least one of a length, a position, or a shape of the first sample string,

wherein the stored sample string vector is allowed to be inserted into a list which is an intra block copy (IBC) merge list, an IBC advanced motion vector prediction (AMVP) list, a sample string vector merge list, or a sample string vector AMVP list.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the availability is based on coding information for each of the one or more neighboring blocks, and

wherein the coding information includes whether each of the one or more neighboring blocks is coded with an intra block copy prediction mode or an intra string copy prediction mode.

14. The non-transitory computer-readable storage medium of claim 12 ,

wherein the stored sample string vector is used to code a subsequent block, and

wherein the second block is coded with an intra string copy prediction mode in which the first sample string is reconstructed directly by copying a group of pixels in a current picture including the second block based on the sample string vector.

15. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining an availability of a block vector of one or more neighboring blocks of a first block of the video for a prediction of a block vector of the first block according to a rule, wherein the neighboring blocks are allowed to include at least one non-adjacent neighboring block; and

generating, based on the determining, the bitstream,

wherein the rule specifies a pre-defined order for checking the one or more neighboring blocks for the availability of the block vector;

wherein the video further comprises a second block and the method further comprises:

determining whether and how to store a sample string vector associated with a first sample string of the second block; and

generating, based on the determining, the bitstream,

wherein the determining is based on at least one of a length, a position, or a shape of the first sample string;

wherein the stored sample string vector is allowed to be inserted into a list which is an intra block copy (IBC) merge list, an IBC advanced motion vector prediction (AMVP) list, a sample string vector merge list, or a sample string vector AMVP list.

16. The non-transitory computer-readable recording medium of claim 15 , wherein the availability is based on coding information for each of the one or more neighboring blocks, and

wherein the coding information includes whether each of the one or more neighboring blocks is coded with an intra block copy prediction mode or an intra string copy prediction mode.

17. The non-transitory computer-readable recording medium of claim 15 ,

wherein the stored sample string vector is used to code a subsequent block, and

wherein the second block is coded with an intra string copy prediction mode in which the first sample string is reconstructed directly by copying a group of pixels in a current picture including the second block based on the sample string vector.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: ZHANG, KAI; ZHANG, LI
To: BYTEDANCE INC.
Reel/Frame 063898/0697 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: MA, SIWEI
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063898/0758 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: LIU, HONGBIN; WANG, YUE; LI, JUNRU
To: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063898/0789 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2023
From: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 063898/0820 →
Priority Claims (1)
WO PCT/CN2020/122105 · Oct 20, 2020 · international
Continuity (2)
Continuation PCTCN2021124935 · Oct 20, 2021
Related Publication 20230262225A1 · Aug 17, 2023
References Cited (57)
US 20080013623A1 · Wang · 2008 [cited by applicant]
US 20150071357A1 · Pang · 2015 [cited by applicant]
US 20150229971A1 · Ye · 2015 [cited by applicant]
US 20150373358A1 · Pang · 2015 [cited by applicant]
US 20160286241A1 · Ridge et al. · 2016 [cited by applicant]
US 20170155914A1 · Jeon · 2017 [cited by applicant]
US 20180359483A1 · Chen · 2018 [cited by applicant]
US 20190174145A1 · Zhang · 2019 [cited by applicant]
US 20190246118A1 · Ye · 2019 [cited by examiner]
US 20200045328A1 · Ye · 2020 [cited by applicant]
US 20200053379A1 · Han · 2020 [cited by applicant]
US 20200059659A1 · Chen · 2020 [cited by applicant]
US 20200077084A1 · Li · 2020 [cited by applicant]
US 20200084468A1 · Chien · 2020 [cited by applicant]
US 20200413084A1 · Xu et al. · 2020 [cited by applicant]
US 20210092432A1 · Rusanovskyy · 2021 [cited by examiner]
US 20210250606A1 · Choi · 2021 [cited by examiner]
US 20210266538A1 · Gao · 2021 [cited by applicant]
US 20210337196A1 · Lee · 2021 [cited by examiner]
US 20220417550A1 · Wang · 2022 [cited by examiner]
US 20230016630A1 · Hu · 2023 [cited by examiner]
CN 104219531A · 2014 [cited by applicant]
CN 105898330A · 2016 [cited by applicant]
CN 105917648A · 2016 [cited by applicant]
CN 106797473A · 2017 [cited by applicant]
CN 110677650A · 2020 [cited by applicant]
CN 110730348A · 2020 [cited by applicant]
CN 110855998A · 2020 [cited by applicant]
CN 111131822A · 2020 [cited by applicant]
CN 111541896A · 2020 [cited by applicant]
EP 3253061A1 · 2017 [cited by applicant]
JP 2014204356A · 2014 [cited by applicant]
TW 201526617A · 2015 [cited by applicant]
WO 2020004978A1 · 2020 [cited by applicant]
WO 2020048366A1 · 2020 [cited by applicant]
Document: JVET-R2001-vA, Bross, B., et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 18th Meeting: by teleconference, Apr. 15-24, 2020, 524 pa… [cited by applicant]
Suehring, K., Retrieved from the internet: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-9.0, Feb. 27, 2023, 2 pages. [cited by applicant]
“Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video High efficiency video coding,” ITU-T and ISO/IEC, Rec. ITU-T H.265 | ISO/IEC 23008-2, Feb. 2018, 692 pages. [cited by applicant]
Retrieved from the internet: https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/, Feb. 27, 2023, 1 page. [cited by applicant]
Document: JVET-L0399, Han, Y., et al., “CE4.4.6: Improvement on Merge/Skip mode,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, 6 pages. [cited by applicant]
Document: JVET-N0329, Xiu, X., et al., “CE8-related: Encoder improvements on IBC search,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 4 p… [cited by applicant]
Document: JVET-K0339, Han, Y., et al., “CE4.2.3: Improvement on Merge/Skip mode,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, 7 pa… [cited by applicant]
Document: JVET-L0323-v1, Gao, M., et al., “CE4.4.2: Long distance merge candidates,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macao, CN, Oct. 3-12, 2018, 4 pages. [cited by applicant]
Document: JCTVC-W1014, Joshi, R., et al., “Screen content coding test model 7 (SCM 7),” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 23rd Meeting: San Diego, USA, F… [cited by applicant]
Document: JVET-J0021, Chen, Y., et al., “Description of SDR, HDR and 360° video coding technology proposal by Qualcomm and Technicolor—low and high complexity versions,” Joint Video Exploration Team (JVET) of ITU-T SG 1… [cited by applicant]
Document: JVET-L1002-v1, Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 3 (VTM 3),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Mac… [cited by applicant]
Document: JVET-M0544-v1, Xu, X., et al., “Non-CE8: CPR with chroma 4×4 sub-block size when dual-tree is on,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakesh, MA,… [cited by applicant]
Document: JCTVC-10272, Zhang, P., et al., “4:4:4 Screen Content Coding using Dual-coder Mixed Chroma-sampling-rate (DMC) Techniques,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC… [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/098526, International Search Report dated Sep. 9, 2021, 9 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/098764, International Search Report dated Sep. 7, 2021, 11 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/113427, International Search Report dated Nov. 17, 2021, 12 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/124935, International Search Report dated Dec. 29, 2021, 10 pages. [cited by applicant]
Foreign Communication From a Related Counterpart Application, PCT Application No. PCT/CN2021/125249, International Search Report dated Jan. 18, 2022, 12 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/074,898 dated May 3, 2024, 17 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/077,783 dated May 9, 2024, 21 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/304,631 dated Sep. 19, 2024, 20 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 18/077,783 dated Nov. 8, 2024, 17 pages. [cited by applicant]