IP Library › Granted Patent US 12,244,817
Granted Patent B2
US 12,244,817 · App. 18/488,247 · Granted Mar 4, 2025

Symmetric motion vector difference coding

Inventors: Hongbin Liu (Beijing, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Zhipin Deng (Beijing, CN); Yue Wang (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/137H04N19/105H04N19/176H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,244,817
App. No.
18/488,247
Granted
Mar 4, 2025
Kind
B2
Abstract

Symmetric motion vector difference coding is described. One example video processing method includes determining, for a conversion between a block of a current picture of video and a bitstream representation of the block, whether a symmetric motion vector difference (SMVD) mode is allowable for the block. Whether the SMVD mode is allowable depends on a derivation process of two target reference pictures of the current picture, which includes a first step for searching a forward target reference picture in the reference picture list 0 and searching a backward target reference picture in the reference picture list 1, and a second step for searching a backward target reference picture in the reference picture list 0 and searching a forward target reference picture in the reference picture list 1. Outputs of the first and second steps are independent of each other. The method further includes performing the conversion based on the determination.

Claims (44)

1. A method of processing video data, comprising:

determining, for a conversion between a block of a current picture of a video and a bitstream of the video, that a first coding mode is applied to the block based on a derivation process of two target reference pictures of the current picture, wherein the derivation process includes:

performing a first step for searching a forward target reference picture in a reference picture list 0 and searching a backward target reference picture in a reference picture list 1,

in response to at least one of the two target reference pictures not being found in the first step, resetting both target reference picture indices to be unavailable before a second step is invoked, and

performing the second step for searching a backward target reference picture in the reference picture list 0 and searching a forward target reference picture in the reference picture list 1; and

performing the conversion based on the determination,

wherein in the first coding mode, a syntax structure indicating a motion vector difference of a reference picture list X is not present in the bitstream, and the motion vector difference of the reference picture list X is derived based on a motion vector difference of a reference picture list (1−X), wherein X is equal to 0 or 1.

2. The method of claim 1 , wherein resetting both the target reference picture indices to be unavailable comprises resetting both RefIdxSymL0 and RefIdxSymL1 to −1, wherein RefIdxSymL0 indicates a target reference picture index in the reference picture list 0 and RefIdxSymL1 indicates a target reference picture index in the reference picture list 1.

3. The method of claim 1 , wherein whether the first coding mode is allowable depends on whether both the target reference pictures are found in one step of the first step and the second step.

4. The method of claim 3 , wherein when both the target reference pictures are found in the first step, the first coding mode is allowed.

5. The method of claim 3 , wherein when both the target reference pictures are found in the second step, the first coding mode is allowed.

6. The method of claim 3 , wherein when at least one of the two target reference pictures are not found in the first step and at least one of the two target reference pictures are not found in the second step, the first coding mode is disallowed.

7. The method of claim 1 , wherein a picture order of the forward target reference picture is smaller than a picture order of the current picture and a picture order of the backward target reference picture is larger than the picture order of the current picture.

8. The method of claim 1 , wherein the conversion includes decoding the block of video from the bitstream.

9. The method of claim 1 , wherein the conversion includes encoding the block of video into the bitstream.

10. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a block of a current picture of a video and a bitstream of the video, that a first coding mode is applied to the block based on a derivation process of two target reference pictures of the current picture, wherein the derivation process includes:

performing a first step for searching a forward target reference picture in a reference picture list 0 and searching a backward target reference picture in a reference picture list 1,

in response to at least one of the two target reference pictures not being found in the first step, resetting both target reference picture indices to be unavailable before a second step is invoked, and

performing the second step for searching a backward target reference picture in the reference picture list 0 and searching a forward target reference picture in the reference picture list 1; and

perform the conversion based on the determination,

wherein in the first coding mode, a syntax structure indicating a motion vector difference of a reference picture list X is not present in the bitstream, and the motion vector difference of the reference picture list X is derived based on a motion vector difference of a reference picture list (1−X), wherein X is equal to 0 or 1.

11. The apparatus of claim 10 , wherein resetting both the target reference picture indices to be unavailable comprises resetting both RefIdxSymL0 and RefIdxSymL1 to −1, wherein RefIdxSymL0 indicates a target reference picture index in the reference picture list 0 and RefIdxSymL1 indicates a target reference picture index in the reference picture list 1.

12. The apparatus of claim 10 , wherein whether the first coding mode is allowable depends on whether both the target reference pictures are found in one step of the first step and the second step.

13. The apparatus of claim 12 , wherein when both the target reference pictures are found in the first step, the first coding mode is allowed.

14. The apparatus of claim 12 , wherein when both the target reference pictures are found in the second step, the first coding mode is allowed.

15. The apparatus of claim 12 , wherein when at least one of the two target reference pictures are not found in the first step and at least one of the two target reference pictures are not found in the second step, the first coding mode is disallowed.

16. The apparatus of claim 10 , wherein a picture order of the forward target reference picture is smaller than a picture order of the current picture and a picture order of the backward target reference picture is larger than the picture order of the current picture.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a block of a current picture of a video and a bitstream of the video, that a first coding mode is applied to the block based on a derivation process of two target reference pictures of the current picture, wherein the derivation process includes:

performing a first step for searching a forward target reference picture in a reference picture list 0 and searching a backward target reference picture in a reference picture list 1,

in response to at least one of the two target reference pictures not being found in the first step, resetting both target reference picture indices to be unavailable before a second step is invoked, and

performing the second step for searching a backward target reference picture in the reference picture list 0 and searching a forward target reference picture in the reference picture list 1; and

perform the conversion based on the determination,

wherein in the first coding mode, a syntax structure indicating a motion vector difference of a reference picture list X is not present in the bitstream, and the motion vector difference of the reference picture list X is derived based on a motion vector difference of a reference picture list (1−X), wherein X is equal to 0 or 1.

18. The non-transitory computer-readable storage medium of claim 17 , wherein resetting both the target reference picture indices to be unavailable comprises resetting both RefIdxSymL0 and RefIdxSymL1 to −1, wherein RefIdxSymL0 indicates a target reference picture index in the reference picture list 0 and RefIdxSymL1 indicates a target reference picture index in the reference picture list 1.

19. The non-transitory computer-readable storage medium of claim 17 , wherein a picture order of the forward target reference picture is smaller than a picture order of the current picture and a picture order of the backward target reference picture is larger than the picture order of the current picture.

20. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining, for a conversion between a block of a current picture of the video and the bitstream of the video, that a first coding mode is applied to the block based on a derivation process of two target reference pictures of the current picture, wherein the derivation process includes:

performing a first step for searching a forward target reference picture in a reference picture list 0 and searching a backward target reference picture in a reference picture list 1,

in response to at least one of the two target reference pictures not being found in the first step, resetting both target reference picture indices to be unavailable before a second step is invoked, and

performing the second step for searching a backward target reference picture in the reference picture list 0 and searching a forward target reference picture in the reference picture list 1; and

performing the conversion based on the determination,

wherein in the first coding mode, a syntax structure indicating a motion vector difference of a reference picture list X is not present in the bitstream, and the motion vector difference of the reference picture list X is derived based on a motion vector difference of a reference picture list (1−X), wherein X is equal to 0 or 1.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2023
From: ZHANG, LI; ZHANG, KAI
To: BYTEDANCE INC.
Reel/Frame 065408/0271 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2023
From: LIU, HONGBIN; DENG, ZHIPIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 065408/0324 →
Priority Claims (3)
WO PCT/CN2019/084768 · Apr 28, 2019 · international
WO PCT/CN2019/087201 · May 16, 2019 · international
WO PCT/CN2019/094954 · Jul 5, 2019 · international
Continuity (4)
Continuation 17729404 · Apr 26, 2022
Continuation 17411204 · Aug 25, 2021
Continuation PCTCN2020087551 · Apr 28, 2020
Related Publication 20240056584A1 · Feb 15, 2024
References Cited (73)
US 10448010B2 · Chen et al. · 2019 [cited by applicant]
US 10462462B2 · Chien et al. · 2019 [cited by applicant]
US 10491917B2 · Chen et al. · 2019 [cited by applicant]
US 10595035B2 · Karczewicz et al. · 2020 [cited by applicant]
US 10701366B2 · Chen et al. · 2020 [cited by applicant]
US 11463703B2 · Liu et al. · 2022 [cited by applicant]
US 20070092008A1 · Chang et al. · 2007 [cited by applicant]
US 20090207915A1 · Yan et al. · 2009 [cited by applicant]
US 20110293010A1 · Jeong et al. · 2011 [cited by applicant]
US 20140016829A1 · Chen · 2014 [cited by applicant]
US 20140185682A1 · Chen et al. · 2014 [cited by applicant]
US 20150023418A1 · Shimada et al. · 2015 [cited by applicant]
US 20160337662A1 · Pang et al. · 2016 [cited by applicant]
US 20180091816A1 · Chien et al. · 2018 [cited by applicant]
US 20180098089A1 · Chen et al. · 2018 [cited by applicant]
US 20180199057A1 · Chuang et al. · 2018 [cited by applicant]
US 20180278949A1 · Karczewicz · 2018 [cited by examiner]
US 20190222848A1 · Chen et al. · 2019 [cited by applicant]
US 20200045310A1 · Chen et al. · 2020 [cited by applicant]
US 20200154124A1 · Lee · 2020 [cited by examiner]
US 20200213594A1 · Liu et al. · 2020 [cited by applicant]
US 20200213612A1 · Liu et al. · 2020 [cited by applicant]
US 20200221117A1 · Liu et al. · 2020 [cited by applicant]
US 20200244979A1 · Li · 2020 [cited by examiner]
US 20200359029A1 · Liu et al. · 2020 [cited by applicant]
US 20210029362A1 · Liu et al. · 2021 [cited by applicant]
US 20210051339A1 · Liu et al. · 2021 [cited by applicant]
US 20210076063A1 · Liu et al. · 2021 [cited by applicant]
US 20210092435A1 · Liu et al. · 2021 [cited by applicant]
US 20210185324A1 · Chen · 2021 [cited by examiner]
US 20210385461A1 · Liu et al. · 2021 [cited by applicant]
US 20220038732A1 · Jang · 2022 [cited by examiner]
CN 104053001A · 2014 [cited by applicant]
CN 104904217A · 2015 [cited by applicant]
CN 105917650A · 2016 [cited by applicant]
CN 107409225A · 2017 [cited by applicant]
CN 107534766A · 2018 [cited by applicant]
CN 107925760A · 2018 [cited by applicant]
CN 108293131A · 2018 [cited by applicant]
CN 109417630A · 2019 [cited by applicant]
CN 109479143A · 2019 [cited by applicant]
JP 7303330B2 · 2023 [cited by applicant]
TW 201832557A · 2018 [cited by applicant]
WO 2018175720A1 · 2018 [cited by applicant]
WO 2019064640A1 · 2019 [cited by applicant]
WO WO2020132272A1 · 2020 [cited by examiner]
Algorithm Description for Versatile Video Coding and Test Model 5 (VTM5)—Jianle Chen, Yan Ye, Seung Hwan Kim; Joint Video Expert Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG 11; 14th Meeting: Geneva, CH, Ma… [cited by examiner]
“New Bi-Prediction Techniques for B Pictures Coding”—Xianyang Ji et al., 2004 IEEE International Conference on Multimedia and Expo (ICME) (Year: 2004). [cited by examiner]
Bross, B., et al., “Versatile Video Coding (Draft 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/ IEC JTC1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, JVET-N1001-v2, 361 pages. [cited by applicant]
vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-4.1, Jan. 9, 2024, 1 page. [cited by applicant]
Sethuraman, S., “CE9: Results of DMVR related Tests CE9.2.1 and CE9.2.2,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, JVET-M0147-v7, 12… [cited by applicant]
“Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, High efficiency video coding,” ITU-T, Telecommunication Standardization Sector of ITU, H.265, Feb. 2018, 692 … [cited by applicant]
Rosewarne, C., et al., “High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Improved Encoder Description Update 7,” Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 2… [cited by applicant]
Chen, J. et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7),” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 7th Meeting: Torino, IT, Jul. 13-21, 2017, JVET-G… [cited by applicant]
JEM-7.0: https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/ HM-16.6-JEM-7.0, Jan. 9, 2024, 1 page. [cited by applicant]
Luo, J., et al., “CE4-related: Simplified symmetric MVD based on CE4.4.3,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, JVET-M0444-v2, … [cited by applicant]
Bross, B., et al., “Versatile Video Coding (Draft 4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/ IEC JTC1/SC 29/WG 11, 13th Meeting: Marrakech, MA, JVET-M1001-v7, Jan. 9-18, 2019, 300 pages. [cited by applicant]
Huanbang, C., et al., “CE4: Symmetrical MVD mode (Test 4.5.1),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, JVET-L0370-v1, Oct. 3-12, 2018, 4 pages. [cited by applicant]
Luo, J., et al., “CE2-related: Symmetric MVD Signaling,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, JVET-N0235-r1, Mar. 19-27, 2019, 2 pages. [cited by applicant]
Bross, B., et al., “Versatile Video Coding (Draft 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/ IEC JTC 1/SC 29/WVG 11, 14th Meeting: Geneva, CH, JVET-N1001-v10, Mar. 19-27, 2019, 407 pages. [cited by applicant]
Lee, H., et al., “Non-CE4: Simplification of decoding process for SMVD references indices,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, JVET-N0472-v3, Mar… [cited by applicant]
Lee, J., et al., “CE3: Simplification of MPM derivation (CE3-3.2),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, Ch, JVET-N0134, Mar. 19-27, 2019, 4 pages. [cited by applicant]
Chen, J., et al., “Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, JVET-N1002-v1, … [cited by applicant]
Lee, H., et al., “Non-CE4: Simplification of decoding process for SMVD reference indices,” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, 6… [cited by applicant]
International Search Report and Written Opinion from International Patent Application No. PCT/CN2020/087551 dated Jul. 7, 2020 (10 pages). [cited by applicant]
International Search Report and Written Opinion from International Patent Application No. PCT/CN2020/087535 dated Jul. 20, 2020 (10 pages). [cited by applicant]
Notice of Allowance from U.S. Appl. No. 17/729,404 dated Jun. 16, 2023. [cited by applicant]
Non Final Office Action from U.S. Appl. No. 17/729,404 dated Dec. 22, 2022. [cited by applicant]
Extended European Search Report from European Patent Application No. 20798918.7 mailed Jun. 1, 2022 (1 page). [cited by applicant]
Japanese Notification of Reasons for Refusal from Japanese Patent Application No. 2023-100212 dated May 21, 2024, 19 pages. [cited by applicant]
Document: JVET-M1001-v6, Bross, B., et al., “Versatile Video Coding (Draft 4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, JVET-N0413_… [cited by applicant]
Jang, H., et al., “AhG2: Mismatch Between Text Specification and Reference Software on SMVD,” Document: JVET-N0470, [online], JVET-N0470 (version 3), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1… [cited by applicant]
Notice of Allowance for Japanese Patent Application No. 2023-100212 dated Oct. 22, 2024, 8 pages. [cited by applicant]
Cited By (1)
US 12,452,431