IP Library › Granted Patent US 12,309,414
Granted Patent B2
US 12,309,414 · App. 18/167,355 · Granted May 20, 2025

Pattern-based motion vector derivation for video coding

Inventors: Tzu-Der Chuang (Zhubei, TW); Ching-Yeh Chen (Taipei, TW); Chih-Wei Hsu (Hsinchu, TW); Yu-Wen Huang (Taipei, TW)
Assignee: HFI Innovation Inc.
H04N19/513H04N19/176H04N19/44H04N19/573
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,309,414
App. No.
18/167,355
Granted
May 20, 2025
Kind
B2
Abstract

The techniques described herein relate to methods, apparatus, and computer readable media configured to receive compressed video data, wherein the compressed video data is related to a set of frames. A decoder-side predictor refinement technique is used to calculate a new motion vector for a current frame from the set of frames, wherein the new motion vector estimates motion for the current frame based on one or more reference frames. An existing motion vector associated with a different frame from a motion vector buffer is retrieved. The new motion vector is calculated based on the existing motion vector using a decoder-side motion vector prediction technique, such that the existing motion vector is in the motion vector buffer after calculating the new motion vector.

Claims (74)

1. A decoding method for decoding video data, the method comprising:

receiving compressed video data related to a set of frames;

deriving a first motion vector for a current block, wherein the first motion vector is an unrefined motion vector;

calculating, using a decoder-side predictor refinement technique according to the first motion vector, a second motion vector, wherein the second motion vector is a refined motion vector for the current block;

performing motion compensation for the current block according to the second motion vector that is the refined motion vector, but not the first motion vector that is the unrefined motion vector for the current block; and

deriving a motion vector predictor for a second block different than the current block by using the first motion vector or the second motion vector of the current block, wherein:

when the second block and the current block are in a same region, the first motion vector of the current block is used; and

when the second block and the current block are not in the same region, the second motion vector of the current block is used.

2. The decoding method of claim 1 further comprising:

determining data from one or more upper coding tree unit (CTU) rows are available wherein the second block and the current block are considered to be in the same region if the second block and the current block are in a same row of the one or more upper coding tree unit (CTU) rows.

3. The decoding method of claim 1 , wherein:

calculating comprises:

storing the second motion vector in a second motion vector buffer so that the first motion vector remains in a first motion vector buffer.

4. The decoding method of claim 1 , comprising:

deriving the unrefined motion vector comprising retrieving motion vectors for a set of neighboring blocks, comprising, for one or more neighboring blocks from the set of neighboring blocks, retrieving one or more first motion vectors associated with the neighboring block.

5. The decoding method of claim 1 , comprising:

determining data from one or more upper coding tree unit (CTU) rows are available; and

deriving the unrefined motion vector comprising retrieving motion vectors for a set of neighboring blocks, comprising, for one or more neighboring blocks from the set of neighboring blocks:

determining the neighboring block is in the one or more upper CTU rows; and

retrieving one or more second motion vectors associated with the neighboring block.

6. The decoding method of claim 1 , wherein calculating comprises:

testing one or more motion vectors pairs derived from a first set of reference frames and a second set of reference frames; and

storing only one of the one or more motion vector pairs associated with the first set of reference frames and the second set of reference frames.

7. The decoding method of claim 1 , wherein calculating comprises:

testing one or more vectors pairs derived from a first set of reference frames, or a second set of reference frames, but not both.

8. The decoding method of claim 1 , further comprising:

receiving a signal indicative of a starting candidate index for a starting motion vector candidate list; and using the received signal to derive the first motion vector.

9. The decoding method of claim 1 , further comprising:

receiving a signal indicative of a starting candidate index for a starting motion vector candidate list, a coarse grain motion vector difference, or both; and

using the received signal to derive the first motion vector.

10. The decoding method of claim 1 , further comprising:

receiving a signal from an advanced motion vector prediction indicative of a starting motion vector; and

using the received signal to derive the first motion vector.

11. An apparatus configured to decode video data, the apparatus comprising a processor in communication with memory, the processor being configured to execute instructions stored in the memory that cause the processor to:

receive compressed video data related to a set of frames;

derive a first motion vector for a current block, wherein the first motion vector is an unrefined motion vector;

calculate, using a decoder-side predictor refinement technique according to the first motion vector, a second motion vector, wherein the second motion vector is a refined motion vector for the current block;

perform motion compensation for the current block according to the second motion vector that is the refined motion vector, but not the first motion vector that is the unrefined motion vector for the current block; and

derive a motion vector predictor for a second block different than the current block by using the first motion vector or the second motion vector of the current block, wherein:

when the second block and the current block are in a same region, the first motion vector of the current block is used; and

when the second block and the current block are not in the same region, the second motion vector of the current block is used.

12. The apparatus of claim 11 , wherein the instructions further cause the processor to:

determine data from one or more upper coding tree unit (CTU) rows are available wherein the second block and the current block are considered to be in the same region if the second block and the current block are in a same row of the one or more upper coding tree unit (CTU) rows.

13. The apparatus of claim 11 , wherein:

calculating comprises:

storing the second motion vector in a second motion vector buffer so that the first motion vector remains in a first motion vector buffer.

14. The apparatus of claim 11 , wherein the instructions further cause the processor to derive the unrefined motion vector comprising retrieving motion vectors for a set of neighboring blocks, comprising, for one or more neighboring blocks from the set of neighboring blocks, retrieving one or more first motion vectors associated with the neighboring block.

15. The apparatus of claim 11 , wherein the instructions further cause the processor to:

determine data from one or more upper coding tree unit (CTU) rows are available; and

deriving the unrefined motion vector comprising retrieving motion vectors for a set of neighboring blocks, comprising, for one or more neighboring blocks from the set of neighboring blocks:

determining the neighboring block is in the one or more upper CTU rows; and

retrieving one or more second motion vectors associated with the neighboring block.

16. The apparatus of claim 11 , wherein calculating comprises:

testing one or more motion vectors pairs derived from a first set of reference frames and a second set of reference frames; and

storing only one of the one or more motion vector pairs associated with the first set of reference frames and the second set of reference frames.

17. The apparatus of claim 11 , wherein calculating comprises:

testing one or more vectors pairs derived from a first set of reference frames, or a second set of reference frames, but not both.

18. The apparatus of claim 11 , wherein the instructions further cause the processor to:

receive a signal indicative of a starting candidate index for a starting motion vector candidate list; and

use the received signal to derive the first motion vector.

19. The apparatus of claim 11 , wherein the instructions further cause the processor to:

receive a signal indicative of a starting candidate index for a starting motion vector candidate list, a coarse grain motion vector difference, or both; and

use the received signal to derive the first motion vector.

20. The apparatus of claim 11 , wherein the instructions further cause the processor to:

receive a signal from an advanced motion vector prediction indicative of a starting motion vector; and

use the received signal to derive the first motion vector.

21. At least one non-transitory computer-readable storage medium encoded with a plurality of computer-executable instructions that, when executed, perform a method comprising:

receiving compressed video data related to a set of frames;

deriving a first motion vector for a current block, wherein the first motion vector is an unrefined motion vector;

calculating, using a decoder-side predictor refinement technique according to the first motion vector, a second motion vector, wherein the second motion vector is a refined motion vector for the current block;

performing motion compensation for the current block according to the second motion vector that is the refined motion vector, but not the first motion vector that is the unrefined motion vector for the current block; and

deriving a motion vector predictor for a second block different than the current block by using the first motion vector or the second motion vector of the current block, wherein:

when the second block and the current block are in a same region, the first motion vector of the current block is used; and

when the second block and the current block are not in the same region, the second motion vector of the current block is used.

Continuity (3)
Continuation 15670109 · Aug 7, 2017
Provisional Application 62371879 · Aug 8, 2016
Related Publication 20230188745A1 · Jun 15, 2023
References Cited (52)
US 9635386B2 · Hendry et al. · 2017 [cited by applicant]
US 11638027B2 · Chuang et al. · 2023 [cited by applicant]
US 20030152149A1 · Denolf · 2003 [cited by examiner]
US 20060088102A1 · Lee et al. · 2006 [cited by applicant]
US 20060256866A1 · Ziauddin · 2006 [cited by examiner]
US 20080043842A1 · Nakaishi · 2008 [cited by applicant]
US 20080212676A1 · Liu et al. · 2008 [cited by applicant]
US 20090238276A1 · Har-Noy et al. · 2009 [cited by applicant]
US 20100284464A1 · Nagori et al. · 2010 [cited by applicant]
US 20110080954A1 · Bossen · 2011 [cited by examiner]
US 20110176611A1 · Huang et al. · 2011 [cited by applicant]
US 20120044998A1 · Kokaram · 2012 [cited by applicant]
US 20120075535A1 · Van Beek · 2012 [cited by applicant]
US 20120294370A1 · Chiu et al. · 2012 [cited by applicant]
US 20120320969A1 · Zheng et al. · 2012 [cited by applicant]
US 20130022119A1 · Chien et al. · 2013 [cited by applicant]
US 20130195188A1 · Sugio et al. · 2013 [cited by applicant]
US 20130243093A1 · Chen et al. · 2013 [cited by applicant]
US 20140294097A1 · Seregin et al. · 2014 [cited by applicant]
US 20150022633A1 · Zhang et al. · 2015 [cited by applicant]
US 20150131739A1 · Lee et al. · 2015 [cited by applicant]
US 20160012855A1 · Krishnan · 2016 [cited by applicant]
US 20170318304A1 · Sanghvi et al. · 2017 [cited by applicant]
US 20170332099A1 · Lee et al. · 2017 [cited by applicant]
US 20170347128A1 · Panusopone · 2017 [cited by examiner]
US 20180041769A1 · Chuang et al. · 2018 [cited by applicant]
US 20180192071A1 · Chuang et al. · 2018 [cited by applicant]
US 20190037216A1 · Cho · 2019 [cited by applicant]
US 20190141334A1 · Lim et al. · 2019 [cited by applicant]
CN 101415122A · 2009 [cited by applicant]
CN 102204254A · 2011 [cited by applicant]
CN 102598670A · 2012 [cited by applicant]
CN 102611886A · 2012 [cited by applicant]
CN 102710934A · 2012 [cited by applicant]
CN 103460695A · 2013 [cited by applicant]
WO WO2012045225A1 · 2012 [cited by applicant]
WO WO2016078511A1 · 2016 [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/CN2017/096382 mailed Oct. 27, 2017. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/CN2018/071518 mailed Mar. 26, 2018. [cited by applicant]
Extended European Search Report for European Application No. 18735922.9 dated Mar. 31, 2020. [cited by applicant]
Alshina et al., Bi-Directional Optical Flow (JCTVC-C204). Oct. 7-15, 2010;7 pages. [cited by applicant]
Aslhina et al., Known Tools Performance Investigation for Next Generation Video Coding (VCEG-AZ05). Jun. 19-26, 2015;16 pages. [cited by applicant]
Chen et al., Algorithm Description of Joint Exploration Test Model 4. Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016, Document JVET-D1001v1. … [cited by applicant]
Chen et al., Algorithm Description of Joint Exploration Test Model 6 (JEM6). Joint Video Exploration Team (JVET) Document JVET-F1001. Mar. 31-Apr. 7, 2017; 49 pages. [cited by applicant]
Chen et al., Decoder-Side Motion Vector Refinement Based on Bilateral Template Matching. Joint Video Exploration Team (JVET) Document JVET-D0029. Oct. 15-Oct. 21, 2016; 4 pages. [cited by applicant]
Chen et al., EE3: Decoder-Side Motion Vector Refinement Based on Bilateral Template Matching. Joint Video Exploration Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 4th Meeting: Chengdu, CN, Oct. 15-21, 2016,… [cited by applicant]
Chiu et al., CE1: Report of self derivation of motion estimation in TMuC 0.9. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 4th Meeting: Daegu, KR, Jan. 20-28, 2011, Doc… [cited by applicant]
Chiu et al., Decoder-side Motion Estimation and Wiener filter for HEVC. Intel Corporation. Dec. 31, 2016. 6 pages. [cited by applicant]
Huang et al., A 249Mpixel/s HEVC video-decoder chip for Quad Full HD applications. Digest of Technical Papers. IEEE International Solid-State Circuits Conference (ISSCC). Feb. 2013;162-4. [cited by applicant]
Marpe et al., Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC Video Compression Standard. IEEE Transactions on Circuits and Sytems for Video Technology. Jul. 2003;13(7): 620-36. [cited by applicant]
Qu et al., A frame rate up-conversion method with quadruple motion vector post-processing. IEEE. 2016. 1686-90. [cited by applicant]
Ueda, CE1: Refinement Motion Compensation using DMVD with merge extension. Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 JCT1/SC29/WG11. JCTVC-D120. Jan. 2011. 13 pages. [cited by applicant]