IP Library › Granted Patent US 12,200,189
Granted Patent B2
US 12,200,189 · App. 17/645,968 · Granted Jan 14, 2025

Systems and methods for inter-frame prediction

Inventors: Feiyang Zeng (Hangzhou, CN); Dong Jiang (Hangzhou, CN); Jucai Lin (Hangzhou, CN); Jun Yin (Hangzhou, CN); Cheng Fang (Hangzhou, CN)
Assignee: ZHEJIANG DAHUA TECHNOLOGY CO., LTD.
H04N19/105H04N19/14H04N19/176H04N19/52H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,200,189
App. No.
17/645,968
Granted
Jan 14, 2025
Kind
B2
Abstract

A method for inter-frame prediction may include obtaining a current coding unit in an image. The method may also include determining feature information of the current coding unit, the feature information being associated with at least one of a size feature or a texture feature. The method may also include determining, based on the feature information of the current coding unit, an MV candidate list. The method may also include determining, based on the MV candidate list, a prediction result of the current coding unit.

Claims (58)

1. A system, comprising:

at least one storage device including a set of instructions; and

at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is directed to cause the system to perform operations including:

obtaining a current coding unit in an image;

determining feature information of the current coding unit, the feature information being associated with a texture feature;

determining, based on the feature information of the current coding unit, a motion vector (MV) candidate list; and

determining, based on the MV candidate list, a prediction result of the current coding unit, wherein the determining, based on the feature information of the current coding unit, a motion vector (MV) candidate list includes:

determining, based on the feature information, image blocks that are located along each of two adjacent boundaries of the current coding unit; and

determining, based on the image blocks that are located along each of the two adjacent boundaries of the current coding unit, the motion vector (MV) candidate list, wherein the two adjacent boundaries of the current coding unit include a first boundary satisfying a condition and a second boundary, and one or more MVs of image blocks located along the first boundary are arranged in the MV candidate list in front of MVs of image blocks located along the second boundary.

2. The system of claim 1 , the determining, based on the feature information of the current coding unit, an MV candidate list includes:

determining, based on the feature information of the current coding unit, one or more motion vectors (MVs) of one or more image blocks associated with the current coding unit, each of the one or more image blocks having been encoded; and

determining, based at least in part on the one or more MVs, the MV candidate list.

3. The system of claim 2 , wherein the determining, based on the feature information of the current coding unit, one or more motion vectors (MVs) of one or more image blocks associated with the current coding unit includes:

obtaining the MVs of the one or more image blocks,

wherein at least one of a count of the one or more image blocks is determined based on the feature information.

4. The system of claim 3 , wherein the determining the one or more image blocks includes:

determining, based on the feature information, a count of image blocks that are located along each of two adjacent boundaries of the current coding unit; and

determining, based on the count of image blocks that are located along each of the two adjacent boundaries of the current coding unit, the one or more image blocks.

5. The system of claim 4 , wherein a first count of image blocks located along the first boundary exceeds a second count of image blocks located along the second boundary.

6. The system of claim 5 , wherein the determining, based on the feature information, a count of image blocks that are located along each of the two adjacent boundaries of the current coding unit includes;

determining, based on a prediction mode of the current coding unit, an initial count of image blocks located along the first boundary and an initial count of image blocks that located along the second boundary;

increasing the initial count of image blocks located along the first boundary to obtain the first count; and

decreasing the initial count of image blocks that located along the second boundary to obtain the second count.

7. The system of claim 5 , wherein

the feature information includes a size feature,

the size feature includes information associated with lengths of the two adjacent boundaries of the current coding unit; and

the first boundary satisfying the condition includes that the first boundary has a longer length among the two adjacent boundaries.

8. The system of claim 5 , wherein

the texture feature includes information associated with a texture direction of the current coding unit; and

the first boundary satisfying the condition includes that the texture direction of the current coding unit points to the first boundary.

9. The system of claim 8 , wherein the determining feature information of the current coding unit includes:

performing an intra-frame prediction on the current coding unit using multiple angular prediction modes with different prediction directions;

determining a rate distortion (RD) cost corresponding to each of the multiple angular prediction modes to obtain multiple RDcosts; and

designating a prediction direction of one of the multiple angular prediction modes corresponding to a minimum RDcost among the multiple RDcosts as the texture direction.

10. The system of claim 3 , wherein

the texture feature includes a texture similarity degree between the current coding unit and each of the one or more image blocks; and

an MV of an image block with a higher texture similarity degree with the current coding unit is arranged in front of an MV of an image block with a lower texture similarity degree with the current coding unit in the MV candidate list.

11. The system of claim 2 , wherein the determining, based at least in part on the one or more MVs, the MV candidate list includes:

determining one or more co-located images of the image;

determining one or more co-located blocks in each of the one or more co-located images, each of the one or more co-located blocks corresponding to one of the one or more image blocks;

generating, based on each of first motion vectors (MVs) of the one or more co-located blocks, multiple second MVs of the current coding unit, each of the first MVs of the one or more co-located blocks corresponding to the multiple second MVs of the current coding unit;

determining, based at least in part on the one or more MVs and the multiple second MVs of the current coding unit, the MV candidate list.

12. The system of claim 11 , wherein the determining, based at least in part on the one or more MVs and the multiple second MVs of the current coding unit, the MV candidate list includes:

obtaining a first MV candidate list, the first MV candidate list including a plurality of historical MVs of a plurality of first image blocks, each of the plurality of historical MVs having been used to encode one of a plurality of first image blocks;

determining, from the plurality of historical MVs, one or more historical MVs of one or more first image blocks according to a first order;

determining, based at least in part on the one or more MVs, the multiple second MVs, the one or more historical MVs of the one or more first image blocks, and a second order, the MV candidate list.

13. The system of claim 1 , wherein the feature information includes a size feature, the size feature includes at least one of information associated with lengths of two adjacent boundaries of the current coding unit, information associated with area of the current coding unit, and information associated with total count of pixels in the current coding unit.

14. The system of claim 1 , wherein the texture feature includes a texture complexity, wherein the texture complexity indicates whether a texture of the current coding unit is flat or irregular.

15. The system of claim 1 , wherein the feature information includes a size feature, the first boundary and the second boundary are determined the size feature or the texture feature of the current coding unit.

16. The system of claim 2 , wherein the determining, based at least in part on the one or more MVs, the MV candidate list includes:

determining, based on the one or more MVs, a preliminary second MV candidate list;

adding one or more first MVs or scaled first MVs into the preliminary second MV candidate list to obtain a second MV candidate list.

17. The system of claim 16 , wherein the adding one or more first MVs or scaled first MVs into the preliminary second MV candidate list to obtain a second MV candidate list includes:

obtaining a first MV candidate list associated with the current coding unit in an image, the first MV candidate list including a plurality of MVs of a plurality of first image blocks, each of the plurality of MVs having been used to encode one of the plurality of first image blocks;

determining, from the plurality of MVs, the one or more first MVs of one or more first image blocks according to a first order; and

adding the one or more first MVs or scaled first MVs into the preliminary second MV candidate list according to a second order to obtain the second MV candidate list.

18. The system of claim 17 , further including:

determining an MV from the second MV candidate list as a motion vector prediction (MVP) of the current coding unit.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2022
From: ZENG, FEIYANG; JIANG, DONG; LIN, JUCAI; YIN, JUN; FANG, CHENG
To: ZHEJIANG DAHUA TECHNOLOGY CO., LTD.
Reel/Frame 059219/0199 →
Priority Claims (3)
CN 201910555587.5 · Jun 25, 2019 · national
CN 201910556619.3 · Jun 25, 2019 · national
CN 201910775404.0 · Aug 21, 2019 · national
Continuity (2)
Continuation PCTCN2020098125 · Jun 24, 2020
Related Publication 20220124321A1 · Apr 21, 2022
References Cited (58)
US 7181050B1 · Daly et al. · 2007 [cited by applicant]
US 10362330B1 · Li et al. · 2019 [cited by applicant]
US 20130294512A1 · Song et al. · 2013 [cited by applicant]
US 20140133571A1 · Kim · 2014 [cited by applicant]
US 20140301471A1 · Lin · 2014 [cited by applicant]
US 20150208086A1 · Chen et al. · 2015 [cited by applicant]
US 20150208087A1 · Park et al. · 2015 [cited by applicant]
US 20150256850A1 · Kottke et al. · 2015 [cited by applicant]
US 20150296218A1 · Pasupuleti et al. · 2015 [cited by applicant]
US 20150326875A1 · Chen et al. · 2015 [cited by applicant]
US 20150373359A1 · He · 2015 [cited by examiner]
US 20160227218A1 · Trudeau · 2016 [cited by examiner]
US 20170310988A1 · Lin et al. · 2017 [cited by applicant]
US 20180070105A1 · Jin et al. · 2018 [cited by applicant]
US 20180192071A1 · Chuang et al. · 2018 [cited by applicant]
US 20180255316A1 · Zhang et al. · 2018 [cited by applicant]
US 20180295379A1 · Lim et al. · 2018 [cited by applicant]
US 20190014338A1 · Laroche et al. · 2019 [cited by applicant]
US 20190149838A1 · Zhang et al. · 2019 [cited by applicant]
US 20190174136A1 · Jun et al. · 2019 [cited by applicant]
US 20200120344A1 · Mao et al. · 2020 [cited by applicant]
US 20200228830A1 · Zhang et al. · 2020 [cited by applicant]
US 20200396462A1 · Zhang et al. · 2020 [cited by applicant]
US 20210160532A1 · Zhang · 2021 [cited by examiner]
US 20210266525A1 · Tsukuba · 2021 [cited by applicant]
US 20210321102A1 · Chen · 2021 [cited by applicant]
CN 1688164A · 2005 [cited by applicant]
CN 103517069A · 2014 [cited by applicant]
CN 104394409A · 2015 [cited by applicant]
CN 105338363A · 2016 [cited by applicant]
CN 105959699A · 2016 [cited by applicant]
CN 108184115A · 2018 [cited by applicant]
CN 110213588A · 2019 [cited by applicant]
CN 110213590A · 2019 [cited by applicant]
CN 110460859A · 2019 [cited by applicant]
EP 2096871A2 · 2009 [cited by applicant]
WO 2014058795A1 · 2014 [cited by applicant]
WO 2017197126A1 · 2017 [cited by applicant]
WO 2018205914A1 · 2018 [cited by applicant]
Fernandez DG, Del Barrio AA, Botella G, Garc-a C, Prieto M, Hermida R. Complexity reduction in the HEVC/H265 standard based on smooth region classification. Digital Signal Processing. Feb. 1, 2018;73:24-39. (Year: 2018). [cited by examiner]
International Search Report in PCT/CN2020/098125 mailed on Oct. 9, 2020, 6 pages. [cited by applicant]
Written Opinion in PCT/CN2020/098125 mailed on Oct. 9, 2020, 6 pages. [cited by applicant]
First Office Action in Chinese Application No. 201910556619.3 mailed on Jan. 13, 2021, 15 pages. [cited by applicant]
First Office Action in Chinese Application No. 201910555587.5 mailed on Jan. 12, 2021, 11 pages. [cited by applicant]
First Office Action in Chinese Application No. 201910775404.0 mailed on Mar. 3, 2021, 23 pages. [cited by applicant]
Mathias Wien, Variable Block-Size Transforms for H.264/AVC, IEEE Transactions On Circuits and Systems for Video Technology, 13(7): 604-613, 2003. [cited by applicant]
Mahsa T. Pourazad et al., HEVC: The New Gold Standard for Video Compression, IEEE Consumer Electronics Magazine, 2012, 11 pages. [cited by applicant]
Chen, Yi-Wen et al., CE4-related: A Second ATMVP Candidate, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0105, 2018. [cited by applicant]
Xu, Liying et al., Candidate List Reordering, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-K0065-V4, 2018. [cited by applicant]
Hahyun Lee, Non-CE4: HMVP Unification Between the Merge and MVP List, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-N0373-V2, 2019. [cited by applicant]
Jie Zhao, CE4-Related: History Based Affine Merge Candidate, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L0305, 2018. [cited by applicant]
Chen, Yi-Wen et al., Non-CE4: Simplification of HMVP in Merge List Construction, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O0589, 2019. [cited by applicant]
Suhong Wang, CE8-1.7: Single HMVP Table for All CUs Inside the Shared Merge List Region for IBC, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O0078, 2019. [cited by applicant]
Xu, Liying et al., CE4-related: Simplification of Candidate List Derivation for MMVD Mode, Joint Video Experts Team (JVET) ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 2019, 7 pages. [cited by applicant]
Xu, Liying et al., CE4 related: Candidate List Reordering, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 2018, 6 pages. [cited by applicant]
Hsu, Chih-Wei et al., Description of SDR Video Coding Technology Proposal by MediaTek, Joint Video Experts Team (JVET) of ITU-T SG 18 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 2018, 64 pages. [cited by applicant]
Gary J. Sullivan et al., Overview of the 1-10,13 High Efficiency Video Coding (HEVC) Standard, IEEE Transactions On Circuits and Systems for Video Technology, 22(12): 1649-1668, 2012. [cited by applicant]
The Partial Supplementary European Search Report in European Application No. 20630559.9 mailed on Oct. 21, 2022, 15 pages. [cited by applicant]
Cited By (1)
US 12,701,261