IP Library › Granted Patent US 12,407,814
Granted Patent B2
US 12,407,814 · App. 18/608,642 · Granted Sep 2, 2025

Video processing method and device thereof

Inventors: Suhong Wang (Shenzhen, CN); Xiaozhen Zheng (Shenzhen, CN); Shanshe Wang (Shenzhen, CN); Siwei Ma (Shenzhen, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
H04N19/105H04N19/129H04N19/137H04N19/176H04N19/1883H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,407,814
App. No.
18/608,642
Granted
Sep 2, 2025
Kind
B2
Abstract

A bitstreams generating method includes obtaining a historical motion information candidate list used for encoding each of a plurality of image blocks included in a region of a current frame, encoding the plurality of image blocks according to the historical motion information candidate list, and generating bitstreams including one or more indexes of motion information. The historical motion information candidate list is a history-based motion vector prediction (HMVP) candidate list. During the process of encoding all the plurality of image blocks included in the region, the historical motion information candidate list is not updated. Only the motion information of the last image block in the region is used to update the historical motion information candidate list after the encoding process of the last image block is finished.

Claims (64)

1. A bitstreams generating method comprising:

obtaining a historical motion information candidate list, wherein the historical motion information candidate list is used for encoding each of a plurality of image blocks included in a region of a current frame; and

encoding the plurality of image blocks according to the historical motion information candidate list, wherein:

the historical motion information candidate list is a history-based motion vector prediction (HMVP) candidate list;

during the process of encoding all the plurality of image blocks included in the region, the historical motion information candidate list is not updated;

only the motion information of the last image block in the region is used to update the historical motion information candidate list after the encoding process of the last image block is finished; and

the historical motion information candidate list is updated by:

deleting candidate historical motion information at a second position in the historical motion information candidate list, the second position being a position in the historical motion information candidate list that is farthest to a first position in the historical motion information candidate list, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list; or

deleting candidate historical motion information that is same as candidate historical motion information at a first position, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list; and

generating bitstreams, the bitstreams including one or more indexes of motion information.

2. The method of claim 1 , wherein the same historical motion information candidate list is used for encoding all the plurality of image blocks included in the region, wherein the used same historical motion information candidate list includes the same motion information and does not include the motion information of all the plurality of image blocks included in the region.

3. The method of claim 1 , wherein:

a size of the region is equal to 16×16 pixels; and

the plurality of image blocks include 4×4 image blocks obtained by dividing the region according to a quadtree method.

4. The method of claim 1 , wherein the historical motion information candidate list is updated by adding new candidate historical motion information to a position in the historical motion information candidate list that is first selected for using in a process of constructing a motion information list, the motion information list being constructed based on the historical motion information candidate list.

5. The method according to claim 1 , wherein:

candidate historical motion information in the historical motion information candidate list adopted to perform the prediction for different ones of the plurality of image blocks has a same order as candidate historical motion information in the historical motion information candidate list.

6. The method of claim 1 , wherein:

the region is a first region and the plurality of image blocks are a plurality of first image blocks; and

the historical motion information candidate list is obtained based on motion information of one or more second image blocks each being an encoded or decoded image block in a second region of the current frame outside the first region.

7. The method according to claim 6 , wherein obtaining the historical motion information candidate list includes:

updating the historical motion information candidate list using the motion information of at least one of the one or more second image blocks, including:

updating the historical motion information candidate list according to the motion information of a last second image block, of the one or more second image blocks, in the second region that was encoded or decoded, a lower right corner of the last second image block in the second region that was encoded or decoded coinciding with a lower right corner of the second region.

8. The method according to claim 7 , wherein updating the historical motion information candidate list further includes:

adding first candidate historical motion information to the first position.

9. The method according to claim 7 , wherein when the historical motion information candidate list is updated:

a number of pieces of candidate historical motion information added to the historical motion information candidate list is smaller than a first preset value; and/or

a number of pieces of candidate historical motion information deleted from the historical motion information candidate list is smaller than a second preset value.

10. The method according to claim 6 , wherein:

the first region and the second region belong to a same coding tree unit (CTU) row or a same CTU column;

the CTU row or the CTU column includes at least one CTU; and

each of the at least one CTU includes at least one region; and

the second region is a region that is encoded before the first region.

11. The method according to claim 6 , wherein during the process of encoding all the plurality of image blocks included in the region:

the historical motion information candidate list does not include the motion information of all the plurality of image blocks included in the region; or

the same historical motion information candidate list is used.

12. A device comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to:

obtain a historical motion information candidate list, wherein the historical motion information candidate list is used for encoding each of a plurality of image blocks included in a region of a current frame; and

encode the plurality of image blocks according to the historical motion information candidate list, wherein:

the historical motion information candidate list is a history-based motion vector prediction (HMVP) candidate list;

during the process of encoding all the plurality of image blocks included in the region, the historical motion information candidate list is not updated;

only the motion information of the last image block in the region is used to update the historical motion information candidate list after the encoding process of the last image block is finished; and

the historical motion information candidate list is updated by:

deleting candidate historical motion information at a second position in the historical motion information candidate list, the second position being a position in the historical motion information candidate list that is farthest to a first position in the historical motion information candidate list, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list; or

deleting candidate historical motion information that is same as candidate historical motion information at a first position, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list.

13. The device of claim 12 , wherein:

the region is a first region and the plurality of image blocks are a plurality of first image blocks;

the current frame includes a second region that is encoded before the first region and includes one or more second image blocks; and

obtain the historical motion information candidate list includes:

update the historical motion information candidate list according to the motion information of a last second image block of the one or more second image blocks, a lower right corner of the last second image block in the second region coinciding with a lower right corner of the second region.

14. The device of claim 12 , wherein during the process of encoding all the plurality of image blocks included in the region:

the historical motion information candidate list does not include the motion information of all the plurality of image blocks included in the region; or

the same historical motion information candidate list is used.

15. A device comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to: obtain a historical motion information candidate list, wherein the historical motion information candidate list is used for encoding each of a plurality of image blocks included in a region of a current frame; and decode the plurality of image blocks according to the historical motion information candidate list, wherein: the historical motion information candidate list is a history-based motion vector prediction (HMVP) candidate list; during the process of decoding all the plurality of image blocks included in the region, the historical motion information candidate list is not updated only the motion information of the last image block in the region is used to update the historical motion information candidate list after the decoding process of the last image block is finished; and the historical motion information candidate list is updated by deleting candidate historical motion information at a second position in the historical motion information candidate list, the second position being a position in the historical motion information candidate list that is farthest to a first position in the historical motion information candidate list, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list; or deleting candidate historical motion information that is same as candidate historical motion information at a first position, and the first position is a position in the historical motion information candidate list that is first selected for constructing a motion information list.

16. The device of claim 15 , wherein the same historical motion information candidate list is used for decoding all the plurality of first image blocks included in the first region, and the same historical motion information candidate list includes the same motion information.

17. The device of claim 15 , wherein the historical motion information candidate list is obtained by:

updating the historical motion information candidate list according to the motion information of a last second image block of the one or more second image blocks that have been encoded or decoded, a lower right corner of the last second image block in the second region coinciding with a lower right corner of the second region.

18. The device of claim 15 , wherein the historical motion information candidate list is updated further by:

adding first candidate historical motion information to the first position.

19. The device of claim 15 , wherein during the process of decoding all the plurality of image blocks included in the region:

the historical motion information candidate list does not include the motion information of all the plurality of image blocks included in the region; or

the same historical motion information candidate list is used.

Priority Claims (1)
WO PCT/CN2018/125843 · Dec 29, 2018 · international
Continuity (3)
Continuation 17362309 · Jun 29, 2021
Continuation PCTCN2019078049 · Mar 13, 2019
Related Publication 20240251073A1 · Jul 25, 2024
References Cited (67)
US 10440378B1 · Xu · 2019 [cited by examiner]
US 11671590B2 · Lee · 2023 [cited by examiner]
US 12284379B2 · Li · 2025 [cited by examiner]
US 20130170549A1 · Li et al. · 2013 [cited by applicant]
US 20130272408A1 · Chen et al. · 2013 [cited by applicant]
US 20130272412A1 · Seregin et al. · 2013 [cited by applicant]
US 20140161186A1 · Zhang et al. · 2014 [cited by applicant]
US 20150317801A1 · Bentley · 2015 [cited by examiner]
US 20160057441A1 · Skupin et al. · 2016 [cited by applicant]
US 20170034512A1 · Casula · 2017 [cited by applicant]
US 20170280159A1 · Xu et al. · 2017 [cited by applicant]
US 20180109805A1 · Takehara et al. · 2018 [cited by applicant]
US 20200059658A1 · Chien · 2020 [cited by examiner]
US 20200077085A1 · Zhang · 2020 [cited by examiner]
US 20200107017A1 · Hung · 2020 [cited by examiner]
US 20200112715A1 · Hung et al. · 2020 [cited by applicant]
US 20200186819A1 · Xu et al. · 2020 [cited by applicant]
US 20200280724A1 · Jeon · 2020 [cited by examiner]
US 20210195229A1 · Esenlik et al. · 2021 [cited by applicant]
US 20210211709A1 · Zhang et al. · 2021 [cited by applicant]
US 20210266590A1 · Abe et al. · 2021 [cited by applicant]
US 20210321089A1 · Lin et al. · 2021 [cited by applicant]
US 20210329227A1 · Zheng et al. · 2021 [cited by applicant]
CN 103535039A · 2014 [cited by applicant]
CN 103636218A · 2014 [cited by applicant]
CN 103765896A · 2014 [cited by applicant]
CN 103931192A · 2014 [cited by applicant]
CN 103931196A · 2014 [cited by applicant]
CN 104079944A · 2014 [cited by applicant]
CN 104796724A · 2015 [cited by applicant]
CN 106878749A · 2017 [cited by applicant]
CN 107005692A · 2017 [cited by applicant]
CN 103535039B · 2018 [cited by applicant]
CN 108781284A · 2018 [cited by applicant]
CN 109068140A · 2018 [cited by applicant]
EP 3843389A1 · 2021 [cited by applicant]
JP 2003209846A · 2003 [cited by applicant]
JP 7415065B2 · 2024 [cited by applicant]
KR 101210892B1 · 2012 [cited by applicant]
KR 20130002242A · 2013 [cited by applicant]
KR 1020140043242A · 2014 [cited by applicant]
WO 2012097742A1 · 2012 [cited by applicant]
WO 2017043734A1 · 2017 [cited by applicant]
WO 2018058526A1 · 2018 [cited by applicant]
WO 2020003266A1 · 2020 [cited by applicant]
WO 2020018297A1 · 2020 [cited by applicant]
WO 2020060329A1 · 2020 [cited by applicant]
WO 2020114420A1 · 2020 [cited by applicant]
WO 20200150577A1 · 2020 [cited by applicant]
Suhong Wang, et al., CE4-related: Simplification for share merge list and HMVP harmonization process. Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-NO486_v4, 14th Meeting: Genev… [cited by applicant]
The World Intellectual Property Organization (WIPO) International Search Report for PCT/CN2019/078049 Sep. 17, 2019 11 pages (translation included). [cited by applicant]
The World Intellectual Property Organization (WIPO) International Search Report for PCT/CN2018/125843 Oct. 10, 2019 10 pages (translation included). [cited by applicant]
The World Intellectual Property Organization (WIPO) International Search Report for PCT/CN2019/109659 Jan. 10, 2020 9 pages (translation included). [cited by applicant]
<<Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macau, CN, Oct. 8-12, 2018, Document: JVET-L0575-v2>>; Oct. 5, 2018; Weiwei Xu et al. CE4-Related: CTU-Level Initializatio… [cited by applicant]
<<Joint Video Experts Team (JVET)>>, Oct. 12, 2018, Li Zhang et al. CE4: History-based Motion Vector Prediction (Test 4.4.7). [cited by applicant]
Benjamin Bross et al: “Versatile Video Coding (Draft 3)”, JVET Meeting; Oct. 3, 2018-Oct. 12, 2018, No. JVET-L1001-v7; JVET-L1001, Dec. 20, 2018, pp. 1-223. [cited by applicant]
Zhao (LGE) J et al: “CE4-related: Simplification to History Based Motion Vector Prediction”, 124. MPEG Meeting; Oct. 8, 2018-Oct. 12, 2018; Macao; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m44333 Oct.… [cited by applicant]
Park, Naeri et al., CE4-related: History-Based Motion Vector Predict ion considering parallel processing , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macau, CN, Oct. … [cited by applicant]
Semih Esenlik, et al. , Non-CE4: Parallel Merge Estimation for VVC , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29/WG 11 , JVET-L0216 , 12th Meeting: Macao, CN, Sep. 2018, pp. 1-7. [cited by applicant]
Li Zhang, et al. , CE4-related: History-based Motion Vector Prediction , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 , JVET-K0104-v5 , 11th Meeting: Ljubljana, SI, Jul. 2018, pp. 1-… [cited by applicant]
Xiaozhong Xu, et al., CE4-related: History based spatial-temporal MV prediction , Joint Video Experts Team (JVET) of ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 , JVET-L0302-v1 , 12th Meeting: Macao, CN, Sep. 2018, pp… [cited by applicant]
Yi-Wen Chen, et al., CE4-related: Modified History-based MVP to support parallel processing , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 , JVET-L0106 , 12th Meeting: Macao, CN, Sep… [cited by applicant]
Timofey Solovyev, et al., CE4-related: History-based MVP without using the last lookup table entry , Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 , JVET-L0425-v6 , 12th Meeting: Maca… [cited by applicant]
B. Girod, “Motion-compensating prediction with fractional-pel accuracy,” IEEE Transactions on Communications, vol. 41, No. 4, pp. 604-612, 1993. [cited by applicant]
https://jvet.hhi.fraunhofer.de/jvet/VVCSoftware_VTM. [cited by applicant]
Truong Quang Vinh, et al., “Frame rate up-conversion using forward-backward jointing motion estimation and spatio-temporal motion vector smoothing”, 2009 International Conference on Computer Engineering and Systems, Dec… [cited by applicant]
Hongyu Hu, et al., “Representation and Analysis of Pedestrian Crossing States Based on Video Tracking”, Jun. 20, 2009. [cited by applicant]