IP Library Granted Patent US 11,438,623
Granted Patent B2
US 11,438,623 · App. 17/270,243 · Granted Sep 6, 2022

Method and device for encoding and decoding video using inter-prediction

Inventors: Je Won Kang (Seoul, KR); Sang Hyo Park (Seoul, KR); Seung Wook Park (Gyeonggi-do, KR); Wha Pyeong Lim (Gyeonggi-do, KR)
Assignees: Hyundai Motor Company; Kia Motors Corporation; Ehwa University-Industry Collaboration Foundation
H04N19/521H04N19/105H04N19/132H04N19/139H04N19/176H04N19/577H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,438,623
App. No.
17/270,243
Granted
Sep 6, 2022
Kind
B2
Abstract

A coding tool is provided to improve the compression performance of inter prediction and is used at the encoder/decoder side to adjust the correction of a motion vector based on a high level syntax. In addition, a method is provided for simply performing an integer sample search step of searching for an integer offset and a fractional sample refinement step of searching for a sub-pixel offset in relation to motion vector refinement among coding tools.

Claims (48)

1. A video decoding method for decoding a sequence of coded pictures on a block-by-block basis by a video decoding apparatus, comprising:

decoding, from a bitstream, at least one first high level syntax element indicating whether each of at least one coding tool is allowed, the at least one coding tool including a motion vector refinement;

decoding, from the bitstream, a second high level syntax element corresponding to each of the at least one coding tool depending on the at least one first high level syntax element, and setting, at a picture level, whether each of the at least one coding tool is allowed;

when it is identified that the motion vector refinement is allowed for a current picture including a target block based on the setting at the picture level, generating a refined motion vector for the target block, by calculating, for pairs of two candidate blocks which are within a preset offset range from initial motion vectors in bi-directional reference pictures of the target block, metrics between the two candidate blocks and searching for a motion vector candidate having a minimum metric; and

generating inter-predicted samples for the target block using the bi-directional reference pictures and the refined motion vector.

2. The method of claim 1 , wherein the first high level syntax element is decoded from a header of the sequence.

3. The method of claim 2 , wherein the decoding of the second high level syntax element includes:

when the first high level syntax element indicates that the motion vector refinement is allowed, decoding, from a picture header of each of the pictures belonging to the sequence, a picture level flag indicating whether the motion vector refinement is allowed.

4. The method of claim 3 , wherein, when the first high level syntax element indicates that the motion vector refinement is not allowed, the picture level flag is set to a value indicating that the motion vector refinement is not allowed without being decoded from the picture header.

5. The method of claim 1 , wherein, when the first high level syntax element indicates that the motion vector refinement is not allowed, a coding tool for the motion vector refinement is not allowed for any of the pictures in the sequence.

6. The method of claim 1 , wherein the generating of the refined motion vector includes:

an integer sample search operation of searching for an integer offset corresponding to a position having the minimum metric by calculating the metrics for pairs of the two candidate blocks which correspond to a pair of initial positions indicated by the initial motion vectors and pairs of two neighboring positions displaced from the initial positions in integer sample units, wherein the neighboring positions in the integer sample units have a square distribution around each of the initial positions;

a fractional sample refinement operation of, when the integer offset is not positioned at an outermost side of the square distribution, searching for an optimal sub-pixel offset by solving a parametric error surface equation using the position with the minimum metric and top, bottom, left, and right neighboring positions of the position with the minimum metric; and

generating an offset between the initial motion vector and the refined motion vector by adding the sub-pixel offset to the integer offset.

7. The method of claim 6 , wherein, when the metric between the two candidate blocks corresponding to the initial motion vectors is less than a preset first threshold, both the integer sample search operation and the fractional sample refinement operation are skipped.

8. The method of claim 6 , wherein, when the integer offset is zero, the fractional sample refinement operation is skipped.

9. The method of claim 6 , wherein when each of differences between the minimum metric and metrics which are acquired at integer coordinates of two or more positions in the integer sample search operation is less than a preset second threshold, the fractional sample refinement operation is skipped.

10. The method of claim 6 , wherein, when the initial motion vectors for the target block are generated by a merge mode, whether to execute the integer sample search operation or the fractional sample refinement operation is determined depending on magnitudes of motion vectors of neighboring blocks considered as merge candidates or a coding mode applied to the neighboring blocks.

11. A video decoding apparatus for decoding a sequence of coded pictures on a block-by-block basis, comprising:

a decoder configured to

decode, from a bitstream, at least one first high level syntax element indicating whether each of at least one coding tool is allowed, and

decode, from the bitstream, a second high level syntax element corresponding to each of the at least one coding tool depending on the at least one first high level syntax element and thereby set, at a picture level, whether each of the at least one coding tool is allowed, the at least one coding tool including motion vector refinement; and

a predictor configured to, when it is identified that the motion vector refinement is allowed for a current picture including a target block based on the setting at the picture level,

generate a refined motion vector for the target block, by calculating, for pairs of two candidate blocks which are within a preset offset range from initial motion vectors in bi-directional reference pictures of the target block, metrics between the two candidate blocks and searching for a motion vector candidate having a minimum metric, and

generate inter-predicted samples for the target block using the bi-directional reference pictures and the refined motion vector.

12. The apparatus of claim 11 , wherein the decoder is configured to decode the first high level syntax element from a header of the sequence.

13. The apparatus of claim 12 , wherein, when the first high level syntax element indicates that the motion vector refinement is allowed, the decoder is configured to decode, from a picture header of each of the pictures belonging to the sequence, a picture level flag indicating whether the motion vector refinement is allowed.

14. The apparatus of claim 13 , wherein, when the first high level syntax element indicates that the motion vector refinement is not allowed, the decoder is configured to set the picture level flag to a value indicating that the motion vector refinement is not allowed without decoding the same from the picture header.

15. The apparatus of claim 11 , wherein the predictor is configured to:

execute an integer sample search operation of searching for an integer offset corresponding to a position having the minimum metric by calculating the metrics for pairs of the two candidate blocks which correspond to a pair of initial positions indicated by the initial motion vectors and pairs of two neighboring positions displaced from the initial positions in integer sample units, wherein the neighboring positions in the integer sample units have a square distribution around each of the initial positions;

execute a fractional sample refinement operation of, when the integer offset is not positioned at an outermost side of the square distribution, searching for an optimal sub-pixel offset to search for sample positions by solving a parametric error surface equation using the position with the minimum metric and top, bottom, left, and right neighboring positions of the position with the minimum metric; and

generate an offset between the initial motion vector and the refined motion vector by adding the sub-pixel offset to the integer offset.

16. The apparatus of claim 15 , wherein, when the metric between the two candidate blocks corresponding to the initial motion vectors is less than a preset first threshold, the predictor is configured to skip both the integer sample search operation and the fractional sample refinement operation.

17. The apparatus of claim 15 , wherein, when the integer offset is zero, the predictor is configured to skip the fractional sample refinement operation.

18. The apparatus of claim 15 , wherein when each of differences between the minimum metric and the metrics which are acquired at integer coordinates of two or more positions in the integer sample search operation is less than a preset second threshold, the predictor is configured to skip the fractional sample refinement operation.

19. The apparatus of claim 15 , wherein, when the initial motion vectors for the target block are generated by a merge mode, the predictor is configured to determine whether to execute the integer sample search operation or the fractional sample refinement operation depending on magnitudes of motion vectors of neighboring blocks considered as merge candidates or a coding mode applied to the neighboring blocks.

20. A video encoding method for encoding a sequence of coded pictures on a block-by-block basis by a video encoding apparatus, comprising:

encoding, in a sequence header, at least one first high level syntax element indicating whether each of at least one coding tool is allowed, the at least one coding tool including motion vector refinement;

encoding a second high level syntax element of a picture level related to whether each of the at least one coding tool is allowed depending on the at least one first high level syntax element;

when it is identified that the motion vector refinement is allowed for a current picture including a target block based on setting of the picture level, generating a refined motion vector for the target block, by calculating, for pairs of two candidate blocks which are within a preset offset range from initial motion vectors in bi-directional reference pictures of the target block, metrics between the two candidate blocks and searching for a motion vector candidate having a minimum metric; and

generating inter-prediction samples for the target block using the bi-directional reference pictures and the refined motion vector.

21. The method of claim 20 , wherein the encoding of the second high level syntax element includes:

when the first high level syntax element indicates that the motion vector refinement is allowed, encoding, in a picture header of each of the pictures belonging to the sequence, a picture level flag indicating whether the motion vector refinement is allowed.

22. The method of claim 20 , wherein the generating of the refined motion vector includes:

an integer sample search operation of searching for an integer offset corresponding to a position having the minimum metric by calculating the metrics for pairs of the two candidate blocks which correspond to a pair of initial positions indicated by the initial motion vectors and pairs of two neighboring positions displaced from the initial positions in integer sample units, wherein the neighboring positions in the integer sample units have a square distribution around each of the initial positions;

a fractional sample refinement operation of, when the integer offset is not positioned at an outermost side of the square distribution, searching for an optimal sub-pixel offset by solving a parametric error surface equation using the position with the minimum metric and top, bottom, left, and right neighboring positions of the position with the minimum metric; and

generating an offset between the initial motion vector and the refined motion vector by adding the sub-pixel offset to the integer offset.

23. The method of claim 22 , wherein, when the metric between the two candidate blocks corresponding to the initial motion vectors is less than a preset first threshold, both the integer sample search operation and the fractional sample refinement operation are skipped.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2021
From: KANG, JE WON; PARK, SANG HYO; PARK, SEUNG WOOK; LIM, WHA PYEONG
To: HYUNDAI MOTOR COMPANY; KIA MOTORS CORPORATION; EWHA UNIVERSITY - INDUSTRY COLLABORATION FOUNDATION
Reel/Frame 055355/0043 →
Priority Claims (3)
KR 10-2019-0081926 · Jul 8, 2019 · national
KR 10-2019-0170099 · Dec 18, 2019 · national
KR 10-2020-0083978 · Jul 8, 2020 · national
Continuity (1)
Related Publication 20220124365A1 · Apr 21, 2022