Encoder which generates prediction image to be used to encode current block
An encoder includes circuitry and memory connected to the circuitry. The circuitry, in operation: derives, as a first parameter, a total sum of absolute values of sums of horizontal gradient values respectively for pairs of relative pixel positions; derives, as a second parameter, a total sum of absolute values of sums of vertical gradient values respectively for the pairs of relative pixel positions; derives, as a third parameter, a total sum of horizontal-related pixel difference values respectively for the pairs of relative pixel positions; derives, as a fourth parameter, a total sum of vertical-related pixel difference values respectively for the pairs of relative pixel positions; derives, as a fifth parameter, a total sum of vertical-related sums of horizontal gradient values respectively for the pairs of relative pixel positions; and generates a prediction image to be used to encode the current block using the first, second, third, fourth, and fifth parameters.
1 . An encoder comprising:
a memory; and
a processor connected to the memory and configured to generate a prediction image in order to encode a current block based on a first parameter, a second parameter, a third parameter, a fourth parameter, and a fifth parameter that are derived in a bi-directional optical flow (BDOF) process, wherein
the first parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
x
1
+
I
x
0
)
;
[
Math
.
1
]
the second parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
y
1
+
I
y
0
)
;
[
Math
.
2
]
the third parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
x
1
+
I
x
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
3
]
the fourth parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
y
1
+
I
y
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
4
]
and
the fifth parameter is derived based on the following expression,
∑
[
i
,
j
]
∈
Ω
(
sign
(
I
y
1
+
I
y
0
)
×
(
I
x
1
+
I
x
0
)
)
,
[
Math
.
5
]
where:
Ω indicates a set of relative pixel positions;
[i, j] indicates a relative pixel position in the set ( 2 defined by a horizontal position i and a vertical position j;
I x 0 indicates a horizontal gradient value at a first pixel position in a first gradient image, and I x 1 indicates a horizontal gradient value at the first pixel position in a second gradient image, the first pixel position being determined based on a relative pixel position [i, j], the first gradient image and the second gradient image corresponding to the current block;
I y 0 indicates a vertical gradient value at the first pixel position in the first gradient image, and I y 1 indicates a vertical gradient value at the first pixel position in the second gradient image;
I 0 indicates a pixel value at the first pixel position in a first interpolated image corresponding to the current block;
I 1 indicates a pixel value at the first pixel position in a second interpolated image corresponding to the current block;
the abs function outputs an absolute value of an argument;
the sign function outputs a sign of an argument which is −1, 0, or 1; and
I x 0 indicates a difference between a value of a right sample and a value of a left sample, the right sample being adjacent to a right side of a target sample, the left sample being adjacent to a left side of the target sample, the target sample being included in a first prediction block.
2 . A decoder comprising:
a memory; and
a processor connected to the memory and configured to generate a prediction image in order to decode a current block based on a first parameter, a second parameter, a third parameter, a fourth parameter, and a fifth parameter that are derived in a bi-directional optical flow (BDOF) process, wherein
the first parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
x
1
+
I
x
0
)
;
[
Math
.
1
]
the second parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
y
1
+
I
y
0
)
;
[
Math
.
2
]
the third parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
x
1
+
I
x
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
3
]
the fourth parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
y
1
+
I
y
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
4
]
and
the fifth parameter is derived based on the following expression,
∑
[
i
,
j
]
∈
Ω
(
sign
(
I
y
1
+
I
y
0
)
×
(
I
x
1
+
I
x
0
)
)
,
[
Math
.
5
]
where:
Ω indicates a set of relative pixel positions;
[i, j] indicates a relative pixel position in the set ( 2 defined by a horizontal position i and a vertical position j;
I x 0 indicates a horizontal gradient value at a first pixel position in a first gradient image, and I x 1 indicates a horizontal gradient value at the first pixel position in a second gradient image, the first pixel position being determined based on a relative pixel position [i, j], the first gradient image and the second gradient image corresponding to the current block;
I y 0 indicates a vertical gradient value at the first pixel position in the first gradient image, and I y 1 indicates a vertical gradient value at the first pixel position in the second gradient image;
I 0 indicates a pixel value at the first pixel position in a first interpolated image corresponding to the current block;
I 1 indicates a pixel value at the first pixel position in a second interpolated image corresponding to the current block;
the abs function outputs an absolute value of an argument;
the sign function outputs a sign of an argument which is −1, 0, or 1; and
I x 0 indicates a difference between a value of a right sample and a value of a left sample, the right sample being adjacent to a right side of a target sample, the left sample being adjacent to a left side of the target sample, the target sample being included in a first prediction block.
3 . A non-transitory computer readable medium that stores a bitstream and computer-executable instructions, the bitstream including a prediction parameter according to which the computer-executable instructions, when executed by a processor, cause a decoder to perform a decoding method of decoding a current block provided in the bitstream, the prediction parameter indicating a prediction process among prediction process candidates, the prediction process candidates including a bi-directional optical flow (BDOF) process in which a first parameter, a second parameter, a third parameter, a fourth parameter, and a fifth parameter are used to generate a prediction image of the current block, wherein
the first parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
x
1
+
I
x
0
)
;
[
Math
.
1
]
the second parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
abs
(
I
y
1
+
I
y
0
)
;
[
Math
.
2
]
the third parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
x
1
+
I
x
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
3
]
the fourth parameter is derived based on the following expression:
∑
[
i
,
j
]
∈
Ω
(
-
sign
(
I
y
1
+
I
y
0
)
×
(
I
0
-
I
1
)
)
;
[
Math
.
4
]
and
the fifth parameter is derived based on the following expression,
∑
[
i
,
j
]
∈
Ω
(
sign
(
I
y
1
+
I
y
0
)
×
(
I
x
1
+
I
x
0
)
)
,
[
Math
.
5
]
where:
Ω indicates a set of relative pixel positions;
[i, j] indicates a relative pixel position in the set Ω defined by a horizontal position i and a vertical position j;
I x 0 indicates a horizontal gradient value at a first pixel position in a first gradient image, and I x 1 indicates a horizontal gradient value at the first pixel position in a second gradient image, the first pixel position being determined based on a relative pixel position [i, j], the first gradient image and the second gradient image corresponding to the current block;
I y 0 indicates a vertical gradient value at the first pixel position in the first gradient image, and I y 1 indicates a vertical gradient value at the first pixel position in the second gradient image;
I 0 indicates a pixel value at the first pixel position in a first interpolated image corresponding to the current block;
I 1 indicates a pixel value at the first pixel position in a second interpolated image corresponding to the current block;
the abs function outputs an absolute value of an argument;
the sign function outputs a sign of an argument which is −1, 0, or 1; and
I x 0 indicates a difference between a value of a right sample and a value of a left sample, the right sample being adjacent to a right side of a target sample, the left sample being adjacent to a left side of the target sample, the target sample being included in a first prediction block.