Encoder, decoder, encoding method, and decoding method

1. An encoder, comprising:

circuitry; and

memory connected to the circuitry,

wherein, in operation, the circuitry:

derives an absolute value of a sum of horizontal gradient values which indicates an absolute value of a sum of a horizontal gradient value at a relative pixel position in a first range and a horizontal gradient value at a relative pixel position in a second range, for each of a plurality of pairs of a relative pixel position in the first range and a relative pixel position in the second range among a plurality of pairs of relative pixel positions, the first range including a first reference block for a current block, the second range including a second reference block for the current block, the plurality of pairs of relative pixel positions being determined commonly and relatively for the first range and the second range;

derives, as a first parameter, a total sum of a plurality of absolute values of sums of horizontal gradient values derived respectively for the plurality of pairs of relative pixel positions;

derives an absolute value of a sum of vertical gradient values which indicates an absolute value of a sum of a vertical gradient value at the relative pixel position in the first range and a vertical gradient value at the relative pixel position in the second range, for each of the plurality of pairs of relative pixel positions;

derives, as a second parameter, a total sum of a plurality of absolute values of sums of vertical gradient values derived respectively for the plurality of pairs of relative pixel positions;

derives, for each of the plurality of pairs of relative pixel positions, a pixel difference value which indicates a difference between the pixel value at the relative pixel position in the first range and the pixel value at the relative pixel position in the second range;

inverts or maintains a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of the sum of the horizontal gradient values which indicates the sum of the horizontal gradient value at the relative pixel position in the first range and the horizontal gradient value at the relative pixel position in the second range, and derives a horizontal-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the horizontal-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the horizontal gradient values;

derives, as a third parameter, a total sum of a plurality of horizontal-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions;

inverts or maintains a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of a sum of vertical gradient values which indicates a sum of the vertical gradient value at the relative pixel position in the first range and the vertical gradient value at the relative pixel position in the second range, and derives a vertical-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the vertical-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the vertical gradient values;

derives, as a fourth parameter, a total sum of a plurality of vertical-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions; and

generates a prediction image to be used to encode the current block using the first parameter, the second parameter, the third parameter, and the fourth parameter,

wherein the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4.

2. A decoder, comprising:

circuitry; and

memory connected to the circuitry,

wherein, in operation, the circuitry:

derives an absolute value of a sum of horizontal gradient values which indicates an absolute value of a sum of a horizontal gradient value at a relative pixel position in a first range and a horizontal gradient value at a relative pixel position in a second range, for each of a plurality of pairs of a relative pixel position in the first range and a relative pixel position in the second range among a plurality of pairs of relative pixel positions, the first range including a first reference block for a current block, the second range including a second reference block for the current block, the plurality of pairs of relative pixel positions being determined commonly and relatively for the first range and the second range;

derives, as a first parameter, a total sum of a plurality of absolute values of sums of horizontal gradient values derived respectively for the plurality of pairs of relative pixel positions;

derives an absolute value of a sum of vertical gradient values which indicates an absolute value of a sum of a vertical gradient value at the relative pixel position in the first range and a vertical gradient value at the relative pixel position in the second range, for each of the plurality of pairs of relative pixel positions;

derives, as a second parameter, a total sum of a plurality of absolute values of sums of vertical gradient values derived respectively for the plurality of pairs of relative pixel positions;

derives, for each of the plurality of pairs of relative pixel positions, a pixel difference value which indicates a difference between the pixel value at the relative pixel position in the first range and the pixel value at the relative pixel position in the second range;

inverts or maintains a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of the sum of the horizontal gradient values which indicates the sum of the horizontal gradient value at the relative pixel position in the first range and the horizontal gradient value at the relative pixel position in the second range, and derives a horizontal-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the horizontal-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the horizontal gradient values;

derives, as a third parameter, a total sum of a plurality of horizontal-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions;

inverts or maintains a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of a sum of vertical gradient values which indicates a sum of the vertical gradient value at the relative pixel position in the first range and the vertical gradient value at the relative pixel position in the second range, and derives a vertical-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the vertical-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the vertical gradient values;

derives, as a fourth parameter, a total sum of a plurality of vertical-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions; and

generates a prediction image to be used to decode the current block using the first parameter, the second parameter, the third parameter, and the fourth parameter,

wherein the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4.

3. An encoding method, comprising:

deriving an absolute value of a sum of horizontal gradient values which indicates an absolute value of a sum of a horizontal gradient value at a relative pixel position in a first range and a horizontal gradient value at a relative pixel position in a second range, for each of a plurality of pairs of a relative pixel position in the first range and a relative pixel position in the second range among a plurality of pairs of relative pixel positions, the first range including a first reference block for a current block, the second range including a second reference block for the current block, the plurality of pairs of relative pixel positions being determined commonly and relatively for the first range and the second range;

deriving, as a first parameter, a total sum of a plurality of absolute values of sums of horizontal gradient values derived respectively for the plurality of pairs of relative pixel positions;

deriving an absolute value of a sum of vertical gradient values which indicates an absolute value of a sum of a vertical gradient value at the relative pixel position in the first range and a vertical gradient value at the relative pixel position in the second range, for each of the plurality of pairs of relative pixel positions;

deriving, as a second parameter, a total sum of a plurality of absolute values of sums of vertical gradient values derived respectively for the plurality of pairs of relative pixel positions;

deriving, for each of the plurality of pairs of relative pixel positions, a pixel difference value which indicates a difference between the pixel value at the relative pixel position in the first range and the pixel value at the relative pixel position in the second range;

inverting or maintaining a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of the sum of the horizontal gradient values which indicates the sum of the horizontal gradient value at the relative pixel position in the first range and the horizontal gradient value at the relative pixel position in the second range, and deriving a horizontal-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the horizontal-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the horizontal gradient values;

deriving, as a third parameter, a total sum of a plurality of horizontal-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions;

inverting or maintaining a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of a sum of vertical gradient values which indicates a sum of the vertical gradient value at the relative pixel position in the first range and the vertical gradient value at the relative pixel position in the second range, and deriving a vertical-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the vertical-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the vertical gradient values;

deriving, as a fourth parameter, a total sum of a plurality of vertical-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions; and

generating a prediction image to be used to encode the current block using the first parameter, the second parameter, the third parameter, and the fourth parameter,

wherein the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4.

4. A decoding method, comprising:

deriving an absolute value of a sum of horizontal gradient values which indicates an absolute value of a sum of a horizontal gradient value at a relative pixel position in a first range and a horizontal gradient value at a relative pixel position in a second range, for each of a plurality of pairs of a relative pixel position in the first range and a relative pixel position in the second range among a plurality of pairs of relative pixel positions, the first range including a first reference block for a current block, the second range including a second reference block for the current block, the plurality of pairs of relative pixel positions being determined commonly and relatively for the first range and the second range;

deriving, as a first parameter, a total sum of a plurality of absolute values of sums of horizontal gradient values derived respectively for the plurality of pairs of relative pixel positions;

deriving an absolute value of a sum of vertical gradient values which indicates an absolute value of a sum of a vertical gradient value at the relative pixel position in the first range and a vertical gradient value at the relative pixel position in the second range, for each of the plurality of pairs of relative pixel positions;

deriving, as a second parameter, a total sum of a plurality of absolute values of sums of vertical gradient values derived respectively for the plurality of pairs of relative pixel positions;

deriving, for each of the plurality of pairs of relative pixel positions, a pixel difference value which indicates a difference between the pixel value at the relative pixel position in the first range and the pixel value at the relative pixel position in the second range;

inverting or maintaining a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of the sum of the horizontal gradient values which indicates the sum of the horizontal gradient value at the relative pixel position in the first range and the horizontal gradient value at the relative pixel position in the second range, and deriving a horizontal-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the horizontal-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the horizontal gradient values;

deriving, as a third parameter, a total sum of a plurality of horizontal-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions;

inverting or maintaining a plus or minus sign of the pixel difference value derived for each of the plurality of pairs of relative pixel positions, according to a plus or minus sign of a sum of vertical gradient values which indicates a sum of the vertical gradient value at the relative pixel position in the first range and the vertical gradient value at the relative pixel position in the second range, and deriving a vertical-related pixel difference value which indicates a pixel difference value derived for the pair of relative pixel positions, the vertical-related pixel difference value having the plus or minus sign inverted or maintained according to the plus or minus sign of the sum of the vertical gradient values;

deriving, as a fourth parameter, a total sum of a plurality of vertical-related pixel difference values derived respectively for the plurality of pairs of relative pixel positions; and

generating a prediction image to be used to decode the current block using the first parameter, the second parameter, the third parameter, and the fourth parameter,

wherein the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4.

5. An encoder comprising:

a memory; and

a processor connected to the memory and configured to:

generate a prediction image based on a first parameter, a second parameter, a third parameter, and a fourth parameter that are derived in a bi-directional optical flow (BDOF) process; and

encode a current block using the generated prediction image,

wherein

the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4;

the first parameter is derived based on the following expression:

Σ [i,j]∈Ω abs( I x 1 +I x 0 );  [Math. 1]

the second parameter is derived based on the following expression:

Σ [i,j]∈Ω abs( I y 1 +I y 0 );  [Math. 2]

the third parameter is derived based on the following expression:

Σ [i,j]∈Ω (sign( I x 1 +I x 0 )×( I 1 −I 0 )); and  [Math. 3]

the fourth parameter is derived based on the following expression:

Σ [i,j]∈Ω (sign( I y 1 +I y 0 )×( I 1 −I 0 )),  [Math. 4]

where:

Ω indicates a set of relative pixel positions;

[i, j] indicates a relative pixel position in the set Ω defined by a horizontal position i and a vertical position j;

I x 0 indicates a horizontal gradient value at a first pixel position in a first horizontal gradient image, I x 1 indicates a horizontal gradient value at the first pixel position in a second horizontal gradient image, I y 0 indicates a vertical gradient value at the first pixel position in a first vertical gradient image, and I y 1 indicates a vertical gradient value at the first pixel position in a second vertical gradient image, the first pixel position being determined based on a relative pixel position [i, j], the first horizontal gradient image, the second horizontal gradient image, the first vertical gradient image, and the second vertical gradient image corresponding to the current block;

I 0 indicates a pixel value at the first pixel position in a first interpolated image corresponding to the current block;

I 1 indicates a pixel value at the first pixel position in a second interpolated image corresponding to the current block;

the abs function outputs an absolute value of an argument; and

the sign function outputs a sign of an argument.

6. A decoder comprising:

a memory; and

a processor connected to the memory and configured to:

generate a prediction image based on a first parameter, a second parameter, a third parameter, and a fourth parameter that are derived in a bi-directional optical flow (BDOF) process; and

decode a current block using the generated prediction image,

wherein

the first parameter, the second parameter, the third parameter, and the fourth parameter are derived on a subblock-by-subblock basis, the current block being divided into subblocks, and a size of each of the subblocks being 4×4;

the first parameter is derived based on the following expression:

Σ [i,j]∈Ω abs( I x 1 +I x 0 );  [Math. 5]

the second parameter is derived based on the following expression:

Σ [i,j]∈Ω abs( I y 1 +I y 0 );  [Math. 6]

the third parameter is derived based on the following expression:

Σ [i,j]∈Ω (sign( I x 1 +I x 0 )×( I 1 −I 0 )); and  [Math. 7]

the fourth parameter is derived based on the following expression:

Σ [i,j]∈Ω (sign( I y 1 +I y 0 )×( I 1 −I 0 )),  [Math. 8]

where:

Ω indicates a set of relative pixel positions;

[i, j] indicates a relative pixel position in the set Ω defined by a horizontal position i and a vertical position j;

I x 0 indicates a horizontal gradient value at a first pixel position in a first horizontal gradient image, I x 1 indicates a horizontal gradient value at the first pixel position in a second horizontal gradient image, I y 0 indicates a vertical gradient value at the first pixel position in a first vertical gradient image, and I y 1 indicates a vertical gradient value at the first pixel position in a second vertical gradient image, the first pixel position being determined based on a relative pixel position [i, j], the first horizontal gradient image, the second horizontal gradient image, the first vertical gradient image, and the second vertical gradient image corresponding to the current block;

I 0 indicates a pixel value at the first pixel position in a first interpolated image corresponding to the current block;

I 1 indicates a pixel value at the first pixel position in a second interpolated image corresponding to the current block;

the abs function outputs an absolute value of an argument; and

the sign function outputs a sign of an argument.