Polynomial fitting for motion compensation and luminance reconstruction in texture synthesis

1. An apparatus for encoding a video picture employing a texture synthesis, the apparatus comprising:

processing circuitry configured to:

identify a texture region within a video picture and a texture patch for the texture region, the texture region including a plurality of picture samples;

determine a first set of parameters specifying:

luminance within the texture region by fitting the plurality of picture samples to a two-dimensional polynomial function of the texture patch determined according to the first set of parameters; and

code the texture patch and the first set of parameters into a bitstream.

2. The apparatus according to claim 1 , wherein the processing circuitry is further configured to:

determine additional parameters including:

a second set of parameters specifying luminance within the texture region by fitting the picture samples to a second two-dimensional polynomial function of the texture patch determined according to the first set of parameters; and

a third set of parameters specifying motion within the texture region by fitting motion estimated between the texture region of the video picture and an adjacent picture to a third two-dimensional polynomial function; and

code the texture patch and the second and third sets of parameters into the bitstream.

3. The apparatus according to claim 1 , wherein the two-dimensional polynomial function has order one or two in each of the two dimensions.

4. The apparatus according to claim 1 , wherein:

the motion is estimated by calculating an optical flow between the texture region in a first video picture and the texture region in a second video picture preceding the first video picture; and

the first set of parameters is determined by fitting the optical flow to the two-dimensional polynomial function.

5. The apparatus according to claim 4 , wherein calculating the optical flow is performed by a function which penalizes higher distances between components of the optical flow and/or higher differences between corresponding samples of the first and the second video picture.

6. The apparatus according to claim 5 , wherein calculating the optical flow is performed by optimization of the function:

E

⁡

(

u

,

v

)

=

∑

i

,

j

⁢

{

ρ

D

⁡

(

I

1

⁡

(

i

,

j

)

-

I

2

⁡

(

i

+

u

i

,

j

,

j

+

v

i

,

j

)

)

+

λ

⁡

[

ρ

S

⁡

(

u

i

,

j

-

u

i

+

1

,

j

)

+

ρ

S

⁡

(

u

i

,

j

-

u

i

,

j

+

1

)

+

ρ

S

⁡

(

v

i

,

j

-

v

i

+

1

,

j

)

+

ρ

S

⁡

(

v

i

,

j

-

v

i

,

j

+

1

)

]

}

,

wherein u and v are horizontal and vertical components of optical flow fields that are calculated from input images I 1 and I 2 , ρ D is a data penalty function, ρ S is a spatial penalty function, and λ is a regularization parameter.

7. The apparatus according to claim 1 , wherein the processing circuitry is further configured to:

determine a further set of parameters specifying weighting factors for reconstructing spectral coefficients of the texture region including fitting the texture region in a spectral domain to a first function of the texture patch, the first function being determined according to the further set of parameters.

8. The apparatus according to claim 7 , wherein the processing circuitry is further configured to:

transform one or more blocks of the texture region into the spectral domain;

transform the texture patch into the spectral domain;

find, for the one or more transformed blocks, respective frequency damping parameters by approximating each transformed block with the transformed texture patch damped with the respective damping parameter; and

code the further set of parameters by fitting the respective frequency damping parameters to a block map function.

9. The apparatus according to claim 8 , wherein the respective frequency damping parameters are scalar values.

10. The apparatus according to claim 8 , wherein the block map function is a further two-dimensional polynomial.

11. The apparatus according to claim 10 , wherein the further two-dimensional polynomial has order one or two in each of two dimensions, the two dimensions being vertical and horizontal.

12. The apparatus according to claim 8 , wherein a transformation used for transforming the texture region and/or the patch into the spectral domain is a block-wise discrete cosine transformation.

13. The apparatus according to claim 1 , wherein the processing circuitry is further configured to:

apply a suppression of block artifacts between blocks of the texture region and blocks of a remaining region, the suppression being performed by:

(i) calculating a distance matrix between luminance values of an overlapped block of the texture region and a block of the remaining region,

(ii) calculating a shortest path in the distance matrix, and

(iii) combining the block of the texture region and the block of the remaining region along the calculated shortest path.

14. The apparatus according to claim 1 , wherein the processing circuitry is further configured to:

detect the texture region within the video picture using clustering,

generate texture region information indicating a location of the texture region within the video picture, and

insert the texture region information into the bitstream.

15. An apparatus for decoding a video picture employing a texture synthesis, the apparatus comprising:

processing circuitry configured to:

decode, from a bitstream, a texture patch and a first set of parameters;

reconstruct a texture region within a video picture from the texture patch, the texture region including a plurality of picture samples,

wherein the processing circuitry is configured to reconstruct the texture region by:

calculating a function of texture region luminance, the function being a first two-dimensional polynomial determined according to one or more parameters from a second set of parameters.

16. The apparatus according to claim 15 , wherein the processing circuitry is configured to reconstruct the texture region by further weighting spectral coefficients of the texture region with a weighting function determined according to one or more parameters of the first set of parameters.

17. The apparatus according to claim 16 , wherein the processing circuitry is further configured to:

decode damping parameters for respective blocks of the texture region according to a block map function determined according to the first set of parameters;

reconstruct the respective blocks including applying the respective damping parameters to the texture patch in a spectral domain; and

transform the reconstructed blocks from the spectral domain to a spatial domain.

18. A method for encoding a video picture employing a texture synthesis, the method comprising:

identifying a texture region within a video picture and a texture patch for the texture region, the texture region including a plurality of picture samples;

determining a first set of parameters specifying:

luminance within the texture region by fitting the plurality of picture samples to a two-dimensional polynomial function of the texture patch determined according to the first set of parameters; and

coding the texture patch and the first set of parameters into a bitstream.

19. A method for decoding a video picture employing a texture synthesis, the method comprising:

decoding, from a bitstream, a texture patch and a first set of parameters;

reconstructing a texture region within a video picture from the texture patch, the texture region including a plurality of picture samples, the reconstructing the texture region including:

calculating a function of texture region luminance, the function being a first two-dimensional polynomial determined according to one or more parameters from a second set of parameters.