Affine prediction using an epipolar-based constraint for pictures of differing resolutions

1 . A method of encoding video data, the method comprising:

receiving a first picture having a first resolution;

receiving a second picture having a second resolution, wherein the second resolution is different than the first resolution, and wherein the first picture and the second picture have at least partially overlapping fields of view; and

encoding a current block of the first picture, relative to the second picture, using affine prediction with a single parameter search, wherein the single parameter search is performed along an epipolar curve and uses a plurality of one-dimensional displacements along the epipolar curve, wherein the single parameter search is used to determine parameters of an affine prediction model, and wherein encoding the current block of the first picture comprises:

determining the plurality of one-dimensional displacements along the epipolar curve;

determining a respective prediction block for each of the plurality of one-dimensional displacements;

determining a best prediction block from among the respective prediction blocks;

determining the parameters of the affine prediction model based on the best prediction block;

calculating a residual block using the current block of the first picture and the best prediction block;

encoding the residual block in an encoded video bitstream; and

encoding one or more syntax elements indicative of the parameters of the affine prediction model in the encoded video bitstream.

2 . The method of claim 1 , wherein determining a best prediction block from among the respective prediction blocks comprises:

minimizing a block prediction error to determine the best prediction block from among the respective prediction blocks.

3 . The method of claim 1 , further comprising:

determining the epipolar curve as a function of a first projection function for a first camera used to capture the first picture, and a second projection function for a second camera used to capture the second picture.

4 . The method of claim 3 , wherein the function is l(δ)=(xf,yf)=(xp,yp)+δ(F1,F2),

wherein l( ) is the epipolar curve, (F1,F2) are components of a fundamental matrix defined by the first projection function and the second projection function, (xp,yp) is a point in the first picture, (xf,yf) is a point in the second picture, and δ is the single parameter along the epipolar curve.

5 . The method of claim 1 , wherein the second picture is distorted relative to the first picture.

6 . The method of claim 5 , further comprising:

capturing the first picture with a pinhole camera; and

capturing the second picture with a fisheye camera.

7 . The method of claim 1 , wherein the affine prediction model is a 4-parameter affine prediction model or a 6-parameter affine model.

8 . An apparatus configured to encode video data, the apparatus comprising:

a memory; and

processing circuitry in communication with the memory, the processing circuitry configured to:

receive a first picture having a first resolution;

receive a second picture having a second resolution, wherein the second resolution is different than the first resolution, and wherein the first picture and the second picture have at least partially overlapping fields of view; and

encode a current block of the first picture, relative to the second picture, using affine prediction with a single parameter search, wherein the single parameter search is performed along an epipolar curve and uses a plurality of one-dimensional displacements along the epipolar curve, wherein the single parameter search is used to determine parameters of an affine prediction model, and wherein to encode the current block of the first picture, the processing circuitry is further configured to:

determine the plurality of one-dimensional displacements along the epipolar curve;

determine a respective prediction block for each of the plurality of one-dimensional displacements;

determine a best prediction block from among the respective prediction blocks;

determine the parameters of the affine prediction model based on the best prediction block;

calculate a residual block using the current block of the first picture and the best prediction block;

encode the residual block in an encoded video bitstream; and

encode one or more syntax elements indicative of the parameters of the affine prediction model in the encoded video bitstream.

9 . The apparatus of claim 8 , wherein to determine a best prediction block from among the respective prediction blocks, the processing circuitry is further configured to:

minimize a block prediction error to determine the best prediction block from among the respective prediction blocks.

10 . The apparatus of claim 8 , wherein the processing circuitry is further configured to:

determine the epipolar curve as a function of a first projection function for a first camera used to capture the first picture, and a second projection function for a second camera used to capture the second picture.

11 . The apparatus of claim 10 , wherein the function is l(δ)=(xf,yf)=(xp,yp)+δ(F1,F2), wherein l( ) is the epipolar curve, (F1,F2) are components of a fundamental matrix defined by the first projection function and the second projection function, (xp,yp) is a point in the first picture, (xf,yf) is a point in the second picture, and δ is the single parameter along the epipolar curve.

12 . The apparatus of claim 8 , wherein the second picture is distorted relative to the first picture.

13 . The apparatus of claim 12 , wherein the processing circuitry is further configured to:

capture the first picture with a pinhole camera; and

capture the second picture with a fisheye camera.

14 . The apparatus of claim 8 , wherein the affine prediction model is a 4-parameter affine prediction model or a 6-parameter affine model.

15 . The apparatus of claim 8 , wherein the apparatus is part of a vehicle, and wherein the vehicle includes a first camera used to capture the first picture and as second camera used to capture the second picture.

16 . A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to decode video data to:

receive a first picture having a first resolution;

receive a second picture having a second resolution, wherein the second resolution is different than the first resolution, and wherein the first picture and the second picture have at least partially overlapping fields of view; and

encode a current block of the first picture, relative to the second picture, using affine prediction with a single parameter search, wherein the single parameter search is performed along an epipolar curve and uses a plurality of one-dimensional displacements along the epipolar curve, wherein the single parameter search is used to determine parameters of an affine prediction model, and wherein to encode the current block of the first picture, the processing circuitry is further configured to:

determine the plurality of one-dimensional displacements along the epipolar curve;

determine a respective prediction block for each of the plurality of one-dimensional displacements;

determine a best prediction block from among the respective prediction blocks;

determine the parameters of the affine prediction model based on the best prediction block;

calculate a residual block using the current block of the first picture and the best prediction block;

encode the residual block in an encoded video bitstream; and

encode one or more syntax elements indicative of the parameters of the affine prediction model in the encoded video bitstream.