Image processing device, image processing method and storage medium
An image processing device includes an image input portion, a motion vector acquisition portion and a distortion component estimation portion. The image input portion inputs a plurality of frame images. The motion vector acquisition portion acquires an amount of parallel movement between two frame images. The distortion component estimation portion performs approximation such that a motion matrix indicating a motion between the frame images consists only of a parallel movement component and a rolling shutter distortion component, uses the amount of parallel movement to calculate, for each of the frame images, a tentative distortion coefficient included in the rolling shutter distortion component and uses the plurality of tentative distortion coefficients calculated to estimate a distortion coefficient.
1. An image processing device that processes a plurality of successive frame images, the image processing device comprising:
an image input portion that inputs the plurality of frame images;
a parallel movement amount acquisition portion that acquires an amount of parallel movement between two frame images among the plurality of frame images; and
a distortion component estimation portion that
performs approximation such that a motion matrix indicating a motion between the frame images consists only of a parallel movement component and a rolling shutter distortion component,
uses the amount of parallel movement to calculate, for each of the frame images, a tentative distortion coefficient included in the rolling shutter distortion component, and
uses the plurality of tentative distortion coefficients calculated to estimate a distortion coefficient.
2. The image processing device according to claim 1 ,
wherein the distortion component estimation portion estimates that an average value or a center value of the plurality of tentative distortion coefficients is the distortion coefficient.
3. The image processing device according to claim 1 ,
wherein the distortion component estimation portion plots the tentative distortion coefficients on a two-dimensional plane having the rolling shutter distortion component and the amount of parallel movement as coordinate axes and estimates the distortion coefficient based on a gradient of a regression line thereof.
4. The image processing device according to claim 1 ,
wherein the distortion component estimation portion performs, when the tentative distortion coefficient of one of the plurality of frame images is greater than a reciprocal of a number of lines of the frame image, a correction such that the tentative distortion coefficient of the one is the reciprocal of the number of lines of the frame image.
5. The image processing device according to claim 1 ,
wherein the distortion component estimation portion estimates, when the tentative distortion coefficient of one of the plurality of frame images is greater than a reciprocal of a number of lines of the frame image, the distortion coefficient without using the tentative distortion coefficient of the one.
6. The image processing device according to claim 1 ,
wherein the distortion component estimation portion performs, when the estimated distortion coefficient of one of the plurality of frame images is greater than a reciprocal of a number of lines of the frame image, a correction such that the estimated distortion coefficient is the reciprocal of the number of lines of the frame image.
7. An image processing method of processing a plurality of successive frame images, the image processing method comprising:
an image input step of inputting the plurality of frame images;
a parallel movement amount acquisition step of acquiring an amount of parallel movement between two frame images among the plurality of frame images; and
a distortion component estimation step of
performing approximation such that a motion matrix indicating a motion between the frame images consists only of a parallel movement component and a rolling shutter distortion component,
using the amount of parallel movement to calculate, for each of the frame images, a tentative distortion coefficient included in the rolling shutter distortion component, and
using the plurality of tentative distortion coefficients calculated to estimate a distortion coefficient.
8. A non-transitory computer readable storage medium in which an image processing program is recorded that operates a computer such that the computer processes a plurality of successive frame images,
wherein the image processing program operates the computer as:
an image input portion that inputs the plurality of frame images;
a parallel movement amount acquisition portion that acquires an amount of parallel movement between two frame images among the plurality of frame images; and
a distortion component estimation portion that
performs approximation such that a motion matrix indicating a motion between the frame images consists only of a parallel movement component and a rolling shutter distortion component,
uses the amount of parallel movement to calculate, for each of the frame images, a tentative distortion coefficient included in the rolling shutter distortion component, and
uses the plurality of tentative distortion coefficients calculated to estimate a distortion coefficient.