Changing the temporal sample rate of a motion-image sequence

1. Apparatus for changing the temporal sample rate of a motion-image sequence in which an input image sequence at an input rate is converted to an output image sequence at an output rate, the apparatus comprising;

a temporal interpolator, which serves to provide a measure of temporal interpolation confidence which measure varies in dependence upon the content of the images being interpolated and which is low when objectionable interpolation artefacts are expected;

a buffer associated with the temporal interpolator and having a buffer occupancy which varies between full and empty limits; and

a controller supplied with said measure of temporal interpolation confidence and a measure of said buffer occupancy;

wherein the apparatus has:

an interpolation mode in which the output image sequence is formed by temporal interpolation between images of the input sequence; and

a synchronisation mode in which the output image sequence is formed by re-timing of images of the input sequence without temporal interpolation images, with said buffer occupancy varying accordingly, it being necessary to drop or repeat an image if the buffer occupancy reaches a limit;

the controller being adapted to select the synchronisation mode when the measure of temporal interpolation confidence is low and the measure of buffer occupancy indicates that images can be retimed without dropping or repeating of images and to select the interpolation mode when the measure of temporal interpolation confidence is high or when the measure of buffer occupancy indicates that images cannot be retimed without dropping or repeating of images.

2. Apparatus according to claim 1 , wherein an exchange rate at which images are exchanged between the buffer and the temporal interpolator is variable and wherein the controller is further adapted in the interpolation mode in dependence upon the measure of temporal interpolation confidence and the measure of buffer occupancy, to vary said exchange rate to optimise the buffer occupancy for retiming of images without dropping or repeating of images in a subsequent synchronisation.

3. Apparatus according to claim 1 in which the said temporal interpolation is motion compensated interpolation using motion vectors and in which the said temporal interpolation confidence measure is derived from confidence in the motion vectors.

4. Apparatus according to claim 1 , wherein the measure of interpolation confidence is derived from analysis of the energy spectrum of the input image data.

5. Apparatus according to claim 1 , wherein the measure of interpolation confidence is derived from a measure of interpolation artefacts present in an interpolated image sequence obtained by comparison of an image parameter of the said interpolated image sequence with a corresponding image parameter of an input image sequence and the said image parameters are temporally aligned prior to comparison by temporal interpolation of one or both of the said image parameters.

6. Apparatus according to claim 1 , wherein selection of modes depends further on a control parameter describing the acceptability of variations in the temporal difference between the portrayal of events in the said input image sequence at an input image rate and the portrayal of the said events in the said output image sequence at an output image rate.

7. A temporal interpolation method for converting images of an input image sequence having an input image rate to images of an output image sequence having a different output image rate that operates in a temporal interpolation mode during a first portion of the said sequence and in a synchronisation mode for a second portion of the said sequence, the method using a FIFO buffer connected to a temporal interpolator for the exchange of images between the buffer and the temporal interpolator at a buffer image exchange rate; wherein

during the said first portion the temporal interpolator provides temporally interpolated images that comprise the respective portion of the said output image sequence;

during at least part of the said first portion the buffer image exchange rate is varied with respect to the input or output image rate; and

during said second portion the buffer provides the respective portion of the said output image sequence comprising un-interpolated input frames synchronised to the output image rate.

8. A method according to claim 7 , wherein an output of the FIFO buffer forms an input to the temporal interpolator and wherein during said part of the said first portion the buffer image exchange rate is varied with respect to the input image rate.

9. A method according to claim 8 , wherein during said part of the said first portion the buffer image exchange rate is increased above the input image rate when the output rate is lower than the input rate and decreased below the input image rate output rate is higher than the input rate.

10. A method according to claim 7 , wherein an output of the temporal interpolator forms an input to the FIFO buffer and wherein during said part of the said first portion the buffer image exchange rate is varied with respect to the output image rate.

11. A method according to claim 10 , wherein during said part of the said first portion the buffer image exchange rate is increased above the output image rate when the output rate is higher than the input rate and decreased below the output image rate output rate is lower than the input rate.

12. A non-transitory computer program product adapted to cause data processing apparatus to implement a method of changing the temporal sample rate of a motion-image sequence in which an input image sequence at an input rate is converted to an output image sequence at an output rate, the method including an interpolation mode in which the output image sequence is formed by temporal interpolation between images of the input sequence; and a synchronisation mode in which the output image sequence is formed by re-timing images of the input sequence without temporal interpolation images; the synchronisation mode being selected when temporal interpolation confidence which varies in dependence upon the content of the images being interpolated and which is low when objectionable interpolation artefacts are expected, is low and images can be retimed without dropping or repeating of images and the interpolation mode being selected when the measure of temporal interpolation confidence is high or when images cannot be retimed without dropping or repeating of images.

13. A computer program product according to claim 12 , wherein images are exchanged between the temporal interpolator and a buffer at an exchange rate which is varied in the interpolation mode to optimise the buffer occupancy for retiming of images without dropping or repeating of images in a subsequent synchronisation.

14. A computer program product according to claim 12 , wherein selection of modes depends further on a control parameter describing the acceptability of variations in the temporal difference between the portrayal of events in the said input image sequence at an input image rate and the portrayal of the said events in the said output image sequence at an output image rate.

15. A temporal interpolation system for converting an input image sequence at an input image timing to an output image sequence at a different output image timing, the system comprising:

a temporal interpolator controlled by an interpolation phase measure; an image buffer that is connected to a temporal interpolator; and an interpolation phase generator that derives a changing interpolation phase measure, there being provided a measure of the likelihood of interpolation artefacts, which measure varies in dependence upon the content of the images being interpolated;

from comparison of input image timing and an output image timing;

wherein the rate of change of the interpolation phase of the temporal interpolator is controlled in response to the fill level of the image buffer and said measure of the likelihood of interpolation artefacts.

16. A system according to claim 15 in which the said temporal interpolation is motion compensated interpolation using motion vectors and in which the said measure of the likelihood of interpolation artefacts is derived from confidence in the motion vectors.

17. A system according to claim 15 , wherein said measure of the likelihood of interpolation artefacts is derived from analysis of the energy spectrum of the input image sequence.