Method and apparatus for reconstructing image
View Patent ↗The present invention relates to the field of video data de-interlacing, and in particular to a method for reconstructing full-resolution frames from a line-skipped-sequence of fields and a corresponding apparatus. It is the particular approach of the present invention to substitute missing lines of a block of a reconstructed full-resolution frame by lines from another field, e.g. the preceding field, and translating the substitute lines vertically and horizontally so as to optimized a smoothness measure computed for the thus reconstructed block. In this manner, an error-prone a priori determination of motion vectors based an interpolation of the interlaced images in the vertical direction can be avoided. The present invention may also be applied to sequences generated from a full-resolution sequence by a line-skipping operation that keeps only every K th line and discards the other K−1 lines.
1. A method for reconstructing full-resolution frames from a line-skipped-sequence of fields, the method comprising:
receiving a target field and at least one reference field of the line-skipped sequence;
up-sampling a target block of the target field in the vertical direction by merging the lines of the target block and the lines of a reference block of the at least one reference field;
determining a smoothness value indicating smoothness of pixel data of the up-sampled target block; and
determining a position of the reference block within the at least one reference field in accordance with the determined smoothness value so as to optimize the smoothness of the up-sampled target block,
wherein in the determining of the position, the position is determined so as to optimize the smoothness value, and
in the up-sampling in the vertical direction, the merging is performed using the reference block at the position determined in the determining of the position so as to optimize the smoothness value.
2. The method according to claim 1 , further comprising
dividing the target field and the at least one reference field into a plurality of blocks, each block consisting of a plurality of pixels,
wherein each of the up-sampling in the vertical direction, the determining of the smoothness value, and the determining of the position is performed on the each block generated in the dividing.
3. The method according to claim 1 ,
wherein each field of the line-skipped sequence of fields has a vertical resolution that is K-fold lower than the vertical resolution of the full-resolution frames, K being an integer larger than 1;
wherein in the up-sampling in the vertical direction, the target field and the at least one reference field represent a plurality of K fields of the line-skipped sequence of fields; and
wherein the target block is up-sampled by merging the lines of the target block with the lines of K−1 reference blocks, each reference block being taken from a different one of the K−1 reference frames.
4. The method according to claim 3 ,
wherein K equals 2 and the line-skipped sequence of fields represents an interlaced video signal.
5. The method according to claim 3 ,
wherein K equals to 3.
6. The method according to claim 1 ,
wherein the target field and the at least one reference field represent a plurality of K consecutive fields of the line-skipped sequence of fields.
7. The method according to claim 1 , further comprising
assembling a full-resolution frame from a plurality of up-sampled target blocks.
8. The method according to claim 1 , further comprising
up-sampling the target field and the at least one reference field in the horizontal direction by performing horizontal interpolation.
9. The method according to claim 8 ,
wherein in the up-sampling in the horizontal direction, the target field and the at least one reference field are up-sampled in the horizontal direction by a factor of 2.
10. The method according to claim 8 ,
wherein in the up-sampling in the horizontal direction, the target field and the at least one reference field are up-sampled in the horizontal direction by a factor of 4.
11. The method according to claim 1 ,
wherein the determining of the smoothness value includes
evaluating a predetermined objective function quantifying smoothness of a block of pixel data, in order to determine the smoothness value.
12. The method according to claim 1 ,
wherein the determining of the smoothness value includes
applying one of a discrete Fourier transformation, a discrete cosine transformation, or a wavelet transformation to the up-sampled target block and computing the power of transform coefficients representing high frequency components of the up-sampled target block, in order to determine the smoothness value.
13. The method according to claim 1 ,
wherein the determining of the smoothness value includes
computing pixel differences of neighboring pixels of the up-sampled target block, in order to determine the smoothness value.
14. The method according to claim 1 ,
wherein the determining of the smoothness value includes
computing a sum of terms containing pixel differences of neighboring pixels of the up-sampled target block, in order to determine the smoothness value.
15. The method according to claim 1 ,
wherein the determining of the position includes
shifting the reference block of the at least one reference field vertically and/or horizontally, in order to determine the position of the reference block so as to optimize the smoothness value.
16. The method according to claim 15 ,
wherein the shifting is performed in accordance with the determined smoothness value so as to optimize the smoothness of the up-sampled target block.
17. The method according to claim 1 , further comprising
selecting the reference fields from the line-skipped sequence of fields,
wherein each of the up-sampling in the vertical direction and the determining of the position is performed on at least one of the reference fields selected in the selecting as the at least one reference field of the line-skipped sequence.
18. The method according to claim 17 ,
wherein the selecting is performed in accordance with the determined smoothness value so as to optimize the smoothness of the up-sampled target block.
19. The method according to claim 1 , further comprising
determining a sequence in which lines of the reference fields are merged into the up-sampled target block.
20. The method according to claim 19 ,
wherein the determining of the sequence is performed in accordance with the determined smoothness value so as to optimize the smoothness of the up-sampled target block.
21. A computer program product comprising a non-transitory computer readable medium having computer readable program code embodied thereon, the program code being adapted to carry out all operations of claim 1 .
22. An apparatus for reconstructing full-resolution frames from a line-skipped-sequence of fields, the apparatus comprising:
a receiving unit operable to receive a target field and at least one reference field of the line-skipped sequence;
a first up-sampling unit operable to up-sample a target block of the target field in the vertical direction by merging the lines of the target block and the lines of a reference block of the at least one reference field;
a smoothness determining unit operable to determine a smoothness value indicating smoothness of pixel data of the up-sampled target block; and
a position determining unit operable to determine a position of the reference block within the at least one reference field in accordance with the determined smoothness value so as to optimize the smoothness of the up-sampled target block,
wherein the position determining unit determines the position so as to optimize the smoothness value, and
the first up-sampling unit performs the merging using the reference block at the position determined by the position determining means so as to optimize the smoothness value.