Method and device for video image processing, calculating the similarity between video frames, and acquiring a synthesized frame by synthesizing a plurality of contiguous sampled frames
View Patent ↗To acquire a high-resolution frame from a plurality of frames sampled from a video image, it is necessary to obtain a high-resolution frame with reduced picture quality degradation regardless of motion of a subject included in the frame. Because of this, between a plurality of contiguous frames Fr N and Fr N+1 , there is estimated a correspondent relationship. Based on the correspondent relationship, the frames Fr N+1 and Fr N are interposed to obtain first and second interpolated frames Fr H1 and Fr H2 . Based on the correspondent relationship, the coordinates of the frame Fr N+1 are transformed, and from a correlation value with the frame Fr N , there is obtained a weighting coefficient α(x°, y°) that makes the weight of the first interpolated frame Fr H1 greater as a correlation becomes greater. With the weighting coefficient, the first and second interpolated frames are weighted and added to acquire a synthesized frame Fr G .
1. A method of acquiring a processed frame by performing image processing on a desired frame sampled from a video image, said method comprising the steps of:
computing a similarity between said desired frame and at least one frame which is temporally before and one frame which is temporally after said desired frame; and
acquiring said processed frame by obtaining a weighting coefficient whose value increases or decreases in correspondence to said similarity, then weighting said at least one frame with said weighting coefficient, and synthesizing said weighted frame and said desired frame; wherein
said desired frame is partitioned into a plurality of areas; and wherein
motion is computed for each area of said plurality of areas;
said areas are grouped into a plurality of subject areas based on said motion of each area of said plurality of areas;
said similarity is computed for each of corresponding subject areas in said at least one frame which correspond to said plurality of subject areas; and
said processed frame is acquired by obtaining weighting coefficients whose values increase or decrease in correspondence to a reference level of the similarity, then weighting said corresponding subject areas of said at least one frame with said weighting coefficients, and synthesizing said weighted subject areas and said plurality of subject areas.
2. An image processor for acquiring a processed frame by performing image processing on a desired frame sampled from a video image, said image processor comprising:
similarity computation means for computing a similarity between said desired frame and at least one frame which is temporally before or after said desired frame; and
synthesis means for obtaining a weighting coefficient whose value increases or decreases in correspondence to said similarity, then weighting said at least one frame with said weighting coefficient, and synthesizing said weighted frame and said desired frame into said processed frame; wherein
said similarity computation means partitions said desired frame into a plurality of areas; and wherein
a motion computation means computes motion for each area of said plurality of areas;
said similarity computation means groups said areas into a plurality of subject areas based on said motion of each area of said plurality of areas, and computes said similarity for each of corresponding subject areas in said at least one frame which correspond to said plurality of subject areas; and
said synthesis means obtains weighting coefficients whose values increase or decrease in correspondence to said similarity, then weights said corresponding subject areas of said at least one frame with said weighting coefficients, and synthesizes said weighted subject areas and said plurality of subject areas into said processed frame.
3. A computer readable medium storing a computer program which, when executed by a computer processor, causes the computer processor to perform an image processing method of acquiring a processed frame by performing image processing on a desired frame sampled from a video image, the method comprising:
computing a similarity between said desired frame and at least one frame which is temporally before and one frame which is temporally after said desired frame; and
obtaining a weighting coefficient whose value increases or decreases in correspondence to said similarity, then weighting said at least one frame with said weighting coefficient, and synthesizing said weighted frame and said desired frame into said processed frame; wherein
said computing a similarity between said desired frame and at least one frame which is temporally before and one frame which is temporally after said desired frame comprises partitioning said desired frame into a plurality of areas, computing motion for each area of said plurality of areas, grouping said areas into a plurality of subject areas based on said motion of each area of said plurality of areas, and computing said similarity for each of corresponding subject areas in said at least one frame which correspond to said plurality of subject areas; and
said obtaining a weighting coefficient whose value increases or decreases in correspondence to said similarity, then weighting said at least one frame with said weighting coefficient, and synthesizing said weighted frame and said desired frame into said processed frame comprises obtaining weighting coefficients whose values increase or decrease in correspondence to said similarity, then weighting said corresponding subject areas of said at least one frame with said weighting coefficients, and synthesizing said weighted subject areas and said plurality of subject areas into said processed frame.
4. The image processor as set forth in claim 2 , wherein
said similarity computation means computes said similarity between said desired frame and at least one frame which is temporally before and one frame which is temporally after said desired frame.
5. The synthesis method as set forth in claim 1 , wherein
a magnitude of said motion for each area of said plurality of areas is calculated; and
said plurality of subject areas comprises a first subject area including areas of said plurality of areas having a motion magnitude that has increased relative to said frame which is temporally before said desired frame, and a second subject area including areas of said plurality of areas having a motion magnitude that has decreased relative to said frame which is temporally before said desired frame.
6. The image processor as set forth in claim 2 , wherein
the motion computation means computes a magnitude of said motion for each area of said plurality of areas; and
said plurality of subject areas comprises a first subject area including areas of said plurality of areas having a motion magnitude that has increased relative to said frame which is temporally before said desired frame, and a second subject area including areas of said plurality of areas having a motion magnitude that has decreased relative to said frame which is temporally before said desired frame.
7. The computer readable medium as set forth in claim 3 , further comprising computing a magnitude of said motion for each area of said plurality of areas; wherein
said plurality of subject areas comprises a first subject area including areas of said plurality of areas having a motion magnitude that has increased relative to said frame which is temporally before said desired frame, and a second subject area including areas of said plurality of areas having a motion magnitude that has decreased relative to said frame which is temporally before said desired frame.