Method and apparatus for processing occlusions in motion estimation
View Patent ↗A method and an apparatus for processing occlusions in motion estimation are described. For processing an occlusion in a forward motion field of a second frame relative to a first frame the forward motion field of the second frame relative to the first frame is determined with a motion estimator. Then an area in the second frame that is occluded in the first frame is identified with an occlusion identifier. Finally, a motion corrector determines a corrected forward motion field by filling the forward motion field in the identified area in the second frame using motion vectors of a previous backward motion field of the second frame relative to a previous third frame.
1. A method for processing an occlusion in a forward motion field of a second frame relative to a first frame, the method comprising:
determining the forward motion field of the second frame relative to the first frame;
identifying an area in the second frame that is occluded in the first frame;
determining a corrected forward motion field of the second frame relative to the first frame by filling the forward motion field in the identified area in the second frame using motion vectors of a previous backward motion field of the second frame relative to a previous third frame; and
applying a filter to the filled motion vectors, the filter being configured to consider a spatial neighborhood of a first map to account for spatial variation of the motion fields and a spatial neighborhood in a facing map, and to discard data of the occlusion to cope with acceleration.
2. The method according to claim 1 , further comprising:
determining a backward motion field of the first frame relative to the second frame; and
storing the determined backward motion field in a memory.
3. The method according to claim 1 , further comprising determining a corrected previous backward motion field by filling the previous backward motion field in areas of the second frame that are occluded in the third frame using motion vectors of the forward motion field of the second frame.
4. The method according to claim 1 , further comprising:
determining an acceleration between the previous backward motion field and the forward motion field; and
taking the determined acceleration into account when filling the forward motion field or the previous backward motion field.
5. The method according to claim 1 , wherein the filter is an interpolation filter.
6. The method according to claim 5 , wherein the interpolation filter is a bilateral filter.
7. The method according to claim 1 , wherein the third frame, the second frame, and the first frame are temporally successive frames.
8. The method according to claim 3 , wherein the third frame, the second frame, and the first frame are temporally non-successive frames, and wherein the relative temporal distances between the frames are taken into account when filling the forward motion field or the previous backward motion field.
9. An apparatus for processing an occlusion in a forward motion field of a second frame relative to a first frame, the apparatus comprising:
a motion estimator configured to determine the forward motion field of the second frame relative to the first frame;
an occlusion identifier configured to identify an area in the first frame that is occluded in the second frame;
a motion corrector configured to determine a corrected forward motion field of the second frame relative to the first frame by filling the forward motion field in the identified area in the second frame using motion vectors of a previous backward motion field of the second frame relative to a previous third frame; and
a filter configured to filter the filled motion vectors, the filter being further configured to consider a spatial neighborhood of a first map to account for spatial variation of the motion fields and a spatial neighborhood in a facing map, and to discard data of the occlusion to cope with acceleration.
10. The apparatus according to claim 9 , wherein the motion estimator is further configured to determine a backward motion field of the first frame relative to the second frame.
11. The apparatus according to claim 9 , further comprising a memory configured to store the determined backward motion field.
12. The apparatus according to claim 9 , wherein the motion corrector is further configured to determine a corrected previous backward motion field by filling the previous backward motion field in areas of the second frame that are occluded in the third frame using motion vectors of the forward motion field of the second frame.
13. The apparatus according to claim 12 , wherein the motion corrector is further configured to determine an acceleration between the previous backward motion field and the forward motion field and to take the determined acceleration into account when filling the forward motion field or the previous backward motion field.
14. The apparatus according to claim 9 , wherein the filter is an interpolation filter configured to interpolate the filled motion vectors.
15. The apparatus according to claim 14 , wherein the interpolation filter is a bilateral filter.
16. The apparatus according to claim 9 , wherein the third frame, the second frame, and the first frame are temporally successive frames.
17. The apparatus according to claim 12 , wherein the third frame, the second frame, and the first frame are temporally non-successive frames, and wherein the motion corrector is configured to take the relative temporal distances between the frames into account when filling the forward motion field or the previous backward motion field.