Multi-view video coding artifact reduction method and system

1 . A multi-view video coding artifact reduction method comprising:

selecting reference fames in neighbors of a current frame constituting a multi-view video;

deriving unidirectional flows and bilateral flows between the current frame and the reference frames;

warping the reference frames based on the derived flows;

generating intermediate frames by fusing the current frame, the reference frames, and the warped reference frames; and

generating a final frame by fusing the current frame and the intermediate frames,

wherein the reference frames comprise:

first reference frames which are positioned in temporal neighbors with reference to the current frame;

second reference frames which are positioned in spatial neighbors in an x-axis direction with reference to the current frame; and

third reference frames which are positioned in spatial neighbors in a y-axis direction with reference to the current frame, and

wherein the generating the intermediate frames comprises:

generating first intermediate frames by fusing the current frame, the first reference frames, and warped first reference frames;

generating second intermediate frames by fusing the current frame, the second reference frames and warped second reference frames; and

generating third intermediate frames by fusing the current frame, the third reference frames and warped third reference frames.

2 . The multi-view video coding artifact reduction method of claim 1 , wherein the deriving comprises:

extracting a current feature vector and reference feature vectors from the current frame and the reference frames;

predicting unidirectional warping vectors from the extracted reference feature vectors through unidirectional motion estimation; and

predicting bilateral warping vectors from the extracted reference feature vectors through bilateral motion estimation.

3 . The multi-view video coding artifact reduction method of claim 2 , wherein the deriving comprises further deriving a refinement flow from the unidirectional flow and the bilateral flow.

4 . The multi-view video coding artifact reduction method of claim 3 , wherein the deriving further comprises a calculating refinement warping vectors by using the predicted bilateral warping vectors and unidirectional warping vectors.

5 . The multi-view video coding artifact reduction method of claim 4 , wherein the calculating comprises calculating unidirectional warping vectors of intermediate frames which are generated by the bilateral warping vectors as refinement warping vectors.

6 . The multi-view video coding artifact reduction method of claim 5 , wherein the generating the intermediate frames comprises:

generating candidate reference feature vectors by applying corresponding warping vectors to the reference feature vectors; and

generating the intermediate frames by fusing the current feature vector with the reference feature vectors and the candidate reference feature vectors.

7 . The multi-view video coding artifact reduction method of claim 1 , wherein the generating the final frame comprises generating the final frame by fusing the current frame with the first intermediate frame, the second intermediate frame, and the third intermediate frame.

8 . A multi-view video coding artifact reduction system comprising:

a processor configured to: select reference fames in neighbors of a current frame constituting a multi-view video; derive unidirectional flows and bilateral flows between the current frame and the reference frames; warp the reference frames based on the derived flows; generate intermediate frames by fusing the current frame, the reference frames, and the warped reference frames; and generate a final frame by fusing the current frame and the intermediate frames; and

a storage unit configured to provide a storage space necessary for functions and operations of the processor,

wherein the reference frames comprise:

first reference frames which are positioned in temporal neighbors with reference to the current frame;

second reference frames which are positioned in spatial neighbors in an x-axis direction with reference to the current frame; and

third reference frames which are positioned in spatial neighbors in a y-axis direction with reference to the current frame, and

wherein, for the generating the intermediate frames, the processor is configured to:

generate first intermediate frames by fusing the current frame, the first reference frames, and warped first reference frames;

generate second intermediate frames by fusing the current frame, the second reference frames and warped second reference frames; and

generate third intermediate frames by fusing the current frame, the third reference frames and warped third reference frames.

9 . The system of claim 8 , wherein, for the deriving, the processor is configured to:

extract a current feature vector and reference feature vectors from the current frame and the reference frames;

predict unidirectional warping vectors from the extracted reference feature vectors through unidirectional motion estimation; and

predict bilateral warping vectors from the extracted reference feature vectors through bilateral motion estimation.

10 . The system of claim 9 , wherein, for the deriving, the processor is further configured to derive a refinement flow from the unidirectional flow and the bilateral flow.

11 . The system of claim 10 , wherein, for the deriving, the processor is further configured to calculate refinement warping vectors by using the predicted bilateral warping vectors and unidirectional warping vectors.

12 . The system of claim 11 , wherein, for the calculating, the processor is configured to calculate unidirectional warping vectors of intermediate frames which are generated by the bilateral warping vectors as refinement warping vectors.

13 . The system of claim 12 , wherein, for the generating the intermediate frames, the processor is configured to:

generate candidate reference feature vectors by applying corresponding warping vectors to the reference feature vectors; and

generate the intermediate frames by fusing the current feature vector with the reference feature vectors and the candidate reference feature vectors.

14 . The system of claim 8 , wherein, for the generating the final frame, the processor is configured to generate the final frame by fusing the current frame with the first intermediate frame, the second intermediate frame, and the third intermediate frame.

15 . A multi-view video coding artifact reduction method including:

deriving unidirectional flows and bilateral flows between a current frame and reference frames which are positioned in neighbors of the current frame;

generating intermediate frames based on the derived flows; and

generating a final frame by fusing the current frame and the intermediate frames,

wherein the reference frames comprise:

first reference frames which are positioned in temporal neighbors with reference to the current frame;

second reference frames which are positioned in spatial neighbors in an x-axis direction with reference to the current frame; and

third reference frames which are positioned in spatial neighbors in a y-axis direction with reference to the current frame, and

wherein the generating the intermediate frames comprises:

generating first intermediate frames by fusing the current frame, the first reference frames, and warped first reference frames;

generating second intermediate frames by fusing the current frame, the second reference frames and warped second reference frames; and

generating third intermediate frames by fusing the current frame, the third reference frames and warped third reference frames.

16 . The multi-view video coding artifact reduction method of claim 15 , wherein the deriving comprises:

extracting a current feature vector and reference feature vectors from the current frame and the reference frames;

predicting unidirectional warping vectors from the extracted reference feature vectors through unidirectional motion estimation; and

predicting bilateral warping vectors from the extracted reference feature vectors through bilateral motion estimation.

17 . The multi-view video coding artifact reduction method of claim 16 , wherein the deriving comprises further deriving a refinement flow from the unidirectional flow and the bilateral flow.

18 . The multi-view video coding artifact reduction method of claim 17 , wherein the deriving further comprises calculating refinement warping vectors by using the predicted bilateral warping vectors and unidirectional warping vectors.

19 . The multi-view video coding artifact reduction method of claim 18 , wherein the calculating comprises calculating unidirectional warping vectors of intermediate frames which are generated by the bilateral warping vectors as refinement warping vectors.

20 . The multi-view video coding artifact reduction method of claim 19 , wherein the generating the intermediate frames comprises:

generating candidate reference feature vectors by applying corresponding warping vectors to the reference feature vectors; and

generating the intermediate frames by fusing the current feature vector with the reference feature vectors and the candidate reference feature vectors.