Method and system for randomly accessing multiview videos with known prediction dependency
View Patent ↗A method randomly accesses multiview videos. Multiview videos are acquired of a scene with corresponding cameras arranged at poses, such that there is view overlap between any pair of cameras. V-frames are generated from the multiview videos. The V-frames are encoded using only spatial prediction. Then, the V-frames are inserted periodically in an encoded bit stream to provide random temporal access to the multiview videos. Additional view dependency information enables the decoding of a reduced number of frames prior to accessing randomly a target frame for a specified view and time, and decoding the target frame.
1. A method for decoding multiview videos, the multiview videos including frames of multiple views over time, comprising the steps of:
receiving a prediction dependency message associated with a current frame of the multiview videos, in which the prediction dependency message indicates a number of views that are used as reference for a particular view, and a set of view indices that indicate which views are used as the reference for the particular view; and
decoding the current frame of the multiview videos based on the prediction dependency message, wherein the steps are performed in a video decoder.
2. The method of claim 1 in which the prediction dependency message indicates a spatial prediction dependency of frames associated with different views in the multiview videos.
3. The method of claim 1 , in which the prediction dependency message indicates a temporal prediction dependency the frames of the specified view.
4. The method of claim 1 , in which the prediction dependency message is received with every independently encoded frame and with every frame encoded using only spatial reference frames in the multiview videos.
5. The method of claim 1 , in which the prediction dependency message is received with each frame in the multiview videos.
6. The method of claim 1 , in which the prediction dependency message is received for all frames in the multiview videos except frames that are not used as a temporal reference.
7. The method of claim 1 , in which the prediction dependency message is in a form of a supplemental enhancement information message as defined by a H.264/AVC standard specification.
8. The method of claim 7 , in which fields of the prediction dependency message are encoded as an unsigned integer Exp-Golomb-coded syntax element with a left-most bit first.
9. The method of claim 1 , further comprising:
decoding the prediction dependency message associated with a target frame in the multiview videos, the target frame having an associated view and time;
calling a marking process method for each range of frames in other views indicated by the prediction dependency message, and for each frame in the specified view that is a temporal reference; and
marking such frames to be decoded.
10. The method of claim 1 , further comprising:
maintaining a reference picture list for the multiview videos, the reference picture list indexing temporal reference pictures and spatial reference pictures of the multiview videos; and
predicting frames of the multiview videos according to reference pictures indexed by the associated reference picture list, in which each frame includes a plurality of macroblocks, and the predicting is macroblock adaptive according to a selected one of a plurality of prediction modes.
11. The method of claim 1 , in which the decoding is also based on a syntax element indicting whether a particular frame is used as a reference picture.
12. The method of claim 1 , in which the decoding is also based on a syntax element indicating whether a particular frame is used only as a reference for spatial prediction.
13. The method of claim 1 , in which the decoding is also based on a syntax element indicating whether a particular the frame is used only as a reference for temporal prediction.