Apparatus, a method and a computer program for video coding and decoding
View Patent ↗There is disclosed a method, an apparatus, a server, a client and a non-transitory computer readable medium comprising a computer program stored therein for video coding and decoding. Depth pictures from a plurality of viewing angles are projected into a single viewing angle, making it possible to have pixel-wise joint filtering to be applied to all projected depth values. This approach enables to suppress the noise in the depth map data and provides improved performance for a view synthesis.
1. An apparatus comprising:
a video encoder configured for encoding multiple camera view representations, wherein said video encoder is further configured for
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
2. The apparatus according to claim 1 , wherein obtaining the second depth picture representing the second view further comprises
receiving the second depth picture representing the second view or projecting a third depth picture representing a third view onto the second depth picture representing the second view.
3. The apparatus according to claim 1 , wherein the video encoder is further configured for
receiving a first texture picture representing the first view;
synthesizing a second texture picture on the basis of the converted depth picture and the first texture picture.
4. The apparatus according to claim 1 , wherein the video encoder is further configured for:
receiving a first depth picture;
encoding the first depth picture into a first coded depth picture to be included in a bitstream;
deriving the first depth picture on the basis of said encoding or said first coded depth picture.
5. The apparatus according to claim 1 , wherein the video encoder is further configured for:
encoding a second input depth picture to a second coded depth picture to be included in the bitstream; and
in said encoding of the second input depth picture, using the first converted depth picture as reference for prediction.
6. The apparatus according to claim 1 , wherein the video encoder is further configured for:
including the at least one filter parameter in the bitstream.
7. A method comprising:
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
8. The method according to claim 7 , wherein obtaining a second depth picture representing the second view further comprises
receiving the second depth picture representing the second view or projecting a third depth picture representing a third view onto the second depth picture representing the second view.
9. The method according to claim 7 , further comprising
receiving a first texture picture representing the first view;
synthesizing a second texture picture on the basis of the converted depth picture and the first texture picture.
10. The method according to claim 7 , wherein the method is carried out in a decoder, the method further comprising
receiving a bitstream; and
deriving the at least one filter parameter from the bitstream.
11. The method according to claim 10 , further comprising
decoding a first coded first depth picture from the bitstream into a first depth picture;
decoding a second coded depth picture from the bitstream; and
in said decoding of the second coded depth picture, using the first converted depth picture as reference for prediction.
12. The method according to claim 7 , wherein the method is carried out in an encoder, the method further comprising
encoding the first converted depth picture into a bitstream.
13. The method according to claim 12 , further comprising
receiving a first depth picture;
encoding the first depth picture into a first coded depth picture to be included in a bitstream;
deriving the first depth picture on the basis of said encoding or said first coded depth picture.
14. The method according to claim 12 , further comprising
encoding a second input depth picture to a second coded depth picture to be included in the bitstream; and
in said encoding of the second input depth picture, using the first converted depth picture as reference for prediction.
15. The method according to claim 12 , further comprising
including the at least one filter parameter in the bitstream.
16. A non-transitory computer readable storage medium stored with code thereon for use by an apparatus, which when executed by a processor, causes the apparatus to perform:
obtaining a first depth picture representing a first view;
project the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtain a second depth picture representing the second view;
obtain at least one filter parameter;
filter the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
project at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
form a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
17. At least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform:
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
18. An apparatus comprising:
a video decoder configured for decoding multiple camera view representations, wherein said video decoder is further configured for
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
19. The apparatus according to claim 18 , wherein obtaining a second depth picture representing the second view further comprises
receiving the second depth picture representing the second view or projecting a third depth picture representing a third view onto the second depth picture representing the second view.
20. The apparatus according to claim 18 , wherein the video decoder is further configured for
receiving a first texture picture representing the first view; and
synthesizing a second texture picture on the basis of the converted depth picture and the first texture picture.
21. The apparatus according to claim 18 , wherein the video decoder is further configured for
receiving a bitstream; and
deriving the at least one filter parameter from the bitstream.
22. The apparatus according to claim 18 , wherein the video decoder is further configured for
decoding a first coded first depth picture from the bitstream into a first depth picture;
decoding a second coded depth picture from the bitstream; and
in said decoding of the second coded depth picture, using the first converted depth picture as reference for prediction.
23. A video encoder configured for encoding multiple camera view representations, wherein said video encoder is further configured for
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.
24. A video decoder configured for decoding multiple camera view representations, wherein said video decoder is further configured for
obtaining a first depth picture representing a first view;
projecting the first depth picture representing the first view of a multiple camera view representation onto a first projected depth picture representing a second view;
obtaining a second depth picture representing the second view;
obtaining at least one filter parameter;
filtering the first projected depth picture and the second depth picture jointly using the at least one filter parameter into a filtered depth picture;
projecting at least a part of the filtered depth picture onto a first back-projected depth picture representing the first view; and
forming a first converted depth picture by replacing at least a part of pixel values of the first depth picture with the first back-projected depth picture.