Method and system for layer based view optimization encoding of 360-degree video

1. A method for doing layer based view optimization encoding to an input video, the method comprising:

receiving, by a server, the input video, the input video comprising multiple groups of pictures (GOPs), each GOP starting from a major anchor frame of the input video and comprising frames till next major anchor frame;

generating a first layer, the first layer comprising one encoded frame per GOP;

generating a first sub-layer, the first sub-layer comprising encoded frames of multiple mini-GOPs and reconstructed frames of encoded frames of the first layer, each mini-GOP comprising frames between two major anchor frames, wherein generating the first sub-layer comprises:

temporal sub-sampling of the input video to obtain sub-major anchor frames of each mini-GOP within a GOP except for the major anchor frame,

replacing missing major anchor frames in the first sub-layer with the reconstructed frames from the first layer, and

performing view optimization on the first sub-layer for a selected view; and outputting encoded video comprising the first layer and the first sub-layer.

2. The method as claimed in claim 1 , wherein the input video is 360-degree video.

3. The method as claimed in claim 1 , wherein generating the first layer comprises:

temporal sub-sampling of the input video to generate sub-sampled major anchor frames, the sub-sampled major anchor frames comprising one major anchor frame from each GOP; and

performing 2D encoding of the sub-sampled major anchor frames.

4. The method as claimed in claim 1 , wherein generating the first layer comprises:

generating the reconstructed frames from the first layer; and

providing the reconstructed frames to the first sub-layer as reference.

5. The method as claimed in claim 1 , wherein generating the first sub-layer further comprises:

performing 2D encoding of the first sub-layer except for the major anchor frames which have been encoded already in the first layer.

6. The method as claimed in claim 5 and further comprising

generating reconstructed frames from the first sub-layer; and

providing the reconstructed frames from the first sub-layer to a second sub-layer as reference.

7. The method as claimed in claim 6 and further comprising:

generating the second sub-layer, the second sub-layer comprising remaining frames that are not encoded in the first layer and the first sub-layer.

8. The method as claimed in claim 7 and further comprising:

replacing the missing major anchor frames and the sub-major anchor frames in the second sub-layer with the reconstructed video frames from the first layer and the first sub-layer;

performing view optimization on the second sub-layer for a selected view; and

performing 2D encoding of the second sub-layer except for the major anchor frames and the sub-major anchor frames which have been encoded already in the first layer and the first sub-layer.

9. A system for encoding an input video, the system comprising:

a first layer encoder for

receiving the input video, the input video comprising multiple groups of pictures (GOPs), each GOP starting from a major anchor frame of the input video and comprising frames till next major anchor frame, and

generating a first layer, the first layer comprising one encoded frame per GOP; and

a first sub-layer encoder for

generating a first sub-layer, the first sub-layer comprising encoded frames of multiple mini-GOPs and reconstructed frames of encoded frames of the first layer, each mini-GOP comprising frames between two major anchor frames, wherein the first sub-layer encoder comprises:

a sub-sampler that obtains sub-major anchor frames of each mini-GOP within a GOP except for the major anchor frame,

a replacer that replaces missing major anchor frames in the first sub-layer with the reconstructed frames from the first layer, and

a view optimizer that performs view optimization on the first sub-layer for a selected view, and

outputting encoded video comprising the first layer and the first sub-layer.

10. The system as claimed in claim 9 , wherein the input video is 360-degree video.

11. The system as claimed in claim 9 , wherein the first layer encoder comprises:

a sub-sampler for sub-sampling major anchor frames to generate sub-sampled major anchor frames, the sub-sampled major anchor frames comprising one major anchor frame from each GOP; and

a 2D video encoder for performing 2D encoding of the sub-sampled major anchor frames.

12. The system as claimed in claim 9 , wherein the first layer encoder comprises:

a producer that generates the reconstructed frames from the first layer, and

a provider that provides the reconstructed frames to sub-layers as reference.

13. The system as claimed in claim 9 , wherein the first sub-layer encoder further comprises:

a 2D encoder that encodes the first sub-layer except for the major anchor frames which have been encoded already in the first layer.

14. The system as claimed in claim 13 , wherein the first sub-layer encoder further comprises:

a producer that generates reconstructed frames from the first sub-layer, and

a provider that provides the reconstructed frames from the first sub-layer to second sub-layer as reference.

15. The system as claimed in claim 9 and further comprising:

a second sub-layer encoder for generating a second sub-layer, the second sub-layer comprising remaining frames that are not encoded in the first layer and the first sub-layer.

16. A method for decoding an encoded video stream, the method comprising:

receiving, by a client device, the encoded video stream, the encoded video stream comprising multiple groups of pictures (GOPs), each GOP starting from a major anchor frame of the encoded video stream and comprising frames till next major anchor frame;

identifying a first layer from the encoded video stream, the first layer comprising one encoded frame per GOP;

performing 2D video decoding on the first layer to generate decoded frames of the first layer;

identifying a first sub-layer from the encoded video stream, the first sub-layer comprising encoded frames of sub-major anchor frames of the mini-GOPs within a GOP except for major anchor frames which are encoded in the first layer;

performing 2D video decoding on the first sub-layer to generate decoded frames of the first sub-layer;

performing 2D video decoding on a second sub-layer to generate decoded frames of the second sub-layer in a similar way as that of the first sub-layer; and

performing post processing on decoded frames of the first layer, the first sub-layer, and the second sub-layer to generate view for display.

17. The method as claimed in claim 16 , wherein the encoded video stream is of 360-degree video.

18. A system for decoding an encoded video stream, the system comprising:

a first layer decoder for

receiving the encoded video stream, the encoded video stream comprising multiple groups of pictures (GOPs), each GOP starting from a major anchor frame of the encoded video stream and comprising frames till next major anchor frame,

identifying a first layer, the first layer comprising one encoded frame per GOP, and

decoding frames of the first layer to generate decoded frames of the first layer;

a first sub-layer decoder for,

identifying a first sub-layer, the first sub-layer comprising encoded frames of multiple mini-GOPs and reconstructed frames of encoded frames of the first layer, each mini-GOP comprising frames between two major anchor frames, and

decoding frames of the first sub-layer to generate decoded frames of the first sub-layer;

a second sub-layer decoder that has similar components as in the first sub-layer decoder, wherein a second sub-layer comprises remaining frames that are not encoded in the first layer and the first sub-layer; and

a post processor for post processing frames of the first layer, the first sub-layer, and the second sub-layer to generate view for display.

19. The system as claimed in claim 18 , wherein the encoded video stream is a 360-degree video.