Apparatus and methods for video compression using multi-resolution scalable coding
View Patent ↗Apparatus and methods for digital video data compression via a scalable, multi-resolution approach. In one embodiment, the video content may be encoded using a multi-resolution and/or multi-quality scalable coding approach that reduces computational and/or energy load on a client device. In one implementation, a low fidelity image is obtained based on a first full resolution image. The low fidelity image may be encoded to obtain a low fidelity bitstream. A second full resolution image may be obtained based on the low fidelity bitstream. A portion of a difference image obtained based on the second full resolution image and the first full resolution may be encoded to obtain a high fidelity bitstream. The low fidelity bitstream and the high fidelity bitstream may be provided to e.g., a receiving device.
1. A method for providing video content, the method comprising:
obtaining a first panoramic image by decoding a first bitstream using a first decoder;
obtaining a view frame image by at least decoding a second bitstream using a second decoder;
obtaining a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;
obtaining an output view frame image at least in part by combining the view frame image and the second panoramic image; and
displaying the obtained output view frame image via a rendering device.
2. The method of claim 1 , wherein the obtaining of the first panoramic image comprises obtaining a low fidelity image characterized by at least a first resolution, and the obtaining of the second panoramic image comprises obtaining a high fidelity image characterized by at least a second resolution that is greater than the first resolution.
3. The method of claim 2 , wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.
4. The method of claim 3 , wherein:
the resolution scalability parameter is based at least on a display capability of a display device; and
the method further comprises displaying the output view frame image via the display device.
5. The method of claim 3 , wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.
6. The method of claim 1 , further comprising:
obtaining an intermediate image by at least decoding an intermediate bitstream using at least a third decoder; and
wherein the up-sampling of the first panoramic image to the resolution of the view frame image further comprises combining the intermediate image with the first panoramic image.
7. A method for providing video content, the method comprising:
for a first viewport display, obtaining a first output image by combining a panoramic image and a first difference image for first coordinates of the first viewport display;
determining when a change occurs in the first viewport display;
responsive to the change, requesting a second difference image; and
obtaining a second output image for the first viewport display by combining the panoramic image and the second difference image.
8. The method of claim 7 , further comprising scaling at least one of a resolution quality, a size, and/or a bitrate associated with the second difference image according to a Quality of Service (QoS) parameter.
9. A computerized apparatus for providing video content, the apparatus comprising:
an electronic storage apparatus configured to store a sequence of images;
a communications interface configured to enable communication of bitstreams to a client device; and
one or more processors configured to execute a plurality of computer readable instructions; and
a storage apparatus in data communication with the one or more processors and comprising at least one computer program, the at least one computer program comprising a plurality of instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:
obtain a first panoramic image by a decode of a first bitstream using a first decoder;
obtain a view frame image by at least a decode of a second bitstream using a second decoder;
obtain a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;
obtain an output view frame image at least in part by a combination of the view frame image and the second panoramic image; and
cause display of the obtained output view frame image via a rendering device.
10. The computerized apparatus of claim 9 , wherein the first panoramic image comprises a low fidelity image characterized by at least a first resolution, and the second panoramic image comprises a high fidelity image characterized by at least a second resolution that is greater than the first resolution.
11. The computerized apparatus of claim 10 , wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.
12. The computerized apparatus of claim 11 , wherein:
the resolution scalability parameter is based at least on a display capability of a display device; and
the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:
display the output view frame image via the display device.
13. The computerized apparatus of claim 11 , wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.
14. The computerized apparatus of claim 9 , wherein the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:
obtain an intermediate image by at least a decode of an intermediate bitstream using at least a third decoder; and
wherein the up-sampling of the first panoramic image to the resolution of the view frame image further comprises a combination of the intermediate image with the first panoramic image.
15. A non-transitory computer readable apparatus comprising a storage medium in data communication with one or more processors and comprising at least one computer program, the at least one computer program comprising a plurality of instructions which, when executed by the one or more processors, cause a computerized apparatus to:
obtain a first panoramic image by a decode of a first bitstream using a first decoder;
obtain a view frame image by at least a decode of a second bitstream using a second decoder;
obtain a second panoramic image at least in part by up-sampling the first panoramic image to the resolution of the view frame image;
obtain an output view frame image at least in part by a combination of the view frame image and the second panoramic image; and
cause display of the obtained output view frame image via a rendering device.
16. The non-transitory computer readable apparatus of claim 15 , wherein the first panoramic image comprises a low fidelity image characterized by at least a first resolution, and the second panoramic image comprises a high fidelity image characterized by at least a second resolution that is greater than the first resolution.
17. The non-transitory computer readable apparatus of claim 16 , wherein a ratio of the first resolution to the second resolution is characterized by a resolution scalability parameter.
18. The non-transitory computer readable apparatus of claim 17 , wherein:
the resolution scalability parameter is based at least on a display capability of a display device; and
the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:
display the output view frame image via the display device.
19. The non-transitory computer readable apparatus of claim 17 , wherein the resolution scalability parameter is based at least on a processing or memory limitation of a display device.
20. The non-transitory computer readable apparatus of claim 15 , wherein the at least one computer program further comprises additional instructions which are configured to, when executed by the one or more processors, cause the computerized apparatus to:
obtain an intermediate image by at least a decode of an intermediate bitstream using at least a third decoder; and
wherein the up-sampling of the first panoramic image to the resolution of the view frame image further comprises a combination of the intermediate image with the first panoramic image.