Systems and methods for providing a cubic transport format for multi-lens spherical imaging
Apparatus and methods for providing a frame packing arrangement for the encoding/decoding of, for example, panoramic content. In one embodiment, the frame packing arrangement utilizes overlapping imaging data so as to enable, for example, a post-decode stitching operation to be performed. The frame packing arrangement may utilize a number of projection formats, such as a cubemap projection, and may utilize any number of differing aspect ratios such as, without limitation, 4×3, 3×2, 4×2, 2×4 aspect ratios. Additionally, the overlapping imaging data may be positioned within the frame packing arrangement chosen so as to improve upon coding efficiency. For example, the overlapping imaging data may be positioned within the frame packing arrangement so as to emphasize image continuity. An encoder apparatus and decoder apparatus for use with the aforementioned frame packing arrangements are also disclosed.
1. A method for encoding a panoramic image, the method comprising:
receiving a panoramic image that has been captured by two or more image capture devices, the panoramic image comprising at least one overlapping portion that has been captured by at least two of the two or more image capture devices;
arranging non-overlapping portions of the panoramic image into a transport format;
arranging the at least one overlapping portion of the panoramic image within the transport format; and
encoding the arranged non-overlapping portions and the arranged at least one overlapping portion of the panoramic image;
wherein the transport format comprises a cubemap projection, the cubemap projection comprising a stitch line that divides four imaging facets of the cubemap projection to generate respective first portions and second portions of the four imaging facets, and the arranging of the at least one overlapping portion of the panoramic image comprises arranging the at least one overlapping portion of the panoramic image as an image representation of the at least one overlapping portion within at least one of the respective first portions and second portions of the divided four imaging facets of the cubemap projection.
2. The method of claim 1 , wherein the division comprises bisection, and the generated respective first portion and second portion comprise a first half and a second half, respectively.
3. The method of claim 1 , further comprising arranging the transport format in a 4×3 aspect ratio, the 4×3 aspect ratio comprising four facets in a width dimension and three facets in a vertical dimension.
4. The method of claim 1 , further comprising arranging the transport format in a 4×2 aspect ratio, the 4×2 aspect ratio comprising four facets in a width dimension and two facets in a vertical dimension.
5. The method of claim 1 , wherein the arranging of the overlapping portion of the panoramic image further comprises arranging the overlapping portion of the panoramic image such that the overlapping portion is arranged in a substantially contiguous manner.
6. A non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by a processor apparatus:
receive a panoramic image that has been captured by two or more image capture devices, the panoramic image comprising an overlapping portion that has been captured by at least two of the two or more image capture devices;
arrange non-overlapping portions of the panoramic image into a transport format;
arrange the overlapping portion of the panoramic image within the transport format; and
encode the arranged non-overlapping portions and the arranged overlapping portion of the panoramic image in order to produce an encoded image;
wherein the transport format comprises a cubemap projection, the cubemap projection comprising a stitch line that bisects four imaging facets of the cubemap projection to produce respective first halves and second halves of the four imaging facets, and the arrangement of the overlapping portion of the panoramic image comprises an arrangement of the overlapping portion of the panoramic image between the respective first halves and second halves of the bisected four imaging facets of the cubemap projection.
7. The non-transitory computer-readable storage apparatus of claim 6 , wherein the arrangement of the overlapping portion of the panoramic image further comprises an arrangement of the overlapping portion of the panoramic image such that the overlapping portion is arranged in a substantially contiguous manner.
8. An encoder apparatus, the encoder apparatus comprising:
a processor apparatus; and
a non-transitory computer-readable storage apparatus, the computer-readable storage apparatus comprising a storage medium comprising computer-readable instructions, the computer-readable instructions being configured to, when executed by the processor apparatus:
receive a panoramic image that has been captured by two or more image capture devices, the panoramic image comprising an overlapping portion that has been captured by at least two of the two or more image capture devices;
arrange non-overlapping portions of the panoramic image into a transport format;
arrange the overlapping portion of the panoramic image within the transport format; and
encode the arranged non-overlapping portions and the arranged overlapping portion of the panoramic image in order to produce an encoded image;
wherein the transport format comprises a cubemap projection, the cubemap projection comprising a stitch line that bifurcates four imaging facets of the cubemap projection to produce respective first portions and second portions associated with the four imaging facets and the arrangement of the overlapping portion of the panoramic image comprises an arrangement of the overlapping portion of the panoramic image between the respective first portions and second portions associated with the bifurcated four imaging facets of the cubemap projection.
9. The encoder apparatus of claim 8 , wherein the arrangement of the overlapping portion of the panoramic image comprises an arrangement of the overlapping portion as a watermark within the four bifurcated imaging facets.
10. The encoder apparatus of claim 8 , wherein the computer-readable instructions are further configured to, when executed by the processor apparatus:
arrange the transport format into a 4×3 aspect ratio, the 4×3 aspect ratio comprising four facets in a width dimension and three facets in a vertical dimension.
11. The encoder apparatus of claim 8 , wherein the arrangement of the overlapping portion of the panoramic image further comprises an arrangement for the overlapping portion of the panoramic image such that the overlapping portion is arranged in a substantially contiguous manner.
12. The encoder apparatus of claim 11 , wherein the arrangement of the overlapping portion in the substantially contiguous manner improves upon a compression efficiency for the transport format as compared with another transport format in which an overlapping portion is not arranged in a contiguous manner.
13. The encoder apparatus of claim 8 , wherein the computer-readable instructions are further configured to, when executed by the processor apparatus:
arrange the transport format in a 4×2aspect ratio, the 4×2 aspect ratio comprising four facets in a width dimension and two facets in a vertical dimension.
14. The encoder apparatus of claim 8 , wherein the encode of the arranged overlapping portion of the panoramic image comprises an encode of the overlapping portion of the panoramic image as metadata.
15. The method of claim 1 , wherein the encoding of the arranged overlapping portion of the panoramic image comprises encoding the overlapping portion of the panoramic image as metadata.
16. The non-transitory computer-readable storage apparatus of claim 6 , wherein the arrangement of the overlapping portion of the panoramic image comprises an arrangement of the overlapping portion as a watermark within the four bifurcated imaging facets.
17. The non-transitory computer-readable storage apparatus of claim 6 , wherein the computer-readable instructions are further configured to, when executed by the processor apparatus:
arrange the transport format in a 4×3 aspect ratio, the 4×3 aspect ratio comprising four facets in a width dimension and three facets in a vertical dimension.
18. The non-transitory computer-readable storage apparatus of claim 6 , wherein the computer-readable instructions are further configured to, when executed by the processor apparatus:
arrange the transport format in a 4×2 aspect ratio, the 4×2 aspect ratio comprising four facets in a width dimension and two facets in a vertical dimension.
19. The non-transitory computer-readable storage apparatus of claim 7 , wherein the arrangement of the overlapping portion in the substantially contiguous manner improves upon a compression efficiency for the transport format as compared with another transport format in which an overlapping portion is not arranged in a contiguous manner.
20. The non-transitory computer-readable storage apparatus of claim 6 , wherein the encode of the arranged overlapping portion of the panoramic image comprises an encode of the overlapping portion of the panoramic image as metadata.