Methods and apparatus for deriving composite tracks with track grouping

1. An encoding method for encoding a composite track derivation for a plurality of sub-picture tracks, the encoding method comprising:

encoding three-dimensional video data in a hierarchical track structure, comprising:

encoding the three-dimensional video data into a plurality of two-dimensional sub-picture tracks associated with a viewport, wherein the plurality of two-dimensional sub-picture tracks are at a first level of the hierarchical track structure; and

encoding a composite track derivation for composing the plurality of two-dimensional sub-picture tracks for the viewport, wherein the composite track derivation is associated with a composite track at a second level in the hierarchical track structure that is above the first level of the plurality of two-dimensional sub-picture tracks, and comprises data indicative of:

the plurality of two-dimensional sub-picture tracks belonging to a same group;

placement information for each of the plurality of two-dimensional sub-picture tracks, wherein the placement information can be used to compose sample images from the plurality of two-dimensional sub-picture tracks into a canvas associated with the viewport, wherein the canvas comprises a plurality of portions and each of the plurality of two-dimensional sub-picture tracks comprises video data for a different portion of the canvas; and

a composition layout operation to adjust the composition of the sample images into the canvas if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas; and

providing the encoded three-dimensional video data and the composition layout operation.

2. The encoding method of claim 1 , wherein the composition layout comprises a gap between the two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas, an overlap of the two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas, or both.

3. The encoding method of claim 1 , wherein encoding the composite track derivation comprises:

encoding a width, a height, or both, of the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

4. The encoding method of claim 1 , wherein encoding the composite track derivation comprises:

encoding a size, a location, or both, of sample images in the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

5. The encoding method of claim 1 , wherein encoding the composite track derivation comprises:

encoding a size, a location, or both, of sample images in the canvas in a track header box of a track containing the plurality of two-dimensional sub-picture tracks; and

encoding the track containing the plurality of two-dimensional sub-picture tracks in a sub-picture composition track group box contained in each two-dimensional sub-picture track of the plurality of two-dimensional sub-picture tracks.

6. The encoding method of claim 5 , wherein encoding the composite track derivation comprises:

encoding a matrix in the sub-picture composition track group box, wherein the matrix used to overlay each of the plurality of two-dimensional sub-picture tracks on the canvas.

7. A decoding method for decoding video data to derive a composite track, the decoding method comprising:

receiving (a) three-dimensional video data encoded into a plurality of two-dimensional sub-picture tracks associated with a viewport, wherein the plurality of two-dimensional sub-picture tracks are at a first level of a hierarchical track structure, and (b) a composite track derivation for composing the plurality of two-dimensional sub-picture tracks for the viewport, wherein the composite track derivation is associated with a composite track at a second level in the hierarchical track structure that is above the first level of the plurality of two-dimensional sub-picture tracks, and comprises data indicative of:

the plurality of two-dimensional sub-picture tracks belonging to a same group;

placement information for each of the plurality of two-dimensional sub-picture tracks, wherein the placement information can be used to compose sample images from the plurality of two-dimensional sub-picture tracks into a canvas associated with the viewport, wherein the canvas comprises a plurality of portions and each of the plurality of two-dimensional sub-picture tracks comprises video data for a different portion of the canvas; and

a composition layout operation to adjust the composition of the sample images into the canvas if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas carried in a derived track;

determining the plurality of two-dimensional sub-picture tracks belonging to a same group; and

composing the plurality of two-dimensional sub-picture tracks into the canvas according to the composite track derivation to derive a composite track, comprising:

determining two or more of the composed two-dimensional sub-picture tracks comprise the composition layout; and

adjusting the composition based on the composition layout operation to compensate for the composition layout.

8. The decoding method of claim 7 , wherein determining the two or more of the composed two-dimensional sub-picture tracks comprise the composition layout comprises determining the two or more of the composed two-dimensional sub-picture tracks comprise a gap between the two or more of the composed plurality of two-dimensional sub-picture tracks composed on the canvas, an overlap of the two or more of the composed plurality of two-dimensional sub-picture tracks composed on the canvas, or both.

9. The decoding method of claim 7 , further comprising decoding the composite track derivation, comprising:

decoding a width, a height, or both, of the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

10. The decoding method of claim 7 , further comprising decoding the composite track derivation, comprising:

decoding a size, a location, or both, of sample images in the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

11. The decoding method of claim 7 , further comprising decoding the composite track derivation, comprising:

decoding a size, a location, or both, of sample images in the canvas in a track header box of a track containing the plurality of two-dimensional sub-picture tracks; and

decoding the track containing the plurality of two-dimensional sub-picture tracks in a sub-picture composition track group box contained in each two-dimensional sub-picture track of the plurality of two-dimensional sub-picture tracks.

12. The decoding method of claim 11 , wherein decoding the composite track derivation further comprises:

decoding a matrix in the sub-picture composition track group box, wherein the matrix used to overlay each of the plurality of two-dimensional sub-picture tracks on the canvas.

13. An apparatus configured to decode video data, the apparatus comprising a processor in communication with memory, the processor being configured to execute instructions stored in the memory that cause the processor to:

receive (a) three-dimensional video data encoded into a plurality of two-dimensional sub-picture tracks associated with a viewport, wherein the plurality of two-dimensional sub-picture tracks are at a first level of a hierarchical track structure, and (b) a composite track derivation for composing the plurality of two-dimensional sub-picture tracks for the viewport, wherein the composite track derivation is associated with a composite track at a second level in the hierarchical track structure that is above the first level of the plurality of two-dimensional sub-picture tracks, and comprises data indicative of:

the plurality of two-dimensional sub-picture tracks belonging to a same group;

placement information for each of the plurality of two-dimensional sub-picture tracks, wherein the placement information can be used to compose sample images from the plurality of two-dimensional sub-picture tracks into a canvas associated with the viewport, wherein the canvas comprises a plurality of portions and each of the plurality of two-dimensional sub-picture tracks comprises video data for a different portion of the canvas; and

a composition layout operation to adjust the composition of the sample images into the canvas if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas carried in a derived track;

determine the plurality of two-dimensional sub-picture tracks belonging to a same group; and

compose the plurality of two-dimensional sub-picture tracks into the canvas according to the composite track derivation to derive a composite track, comprising:

determining two or more of the composed two-dimensional sub-picture tracks comprise the composition layout; and

adjusting the composition based on the composition layout operation to compensate for the composition layout.

14. The apparatus of claim 13 , wherein determining the two or more of the composed two-dimensional sub-picture tracks comprise the composition layout comprises determining the two or more of the composed two-dimensional sub-picture tracks comprise a gap between the two or more of the composed plurality of two-dimensional sub-picture tracks composed on the canvas, an overlap of the two or more of the composed plurality of two-dimensional sub-picture tracks composed on the canvas, or both.

15. The apparatus of claim 13 , wherein the instructions are further configured to cause the processor to decode the composite track derivation, comprising:

decoding a width, a height, or both, of the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

16. The apparatus of claim 13 , wherein the instructions are further configured to cause the processor to decode the composite track derivation, comprising:

decoding a size, a location, or both, of sample images in the canvas in a sub-picture composition track group box contained in each of the plurality of two-dimensional sub-picture tracks.

17. The apparatus of claim 13 , wherein the instructions are further configured to cause the processor to decode the composite track derivation, comprising:

decoding a size, a location, or both, of sample images in the canvas in a track header box of a track containing the plurality of two-dimensional sub-picture tracks; and

decoding the track containing the plurality of two-dimensional sub-picture tracks in a sub-picture composition track group box contained in each of the two-dimensional sub-picture track of the plurality of two-dimensional sub-picture tracks.

18. The apparatus of claim 17 , wherein decoding the composite track derivation further comprises:

decoding a matrix in the sub-picture composition track group box, wherein the matrix used to overlay each of the plurality of two-dimensional sub-picture tracks on the canvas.

19. An apparatus for encoding video data, the apparatus comprising a processor in communication with memory, the processor being configured to execute instructions stored in the memory that cause the processor to:

encode three-dimensional video data in a hierarchical track structure, comprising:

encoding the three-dimensional video data into a plurality of two-dimensional sub-picture tracks associated with a viewport, wherein the plurality of two-dimensional sub-picture tracks are at a first level of the hierarchical track structure; and

encode a composite track derivation for composing the plurality of two-dimensional sub-picture tracks for the viewport, wherein the composite track derivation is associated with a composite track at a second level in the hierarchical track structure that is above the first level of the plurality of two-dimensional sub-picture tracks, and comprises data indicative of:

the plurality of two-dimensional sub-picture tracks belonging to a same group;

placement information for each of the plurality of two-dimensional sub-picture tracks, wherein the placement information can be used to compose sample images from the plurality of two-dimensional sub-picture tracks into a canvas associated with the viewport, wherein the canvas comprises a plurality of portions and each of the plurality of two-dimensional sub-picture tracks comprises video data for a different portion of the canvas; and

a composition layout operation to adjust the composition of the sample images into the canvas if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas; and

provide the encoded three-dimensional video data and the composition layout.