Method for output layer set mode in multilayered video stream

1. A method of decoding, the method comprising:

processing a coded video bitstream comprising multiple layers;

determining, from the coded video bitstream, a syntax element ols_mode_idc in a video parameter set (VPS);

decoding one or more layers of a current picture, the decoding comprising:

setting a syntax element PictureOutputFlag of the current picture to 0 when a syntax element sps_video_parameter_set_id is greater than 0, a syntax element each_layer_is_an_ols_flag is equal to 0, and the syntax element ols_mode_idc is equal to 2,

otherwise, setting the syntax element PictureOutputFlag of the current picture to a syntax element pic output flag signaled in a picture header; and

controlling display of one or more picture output layers of the decoded one or more layers of the current picture, wherein a non-output layer of the current picture has a PictureOutputFlag equal to 0 and is not displayed.

2. The method of claim 1 , further comprising identifying output layer set signaling based on the syntax element ols_mode_idc, including:

in a case that the syntax element ols_mode_idc in the VPS is a first value, identifying a highest layer in the coded video bitstream as the one or more picture output layers;

in a case that the syntax element ols_mode_idc in the VPS is a second value, identifying all layers in the coded video bitstream as the one or more picture output layers; and

in a case that the syntax element ols_mode_idc in the VPS is a third value, identifying the one or more picture output layers based on explicit signaling in the VPS, wherein

the first value is different from the second value and different from the third value, and

the second value is different from the third value.

3. The method of claim 2 , wherein the first value is 0, the second value is 1, and the third value is 2.

4. The method of claim 2 , wherein the identifying the one or more picture output layers by the explicit signaling in the VPS includes: (i) parsing or deriving, from the VPS, a syntax element output_layer_flag, and (ii) setting layers that have the syntax element output_layer_flag equal to 1 as the one or more picture output layers.

5. The method of claim 1 , further comprising identifying output layer set signaling based on the syntax element ols_mode_idc, including:

in a case that the syntax element ols_mode_idc in the VPS is a predetermined value, the identifying the output layer set signaling includes identifying the one or more picture output layers based on explicit signaling in the VPS.

6. The method of claim 5 , wherein the identifying the one or more picture output layers by the explicit signaling in the VPS includes: (i) parsing or deriving, from the VPS, a syntax element output_layer_flag, and (ii) setting layers that have the syntax element output_layer_flag equal to 1 as the one or more picture output layers, wherein a number of the multiple layers is greater than 2.

7. The method of claim 5 , wherein the identifying the output layer set signaling includes identifying the one or more picture output layers based on the explicit signaling in the VPS when the syntax element ols_mode_idc equals 2, and a number of layers of the multiple layers is greater than 2.

8. The method of claim 1 , further comprising

identifying output layer set signaling including identifying the highest layer in the coded video bitstream or all layers in the coded video bitstream as the one or more picture output layers by inferring the one or more picture output layers, when the syntax element ols_mode_idc is less than 2, and a number of the multiple layers is 2.

9. The method of claim 8 , wherein a syntax element num_output_layer_sets_minus1 in the VPS indicates a total number of output layer sets.

10. The method of claim 9 , wherein a syntax element vps_max_layers_minus1 in the VPS indicates a maximum allowed number of layers in each coded video sequence (CVS) referring to the VPS.

11. The method of claim 10 , wherein a syntax element ols_output_layer_flag[i][j] in the VPS indicates whether a j-th layer of an i-th output layer set is an output layer or not.

12. The method of claim 2 , wherein if all of the multiple layers in the coded video bitstream are independent layers that do not have parsing and decoding dependency on another layer, where a syntax element vps_all_independent_layers_flag of the VPS equals 1, the syntax element ols_mode_idc is not signaled, and a value of the syntax element ols_mode_idc is inferred to be the second value.

13. The method of claim 1 , wherein the decoding comprises the setting the syntax element PictureOutputFlag of the VPS equal to the syntax element pic_output_flag signaled in the picture header, regardless of a value of the syntax element ols_mode_idc.

14. The method of claim 1 , wherein the decoding comprises the setting the syntax element PictureOutputFlag of the VPS equal to 0 when the syntax element sps_video_parameter_set_id is greater than 0, the syntax element each_layer_is_an_ols_flag is equal to 0, and the syntax element ols_mode_idc is equal to 2.

15. A method of processing visual media data, the method comprising:

processing a bitstream of the visual media data according to a format rule, wherein the bitstream includes multiple layers and coded information of a current picture, the format rule specifies that

a syntax element ols_mode_idc in a video parameter set (VPS) is determined from the bitstream;

one or more layers of the current picture are decoded, the decoding comprising:

setting a syntax element PictureOutputFlag of the current picture to 0 when a syntax element sps_video_parameter_set_id is greater than 0, a syntax element each_layer_is_an_ols_flag is equal to 0, and the syntax element ols_mode_idc is equal to 2,

otherwise, setting the syntax element PictureOutputFlag of the current picture to a syntax element pic_output_flag signaled in a picture header; and

display of one or more picture output layers of the decoded one or more layers of the current picture is controlled, wherein a non-output layer of the current picture has a PictureOutputFlag equal to 0 and is not displayed.

16. An apparatus for video encoding comprising:

at least one processor configured to

determine a syntax element ols_mode_idc to be included in a video parameter set (VPS) of a coded video bitstream comprising multiple layers;

encode one or more layers of a current picture to be included in the coded video bitstream, including

setting a syntax element PictureOutputFlag of the current picture equal to 0 when a syntax element sps_video_parameter_set_id is greater than 0, a syntax element each_layer_is_an_ols_flag is equal to 0, and the syntax element ols_mode_idc is equal to 2,

otherwise, setting the syntax element PictureOutputFlag of the current picture to a syntax element pic output flag signaled in a picture header; and

wherein a non-output layer of the current picture has a PictureOutputFlag equal to 0 and is not displayed.

17. The method of claim 15 , wherein the method further comprises identifying the output layer set signaling based on the syntax element ols_mode_idc by:

in a case that the syntax element ols_mode_idc in the VPS is a first value, identifying a highest layer in the bitstream as the one or more picture output layers;

in a case that the syntax element ols_mode_idc in the VPS is a second value, identifying all layers in the bitstream as the one or more picture output layers; and

in a case that the syntax element ols_mode_idc in the VPS is a third value, identifying the one or more picture output layers based on explicit signaling in the VPS,

wherein the first value is different from the second value and different from the third value, and

wherein the second value is different from the third value.