Granted Patent B2

US 12,506,893 · App. 18/479,946 · Granted Dec 23, 2025

Inventor: Ye-Kui Wang (San Diego, CA)

Assignee: Bytedance Inc.

H04N19/46H04N19/105H04N19/172H04N19/188H04N19/423H04N19/70

Abstract

Claims (91)

1 . A method of video processing, comprising:

performing a conversion between a video comprising one or more pictures and a bitstream of the video,

wherein the bitstream conforms to a format rule,

wherein the format rule specifies that a plurality of variables used in a picture timing supplemental enhancement information (SEI) message semantics, and a plurality of syntax elements present in the picture timing SEI message, when included in the bitstream, are access unit (AU) specific,

wherein a first variable of the plurality of variables is a buffering period (BP) reset flag, a value of which is based on whether a current AU is associated with a BP SEI message,

wherein a second variable and a third variable of the plurality of variables are related to a coded picture buffer (CPB) removal delay, values of which are based on whether the current AU is an AU that initializes a hypothetical reference decoder (HRD),

wherein a first syntax element of the plurality of syntax elements specifies a number of clock ticks to wait after a removal of an AU from the CPB before one or more decoded pictures of the AU are output from a decoded picture buffer (DPB),

wherein a second syntax element of the plurality of syntax elements specifies a number of sub clock ticks to wait after a removal of a last decoding unit (DU) in an AU from the CPB before the one or more decoded pictures of the AU are output from the DPB when DecodingUnitHrdFlag is equal to 1, and

wherein a third syntax element of the plurality of syntax elements specifies a number of elemental picture period intervals that one or more decoded pictures of the current AU occupy for a display model.

2 . The method of claim 1 , wherein the first variable is BpResetFlag, the second variable is CpbRemovalDelayMsb, the third variable is CpbRemovalDelayVal, the first syntax element is pt_dpb_output_delay, the second syntax element is pt_dpb_output_du_delay, and the third syntax element is pt_display_elemental_periods_minus1.

3 . The method of claim 1 , wherein the format rule specifies that a fourth syntax element and a fifth syntax element, when included in the bitstream, are AU specific,

wherein the fourth syntax element indicates, in response to the current AU not being a first AU in the bitstream in a decoding order, whether a nominal CPB removal time of the current AU is determined relative to (a) a nominal CPB removal time of a previous AU associated with a BP SEI message, or (b) a nominal CPB removal time of a previous AU, and

wherein the fifth syntax element specifies, in response to the current AU not being a first AU in the bitstream in the decoding order, a CPB removal delay increment value relative to the nominal CPB removal time of the previous AU.

4 . The method of claim 3 , wherein a length of the fifth syntax element is indicated by a sixth syntax element in a syntax structure of the BP SEI message,

wherein the sixth syntax element is bp_cpb_removal_delay_length_minus, and the length of the fifth syntax element is (bp_cpb_removal_delay_length_minus1+1) bits.

5 . The method of claim 3 , wherein the fourth syntax element is bp_concatenation_flag and the fifth syntax element is bp_cpb_removal_delay_delta_minus1.

6 . The method of claim 1 , wherein the format rule specifies that a seventh syntax element used to compute a decoded picture buffer (DPB) output time included in a DU information SEI message, when included in the bitstream, is AU specific,

wherein the seventh syntax element specifies a number of sub clock ticks to wait after removal of a last DU in an AU from the CPB before one or more decoded pictures of the AU are output from the DPB, and

wherein the seventh syntax element is dui_dpb_output_du_delay.

7 . The method of claim 1 , wherein the format rule specifies whether a second flag is equal to zero or not is AU specific, wherein the second flag is included in a picture timing (PT) SEI message and indicates whether one or more syntax elements related to timing information are allowed to be present in the PT SEI message,

wherein when all pictures in an associated AU are random access skipped leading (RASL) pictures with a first flag equal to zero, the second flag is equal to zero, wherein the first flag equal to zero indicates each picture referring to a picture parameter set (PPS) has one or more video coding layer (VCL) network abstraction layer (NAL) units and the VCL NAL units are of a same type, and wherein the same type is only a RASL NAL unit type (NUT), and

wherein the first flag is a pps_mixed_nalu_types_in_pic_flag and the second flag is a pt_cpb_alt_timing_info_present_flag.

8 . The method of claim 7 , wherein the one or more syntax elements comprise at least one of:

an eighth syntax element indicating an alternative initial CPB removal delay delta for an i-th sublayer for a j-th CPB for a NAL HRD in units of a 90 kilohertz (kHz) clock,

a ninth syntax element indicating an alternative initial CPB removal offset delta for the i-th sublayer for the j-th CPB for the NAL HRD in units of a 90 kHz clock,

a tenth syntax element indicating, for the i-th sublayer for the NAL HRD, an offset to be used in a derivation of a nominal CPB removal time of the AU associated with the PT SEI message and of one or more subsequent AUs in a decoding order when the AU associated with the PT SEI message directly follows, in the decoding order, an AU associated with a BP SEI message,

an eleventh syntax element indicating, for the i-th sublayer for the NAL HRD, an offset to be used in a derivation of a DPB output time of an intra random access point (TRAP) AU associated with the BP SEI message when the AU associated with the PT SEI message directly follows, in the decoding order, the TRAP AU associated with the BP SEI message,

a twelfth syntax element indicating an alternative initial CPB removal delay delta for an i-th sublayer for a j-th CPB for a VCL HRD in units of a 90 kHz clock,

a thirteenth syntax element indicating an alternative initial CPB removal offset delta for the i-th sublayer for the j-th CPB for the VCL HRD in units of a 90 kHz clock,

a fourteenth syntax element indicating, for the i-th sublayer for the VCL HRD, an offset to be used in the derivation of the nominal CPB removal time of the AU associated with the PT SEI message and of the one or more subsequent AUs in the decoding order when the AU associated with the PT SEI message directly follows, in the decoding order, the AU associated with the BP SEI message, and

an fifteenth syntax element indicating, for the i-th sublayer for the VCL HRD, an offset to be used in the derivation of the DPB output time of the TRAP AU associated with the BP SEI message when the AU associated with the PT SEI message directly follows, in the decoding order, the TRAP AU associated with the BP SEI message.

9 . The method of claim 7 , wherein a value of the second flag is allowed to be equal to 1 for more than one AU following an IRAP AU in a decoding order, wherein an alternative timing is only applied to a first AU that has the second flag equal to 1 and follows the IRAP AU in the decoding order.

10 . The method of claim 1 , wherein the format rule specifies that a derivation of a previous picture in a semantics associated with a third flag and in a decoding process for a picture order count is based on a fourth flag,

wherein the previous picture is a previous picture in a decoding order that has (i) a first identifier that is the same as a slice or picture header referring to a reference picture list syntax structure, (ii) a second identifier and the fourth flag being both equal to zero, and (iii) a picture type different from a random access skipped leading (RASL) picture and a random access decodable leading (RADL) picture,

wherein the third flag indicates whether a fifth flag is present in the bitstream,

wherein the fourth flag indicates whether a current picture is allowed to be used as a reference picture,

wherein the fifth flag is used to determine a fourth variable associated with a picture order count value of a long-term reference picture,

wherein the first identifier is an identifier of a layer and is a syntax element, and the second identifier is a temporal identifier and is a variable,

wherein the third flag is delta_poc_msb_cycle_present_flag, the fourth flag is ph_non_ref_pic_flag, and the fifth flag is delta_poc_msb_cycle_lt, and wherein the first identifier is nuh_layer_id and the second identifier is TemporalId,

wherein the format rule specifies that a fifth variable prevNonDiscardableAu, which is used to determine a timing of a removal of a DU or a decoding of the DU, is AU specific, and is derived based on the fourth flag, and

wherein the fourth flag equaling one specifies that the current picture is never used as the reference picture, and the fourth flag equaling zero specifies that the current picture is allowed to be or not to be used as the reference picture.

11 . The method of claim 10 , wherein the format rule specifies that a BP SEI message and a picture timing SEI message, when included in the bitstream, are AU specific,

wherein a sixth variable associated with the BP SEI message and a seventh variable associated with the BP SEI message and the picture timing SEI message are derived based on the fourth flag,

wherein the sixth variable is indicative of an AU comprising (i) the second identifier of the AU that is equal to zero and (ii) at least one picture that is not a RASL picture or a RADL picture and for which the fourth flag is equal to zero,

wherein when the current AU not being a first AU in the decoding order, the seventh variable is indicative of a previous AU in the decoding order comprising (i) the second identifier that is equal to zero and (ii) at least one picture that is not a RASL picture or a RADL picture and for which the fourth flag is equal to zero, and

wherein the sixth variable is notDiscardableAu, and the seventh variable is prevNonDiscardableAu.

12 . The method of claim 10 , wherein the format rule specifies that a derivation of an eighth variable and a ninth variable associated with a first picture and a second picture is based on the fourth flag,

wherein the first picture is a current picture and the second picture is a previous picture in the decoding order that (i) comprises the second identifier that is equal to zero, (ii) comprises the fourth flag that is equal to zero, and (iii) is not a RASL picture or a RADL picture,

wherein the eighth variable and the ninth variable are a maximum value and a minimum value, respectively, of a picture order count of each of the following pictures with the first identifier that is equal to that of the first picture: (i) the first picture, (ii) the second picture, (iii) one or more short-term reference pictures referred to by all entries in reference picture lists of the first picture, and (iv) each picture that has been output with a CPB removal time less than the CPB removal time of the first picture and a DPB output time greater than or equal to the CPB removal time of the first picture, and

wherein the eighth variable is maxPicOrderCnt and indicates the maximum value of the picture order count, and the ninth variable is minPicOrderCnt and indicates the minimum value of the picture order count.

13 . The method of claim 1 , wherein the format rule specifies that a sixteenth syntax element, when included in the bitstream, is AU specific,

wherein a value of the sixteenth syntax element is based on whether an associated AU is an intra random access points (TRAP) AU or a gradual decoding refresh (GDR) AU,

wherein the value of the sixteenth syntax element specifies (i) whether a seventeenth syntax element is present in a BP SEI message, and (ii) whether alternative timing information is present in a picture timing (PT) SEI message in a current BP, and

wherein a value of the seventeenth syntax element indicates whether to use alternative CPB parameters or not.

14 . The method of claim 13 , wherein the value of the sixteenth syntax element is equal to zero in response to the associated AU not being an TRAP AU or a GDR AU,

wherein the value of the sixteenth syntax element is inferred to be zero in response to the sixteenth syntax element not being included in the bitstream,

wherein the value of the sixteenth syntax element being one specifies that the seventeenth syntax element is present in the BP SEI message and the alternative timing information is present in the PT SEI message in the current BP, and

wherein the sixteenth syntax element is bp_alt_cpb_params_present_flag and the seventeenth syntax element is bp_use_alt_cpb_params_flag.

15 . The method of claim 13 , wherein the format rule specifies that a value of a third syntax element is based on an eighteenth syntax element that indicates whether a temporal distance between output times of consecutive pictures of a HRD is constrained and a variable that identifies a highest temporal sublayer to be decoded,

wherein the third syntax element specifies a number of elemental picture period intervals that one or more decoded pictures of the current AU occupy for a display model,

wherein the third syntax element is included in a picture timing SEI message, wherein the eighteenth syntax element is included in an output layer set (OLS) timing and HRD parameters syntax structure, and

wherein the third syntax element is pt_display_elemental_periods_minus1, the eighteenth syntax element is fixed_pic_rate_within_cvs_flag, and the variable is Htid.

16 . The method of claim 1 , wherein the conversion comprises decoding the video from the bitstream.

17 . The method of claim 1 , wherein the conversion comprises encoding the video into the bitstream.

18 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

perform a conversion between a video comprising one or more pictures and a bitstream of the video,

wherein the bitstream conforms to a format rule,

wherein the format rule specifies that a plurality of variables used in a picture timing supplemental enhancement information (SEI) message semantics, and a plurality of syntax elements present in the picture timing SEI message, when included in the bitstream, are access unit (AU) specific,

wherein a first variable of the plurality of variables is a buffering period (BP) reset flag, a value of which is based on whether a current AU is associated with a BP SEI message,

wherein a second variable and a third variable of the plurality of variables are related to a coded picture buffer (CPB) removal delay, values of which are based on whether the current AU is an AU that initializes a hypothetical reference decoder (HRD),

wherein a first syntax element of the plurality of syntax elements specifies a number of clock ticks to wait after a removal of an AU from the CPB before one or more decoded pictures of the AU are output from a decoded picture buffer (DPB),

wherein a second syntax element of the plurality of syntax elements specifies a number of sub clock ticks to wait after a removal of a last decoding unit (DU) in an AU from the CPB before the one or more decoded pictures of the AU are output from the DPB when DecodingUnitHrdFlag is equal to 1, and

wherein a third syntax element of the plurality of syntax elements specifies a number of elemental picture period intervals that one or more decoded pictures of the current AU occupy for a display model.

19 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:

perform a conversion between a video comprising one or more pictures and a bitstream of the video,

wherein the bitstream conforms to a format rule,

wherein the format rule specifies that a plurality of variables used in a picture timing supplemental enhancement information (SEI) message semantics, and a plurality of syntax elements present in the picture timing SEI message, when included in the bitstream, are access unit (AU) specific,

wherein a first variable of the plurality of variables is a buffering period (BP) reset flag, a value of which is based on whether a current AU is associated with a BP SEI message,

wherein a second variable and a third variable of the plurality of variables are related to a coded picture buffer (CPB) removal delay, values of which are based on whether the current AU is an AU that initializes a hypothetical reference decoder (HRD),

wherein a first syntax element of the plurality of syntax elements specifies a number of clock ticks to wait after a removal of an AU from the CPB before one or more decoded pictures of the AU are output from a decoded picture buffer (DPB),

wherein a second syntax element of the plurality of syntax elements specifies a number of sub clock ticks to wait after a removal of a last decoding unit (DU) in an AU from the CPB before the one or more decoded pictures of the AU are output from the DPB when DecodingUnitHrdFlag is equal to 1, and

wherein a third syntax element of the plurality of syntax elements specifies a number of elemental picture period intervals that one or more decoded pictures of the current AU occupy for a display model.

20 . A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

generating the bitstream of the video comprising one or more pictures,

wherein the bitstream conforms to a format rule,

wherein the format rule specifies that a plurality of variables used in a picture timing supplemental enhancement information (SEI) message semantics, and a plurality of syntax elements present in the picture timing SEI message, when included in the bitstream, are access unit (AU) specific,

wherein a first variable of the plurality of variables is a buffering period (BP) reset flag, a value of which is based on whether a current AU is associated with a BP SEI message,

wherein a second variable and a third variable of the plurality of variables are related to a coded picture buffer (CPB) removal delay, values of which are based on whether the current AU is an AU that initializes a hypothetical reference decoder (HRD),

wherein a first syntax element of the plurality of syntax elements specifies a number of clock ticks to wait after a removal of an AU from the CPB before one or more decoded pictures of the AU are output from a decoded picture buffer (DPB),

wherein a second syntax element of the plurality of syntax elements specifies a number of sub clock ticks to wait after a removal of a last decoding unit (DU) in an AU from the CPB before the one or more decoded pictures of the AU are output from the DPB when DecodingUnitHrdFlag is equal to 1, and

wherein a third syntax element of the plurality of syntax elements specifies a number of elemental picture period intervals that one or more decoded pictures of the current AU occupy for a display model.

Assignments (1)

Continuity (4)

References Cited (138)