IP Library Granted Patent US 12,200,236
Granted Patent B2
US 12,200,236 · App. 18/340,402 · Granted Jan 14, 2025

Video encoder, video decoder, methods for encoding and decoding and video data stream for realizing advanced video coding concepts

Inventors: Yago Sánchez De La Fuente (Berlin, DE); Karsten Sühring (Berlin, DE); Cornelius Hellge (Berlin, DE); Thomas Schierl (Berlin, DE); Robert Skupin (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: GE Video Compression, LLC
H04N19/423H04N19/172H04N19/184H04N19/188H04N19/44H04N19/46H04N19/70H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,200,236
App. No.
18/340,402
Granted
Jan 14, 2025
Kind
B2
Abstract

An apparatus ( 200 ) for receiving an input video data stream according to an embodiment is provided. The input video data stream has a video encoded thereinto. The apparatus ( 200 ) is configured to generate an output video data stream from the input video data stream.

Claims (34)

1. An apparatus for receiving one or more input video data streams, wherein each of the one or more input video data streams has an input video encoded thereinto,

wherein the apparatus is configured to generate an output video data stream from the one or more input video data streams, the output video data stream encoding an output video, wherein the apparatus is configured to generate the output video data stream such that the output video is the input video being encoded within one of the one or more input video data streams, or such that the output video depends on the input video of at least one of the one or more input video data streams,

wherein, if a concatenation flag is set to a first value, the apparatus is configured to determine that coded picture buffer delay offset information is to be used for determining an access unit removal time of a current picture of a plurality of pictures of the output video from a coded picture buffer, and

if the concatenation flag is set to a second value, the apparatus is configured to determine that the coded picture buffer delay offset information is not to be used to determine the access unit removal time.

2. The apparatus according to claim 1 , wherein the apparatus is configured to drop a group of one or more pictures of the input video of a first video data stream of the one or more input video data streams to generate the output video data stream.

3. The apparatus according to claim 1 , wherein a first video received by the apparatus is a preprocessed video which results from an original video from which a group of one or more pictures has been dropped to generate the preprocessed video.

4. The apparatus according to claim 2 , wherein the coded buffer delay offset information depends on a number of pictures of the input video that have been dropped.

5. The apparatus according to claim 2 ,

wherein the one or more input video data streams are two or more input video data streams, and

wherein the apparatus is configured to splice the processed video and the input video of a second video data stream of the two or more input video data streams to obtain the output video, and is configured to encode the output video into the output video data stream.

6. The apparatus according to claim 5 ,

wherein the apparatus is configured to determine whether or not to use coded picture buffer delay offset information for determining the access unit removal time of the current picture depending on a location of the current picture within the output video, or

wherein the apparatus is configured to determine whether or not to set a coded picture buffer delay offset value of the coded picture buffer delay offset information to 0 for determining the access unit removal time of the current picture depending on the location of the current picture within the output video.

7. The apparatus according to claim 1 , wherein the apparatus is configured to determine the access unit removal time of the current picture depending on a removal time of a preceding picture.

8. The apparatus according to claim 1 , wherein the apparatus is configured to determine the access unit removal time of the current picture depending on initial coded picture buffer removal delay information.

9. The apparatus according to claim 8 , wherein the apparatus is configured to update the initial coded picture buffer removal delay information depending on a clock tick to obtain temporary coded picture buffer removal delay information to determine the access unit removal time of the current picture.

10. The apparatus according to claim 1 , wherein the apparatus is configured to signal to a video decoder, whether or not to use coded picture buffer delay offset information for determining the access unit removal time of the current picture from the coded picture buffer.

11. The apparatus according to claim 10 , wherein the current picture is located at a splicing point of the output video, where two input videos have been spliced.

12. A video encoder,

wherein the video encoder is configured to encode a video into a video data stream,

wherein the video encoder is configured to generate the video data stream such that the video data stream comprises coded picture buffer delay offset information,

wherein the video encoder is configured to determine whether or not to use the coded picture buffer delay offset information for determining a removal time of a picture from a coded picture buffer,

wherein, if a concatenation flag is set to a first value, the coded picture buffer delay offset information is determined to be used for determining the removal time, and

if the concatenation flag is set to a second value, the coded picture buffer delay offset information is determined not to be used to determine the removal time.

13. The video encoder according to claim 12 ,

wherein the video encoder is configured to generate the video data stream such that the video data stream comprises the concatenation flag.

14. The video encoder according to claim 12 , wherein the coded buffer delay offset information depends on a number of pictures of the video that have been dropped.

15. A method for encoding a video into a video data stream, comprising:

receiving one or more input data streams, wherein each of the one or more input data streams has the video encoded thereinto;

generating the video data stream from the one or more input data streams such that the video data stream comprises coded picture buffer delay offset information;

determining whether or not to use the coded picture buffer delay offset information for determining a removal time of a picture from a coded picture buffer, wherein:

if a concatenation flag is set to a first value, the coded picture buffer delay offset information is determined to be used for determining the removal time, and

if the concatenation flag is set to a second value, the coded picture buffer delay offset information is determined not to be used to determine the removal time.

16. A method for storing a data stream, comprising storing, on a digital storage medium, a video data stream having a video encoded thereinto by a method according to claim 15 .

Assignments (4)
CHANGE OF NAME Recorded Jan 30, 2026
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 074536/0781 →
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0772 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2023
From: SÁNCHEZ DE LA FUENTE, YAGO; SÜHRING, KARSTEN; HELLGE, CORNELIUS; SCHIERL, THOMAS; SKUPIN, ROBERT; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 064045/0150 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2023
From: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
To: GE VIDEO COMPRESSION, LLC
Reel/Frame 064045/0213 →
Priority Claims (2)
EP 20176178 · May 22, 2020 · regional
EP 20176206 · May 22, 2020 · regional
Continuity (3)
Continuation 17934007 · Sep 21, 2022
Continuation PCTEP2021063587 · May 21, 2021
Related Publication 20230345022A1 · Oct 26, 2023
References Cited (76)
US 8542492B2 · Dunn · 2013 [cited by applicant]
US 8594202B2 · Eleftheriadis · 2013 [cited by applicant]
US 9154785B2 · Wang · 2015 [cited by applicant]
US 9270989B2 · Hannuksela · 2016 [cited by applicant]
US 9521393B2 · Wang et al. · 2016 [cited by applicant]
US 9641862B2 · Hannuksela · 2017 [cited by applicant]
US 9712833B2 · Wang et al. · 2017 [cited by applicant]
US 9894370B2 · Wang et al. · 2018 [cited by applicant]
US 10003815B2 · Ramasubramonian · 2018 [cited by examiner]
US 10021394B2 · Wang · 2018 [cited by applicant]
US 10284858B2 · Chen et al. · 2019 [cited by applicant]
US 10432951B2 · Wang · 2019 [cited by applicant]
US 11356681B2 · Choi et al. · 2022 [cited by applicant]
US 11936889B2 · Skupin et al. · 2024 [cited by applicant]
US 11956471B2 · Pettersson et al. · 2024 [cited by applicant]
US 20080013620A1 · Hannuksela et al. · 2008 [cited by applicant]
US 20120294366A1 · Eliyahu · 2012 [cited by applicant]
US 20130266075A1 · Wang et al. · 2013 [cited by applicant]
US 20140098894A1 · Wang · 2014 [cited by applicant]
US 20140192149A1 · Wang et al. · 2014 [cited by applicant]
US 20140192893A1 · Sullivan et al. · 2014 [cited by applicant]
US 20150016547A1 · Tabatabai et al. · 2015 [cited by applicant]
US 20150103895A1 · Deshpande · 2015 [cited by applicant]
US 20160212434A1 · Tsukagoshi · 2016 [cited by applicant]
US 20160234526A1 · Wang et al. · 2016 [cited by applicant]
US 20170085896A1 · Ramasubramonian et al. · 2017 [cited by applicant]
US 20210368183A1 · Deshpande · 2021 [cited by applicant]
US 20220217375A1 · Wang · 2022 [cited by applicant]
US 20220217376A1 · Wang · 2022 [cited by applicant]
US 20220217393A1 · Wang · 2022 [cited by applicant]
JP WO2014049980A1 · 2014 [cited by applicant]
RU 2406254C2 · 2010 [cited by applicant]
WO 2014170547A1 · 2014 [cited by applicant]
WO 2015053157A1 · 2015 [cited by applicant]
WO 2015104451A1 · 2015 [cited by applicant]
WO 2015140401A1 · 2015 [cited by applicant]
WO 2015140391A1 · 2015 [cited by applicant]
WO 2015192990A1 · 2015 [cited by applicant]
WO 2020143114A1 · 2020 [cited by applicant]
WO 2020253816A1 · 2020 [cited by applicant]
WO 2021123159A1 · 2021 [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 9)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 19/ WG 11, 18th Meeting, Apr. 15-24, 2020, JVET-R2001-v1 (Year: 2020). [cited by examiner]
U.S. Appl. No. 62/905,244, filed Sep. 24, 2019. [cited by applicant]
Office Action issued in corresponding Uzbek Patent Application IAP 2023 0017 dated Apr. 25, 2023. [cited by applicant]
Office Action issued in corresponding Uzbek Patent Application IAP 2023 0018 dated Apr. 25, 2023. [cited by applicant]
Office Action issued in corresponding Eurasian Patent Application 202293135 dated Mar. 22, 2023, with English translation. [cited by applicant]
Sanchez Yago et al. AHG9: On OLS extraction and scalable nesting SEI message. Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11. JVET-Q0394. 17th Meeting: Brussels Jan. 7-17, 2020, section 2-4. [cited by applicant]
Office Action issued in corresponding Eurasian Patent Application 202293134 dated Mar. 22, 2023, with English Translation. [cited by applicant]
Wang, Ye-Kui, AHG8/AHG17: Scalable nesting SEI message. Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11. JVET-P0190-v2. 16th Meeting: Geneva, Oct. 1-11, 2019, pp. 1-4. [cited by applicant]
Skupin, Robert et al., AHG12: On subpicture extraction. Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11. JVET-Q0397-v3. 17th Meeting: Brussels, Jan. 7-17, 2020, section 2-4. [cited by applicant]
Notice of Acceptance for patent application issued in corresponding Australian Application No. 2021277524 dated Mar. 31, 2023. [cited by applicant]
Office Action issued in corresponding U.S. Appl. No. 17/981,969 dated Mar. 22, 2023. [cited by applicant]
Ye-Kui WANG et al., “SVC temporal layer switching points”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 24th Meeting: Geneva, CH, Jun. 29-Jul. 5, 2007. [cited by applicant]
Alex Eleftheriadis et al., “Showcase for the Temporal Level Nesting Flag”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 22nd Meeting: Marrakech, Morocco, Jan. 13-19, 200… [cited by applicant]
Ci He et al., “Showcase of Temporal Level Switching Point SEI Message”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T S.G16 Q.6) 25th Meeting: Shenzhen, CN, Oct. 21-26, 2007. [cited by applicant]
Office Action issued in corresponding U.S. Appl. No. 17/982,015 dated Apr. 26, 2023. [cited by applicant]
Notice of Allowance issued in corresponding Taiwanese Patent Application No. 110118547 dated Jan. 7, 2023, with English translation. [cited by applicant]
ITU-T H.266 (Aug. 2020), Series H Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Versatile Video Coding. [cited by applicant]
ISO/IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO/IEC 23008 10 (HEVC), edition 1, 2013. [cited by applicant]
ISO/IEC, ITU-T. High efficiency video coding. ITU-T Recommendation H.265 | ISO/IEC 23008 10 (HEVC), edition 2, 2014. [cited by applicant]
Office Action issued in corresponding Australian Patent Application No. 2022271427 dated Jan. 5, 2023. [cited by applicant]
International Telecommunication Union, Telecommunication Standardization Sector of ITU, 'H series: Audiovisual and Multimedia systems, Infrastructure of Audiovisual Services—Coding of Moving Video', H.265, Nov. 2019; ap… [cited by applicant]
International Telecommunication Union, Telecommunication Standardization Sector of ITU, 'Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual services—Coding of Moving Video—Advanced Video Coding f… [cited by applicant]
Office Action issued in corresponding Australian Patent Application No. 2022271423 dated Jan. 10, 2023. [cited by applicant]
Office Action issued in corresponding Australian Patent Application 2021277524 dated Dec. 20, 2022. [cited by applicant]
Bross, B. et al., “Versatile Video Coding (Draft 9)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting: by teleconference, Apr. 15-24, 2020, Document: JVET-R2001-vA. [cited by applicant]
Office Action issued in corresponding Eurasian Patent Application 202293010 dated Dec. 21, 2022, with English translation. [cited by applicant]
International Search Report mailed Jul. 23, 2021 for International Patent Application No. PCT/EP2021/063587. [cited by applicant]
Written Opinion mailed mailed Jul. 23, 2021 for International Patent Application No. PCT/EP2021/063587. [cited by applicant]
Pettersson M et al, “1HLS: Dependent RAP indication SEI message”, 18. JCT-VC Meeting; Jun. 30, 2014-Sep. 7, 2014; Sapporo; (Joint Collaborative Team On Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16 ) ; URL: htt… [cited by applicant]
Sanchez (Fraunhofer) Y., et al.: “AHG9: On DRAP output”, 131. MPEG Meeting; Jun. 29, 2020-Jul. 3, 2020; Online; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m54090; JVET-S0104 May 22, 2020 (May 22, 2020)… [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 9)”, Joing Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 18th Meeting, Apr. 15-24, 2020, JVET-R2001-v1 (Year: 2020). [cited by applicant]
Office Action issued in corresponding U.S. Appl. No. 17/981,969 dated Jul. 24, 2023. [cited by applicant]
Office Action issued in corresponding New Zealand Patent Application No. 801428 dated Jul. 31, 2023. [cited by applicant]
Office Action issued in corresponding Eurasian Patent Application 202293010 dated Jul. 7, 2023, with English translation. [cited by applicant]
Office Action is issued in corresponding Georgian Patent Application 16098/1 dated Jul. 1, 2023, with English translation. [cited by applicant]