IP Library › Granted Patent US 12,432,365
Granted Patent B2
US 12,432,365 · App. 19/203,026 · Granted Sep 30, 2025

Concept for picture/video data streams allowing efficient reducibility or efficient random access

Inventors: Robert Skupin (Berlin, DE); Yago Sánchez de la Fuente (Berlin, DE); Thomas Schierl (Berlin, DE); Cornelius Hellge (Berlin, DE); Karsten Grueneberg (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V.
H04N19/33H04N19/119H04N19/132H04N19/146H04N19/167H04N19/174H04N19/34H04N19/46H04N19/70H04N19/88
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,432,365
App. No.
19/203,026
Granted
Sep 30, 2025
Kind
B2
Abstract

A video data stream is rendered reducible in a manner so that the reduction leads to a restriction of pictures of the reduced video data stream to merely a predetermined subarea of the pictures of the original video data stream and in a manner so that transcoding, such as re-quantization, may be avoided and a conformance of the reduced video data stream relative to the codec underlying the original video data stream be maintained. This is achieved by providing the video data stream with information including an indication of the predetermined subarea and replacement indices for redirecting the indices included by the payload portion so as to refer to, and/or replacement parameters for adjusting the first set of coding parameter settings so as to result in, a second set of coding parameter settings.

Claims (68)

1. A method for storing a video, comprising:

storing, a non-transitory digital storage medium, a video data stream representing the video, the video data stream comprising:

a parameter set portion indicating coding parameter settings;

a payload portion into which pictures of the video are coded in a manner parameterized using a first set of the coding parameter settings, the first set being indexed by indices comprised by the payload portion,

wherein the video data stream comprises an information comprising:

an indication of a predetermined subarea of the pictures, and

replacement parameters for adjusting the first set of coding parameter settings so as to result in a second set of coding parameter settings, wherein the coding parameter settings define buffer size and timing,

wherein the second set of coding parameters are selected so that a reduced video data stream modified compared to the video data stream by:

removing portions of the payload portion referring to an area of the pictures outside the predetermined subarea, and

changing location indications in the payload portion so to indicate a location in a manner measured from a circumference of the predetermined subarea instead of the pictures,

comprises a reduced payload portion having encoded thereinto subarea-specific pictures showing the predetermined subarea of the pictures in a manner parameterized using the second set of coding parameter settings,

wherein the video data stream and the reduced video data stream are conforming to HEVC, and

wherein the coding parameter settings define one or more of:

picture size,

tile structure,

buffer size and timing.

2. The method according to claim 1 wherein the parameter set portion is comprised by at least one of SPS NAL units and VPS NAL units and PPS NAL units.

3. The method according to claim 1 wherein the indices are comprised in slice headers of slices of the payload portion, each slice having encoded thereinto a corresponding area of a picture which does not cross a boundary of the predetermined subarea.

4. The method according to claim 1 wherein

a portion of the first set of coding parameter settings indexed by the indices from which the redirection by the replacement indices is to be performed, and/or which is replaced by the replacement parameters, pertains one or more of:

picture size,

tile structure,

buffer size and timing.

5. A method for encoding a video into a video data stream, comprising:

a parameter setter configured to determine coding parameter settings and generate a parameter set portion of the video data stream indicating the coding parameter settings;

a coding core configured to encode pictures of the video into a payload portion of the video data stream in a manner parameterized using a first set of the coding parameter settings, the first set being indexed by indices comprised by the payload portion,

wherein the method comprises providing the video data stream with an information comprising:

an indication of a predetermined subarea of the pictures, and

replacement parameters for adjusting the first set of coding parameter settings so as to result in a second set of coding parameter settings, wherein the coding parameter settings define buffer size and timing,

wherein the second set of coding parameters are selected so that a reduced video data stream modified compared to the video data stream by:

removing portions of the payload portion referring to an area of the pictures outside the predetermined subarea, and

changing location indications in the payload portion so to indicate a location in a manner measured from a circumference of the predetermined subarea instead of the pictures,

comprises a reduced payload portion having encoded thereinto subarea-specific pictures showing the subarea of the pictures in a manner parameterized using the second set of coding parameter settings,

wherein the video data stream and the reduced video data stream are conforming to HEVC,

wherein the coding parameter settings define one or more of:

picture size,

tile structure,

buffer size and timing.

6. The method according to claim 5 wherein the parameter set portion is comprised by one or more of SPS NAL units and VPS NAL units and PPS NAL units.

7. The method according to claim 5 wherein the encoder is configured to insert the indices into slice headers of slices of the payload portion, and to encode into each slice a corresponding area of a picture which does not cross a boundary of the predetermined subarea.

8. The method according to claim 5 wherein

a portion of the first set of coding parameter settings indexed by the indices from which the redirection by the replacement indices is to be performed, and/or which is replaced by the replacement parameters, pertains one or more of:

picture size,

tile structure,

buffer timing.

9. A method for processing a video data stream so as to be subject to video decoding, the video data stream comprising:

a parameter set portion indicating coding parameter settings;

a payload portion into which pictures of the video are coded in a manner parameterized using a first set of the coding parameter settings, the first set being indexed by indices comprised by the payload portion,

wherein the method comprises:

reading from the video data stream an information comprising:

an indication of a predetermined subarea of the pictures, and

replacement parameters for adjusting the first set of coding parameter settings so as to result in, a second set of coding parameter settings, wherein the coding parameter settings define buffer size and timing,

reducing the video data stream to a reduced video data stream modified by:

performing the redirection and/or adjustment so that the second set of coding parameter settings is indexed by the payload portion's indices;

removing portions of the payload portion referring to an area of the pictures outside the predetermined subarea, and

changing location indications in the payload portion so to indicate a location measured from a circumference of the predetermined subarea instead of the pictures, so that the reduced video data stream comprises a reduced payload portion which has encoded thereinto subarea-specific pictures showing the predetermined subarea of the pictures in a manner parameterized using the second set of coding parameter settings,

wherein the video data stream and the reduced video data stream are conforming to HEVC,

wherein the coding parameter settings define one or more of:

picture size,

tile structure,

buffer size and timing.

10. The method according to claim 9 wherein the parameter set portion is comprised by one or more of SPS NAL units and VPS NAL units and PPS NAL units.

11. The method according to claim 9 wherein the indices are located in slice headers of slices of the payload portion, each slice having encoded thereinto a corresponding area of a picture which does not cross a boundary of the predetermined subarea.

12. The method according to claim 9 wherein

a portion of the first set of coding parameter settings indexed by the indices from which the redirection by the replacement indices is to be performed, and/or which is replaced by the replacement parameters, pertains one or more of:

picture size,

tile structure,

buffer size and timing.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2025
From: SKUPIN, ROBERT; SÁNCHEZ DE LA FUENTE, YAGO; SCHIERL, THOMAS; HELLGE, CORNELIUS; GRUENEBERG, KARSTEN; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 071472/0723 →
Priority Claims (1)
EP 16154947 · Feb 9, 2016 · regional
Continuity (7)
Continuation 19026225 · Jan 16, 2025
Continuation 18666025 · May 16, 2024
Continuation 18359202 · Jul 26, 2023
Continuation 17122753 · Dec 15, 2020
Continuation 16118146 · Aug 30, 2018
Continuation PCTEP2017052769 · Feb 8, 2017
Related Publication 20250267299A1 · Aug 21, 2025
References Cited (105)
US 6141034A · Mccutchen · 2000 [cited by applicant]
US 6259826B1 · Pollard et al. · 2001 [cited by applicant]
US 7588314B1 · Nisnevich et al. · 2009 [cited by applicant]
US 10244252B2 · Lim et al. · 2019 [cited by applicant]
US 10958921B2 · Skupin et al. · 2021 [cited by applicant]
US 20030095338A1 · Singh et al. · 2003 [cited by applicant]
US 20040008766A1 · Wang et al. · 2004 [cited by applicant]
US 20040239763A1 · Notea et al. · 2004 [cited by applicant]
US 20050190774A1 · Wiegand · 2005 [cited by applicant]
US 20060115001A1 · Wang et al. · 2006 [cited by applicant]
US 20060239353A1 · De et al. · 2006 [cited by applicant]
US 20060250505A1 · Gennetten et al. · 2006 [cited by applicant]
US 20070041611A1 · Hersch et al. · 2007 [cited by applicant]
US 20070154072A1 · Taraba et al. · 2007 [cited by applicant]
US 20090041130A1 · Yoon et al. · 2009 [cited by applicant]
US 20090074070A1 · Yin et al. · 2009 [cited by applicant]
US 20090290009A1 · Suzuki et al. · 2009 [cited by applicant]
US 20100046803A1 · Tomita · 2010 [cited by applicant]
US 20100128801A1 · Hashimoto · 2010 [cited by applicant]
US 20100169349A1 · Zou et al. · 2010 [cited by applicant]
US 20100253793A1 · Auberger et al. · 2010 [cited by applicant]
US 20120013748A1 · Stanwood et al. · 2012 [cited by applicant]
US 20120033731A1 · Yamamoto et al. · 2012 [cited by applicant]
US 20120195356A1 · Yi et al. · 2012 [cited by applicant]
US 20120265901A1 · Swenson et al. · 2012 [cited by applicant]
US 20130076855A1 · Miyamoto et al. · 2013 [cited by applicant]
US 20130114694A1 · Chen et al. · 2013 [cited by applicant]
US 20130294499A1 · Wang · 2013 [cited by applicant]
US 20130294500A1 · Wang · 2013 [cited by applicant]
US 20130308708A1 · Sugio et al. · 2013 [cited by applicant]
US 20140022240A1 · Lee et al. · 2014 [cited by applicant]
US 20140086328A1 · Chen et al. · 2014 [cited by applicant]
US 20140092964A1 · Ugur et al. · 2014 [cited by applicant]
US 20140301464A1 · Wu et al. · 2014 [cited by applicant]
US 20140301479A1 · Pandit et al. · 2014 [cited by applicant]
US 20140348235A1 · Yamamoto et al. · 2014 [cited by applicant]
US 20140362173A1 · Doepke et al. · 2014 [cited by applicant]
US 20150003513A1 · Hashimoto et al. · 2015 [cited by applicant]
US 20150023405A1 · Joshi et al. · 2015 [cited by applicant]
US 20150103921A1 · Hannuksela · 2015 [cited by examiner]
US 20150103926A1 · Hannuksela · 2015 [cited by applicant]
US 20150110192A1 · Wang et al. · 2015 [cited by applicant]
US 20150156501A1 · Hannuksela · 2015 [cited by applicant]
US 20150189298A1 · Ye et al. · 2015 [cited by applicant]
US 20150195577A1 · Hannuksela · 2015 [cited by applicant]
US 20150264404A1 · Hannuksela · 2015 [cited by applicant]
US 20150304667A1 · Suehring et al. · 2015 [cited by applicant]
US 20150341552A1 · Chen et al. · 2015 [cited by applicant]
US 20160005435A1 · Campbell et al. · 2016 [cited by applicant]
US 20160100187A1 · Lee et al. · 2016 [cited by applicant]
US 20160100189A1 · Pang et al. · 2016 [cited by applicant]
US 20160105670A1 · Pang et al. · 2016 [cited by applicant]
US 20160337668A1 · Le Leannec et al. · 2016 [cited by applicant]
US 20170118540A1 · Thomas et al. · 2017 [cited by applicant]
US 20170142442A1 · Tsukuba et al. · 2017 [cited by applicant]
US 20170155912A1 · Thomas et al. · 2017 [cited by applicant]
US 20170200255A1 · Lin et al. · 2017 [cited by applicant]
US 20180005343A1 · Rhoads et al. · 2018 [cited by applicant]
US 20180041762A1 · Ikai et al. · 2018 [cited by applicant]
US 20180098091A1 · Wiegand et al. · 2018 [cited by applicant]
US 20180176601A1 · Jeong et al. · 2018 [cited by applicant]
US 20180234697A1 · Jang et al. · 2018 [cited by applicant]
US 20180249192A1 · Denoual et al. · 2018 [cited by applicant]
US 20190014337A1 · Skupin et al. · 2019 [cited by applicant]
US 20190020884A1 · Skupin et al. · 2019 [cited by applicant]
US 20190068985A1 · Yamamoto et al. · 2019 [cited by applicant]
US 20190253734A1 · Lee et al. · 2019 [cited by applicant]
US 20190297326A1 · Reda et al. · 2019 [cited by applicant]
US 20200045329A1 · Hashimoto et al. · 2020 [cited by applicant]
US 20200128252A1 · Fu et al. · 2020 [cited by applicant]
US 20200204829A1 · Stepin et al. · 2020 [cited by applicant]
US 20200221104A1 · Deshpande · 2020 [cited by applicant]
US 20200275129A1 · Deshpande · 2020 [cited by applicant]
US 20210211695A1 · Skupin et al. · 2021 [cited by applicant]
CN 1774934A · 2006 [cited by applicant]
CN 1781316A · 2006 [cited by applicant]
CN 101160975A · 2008 [cited by applicant]
CN 101636761A · 2010 [cited by applicant]
CN 101656822A · 2010 [cited by applicant]
CN 102209241A · 2011 [cited by applicant]
CN 103546737A · 2014 [cited by applicant]
CN 104813660A · 2015 [cited by applicant]
CN 105144720A · 2015 [cited by applicant]
JP H11346370A · 1999 [cited by applicant]
JP 2008005531A · 2008 [cited by applicant]
JP 2011120305A · 2011 [cited by applicant]
JP 5729237B2 · 2015 [cited by applicant]
KR 20180111969A · 2018 [cited by applicant]
KR 20210019137A · 2021 [cited by applicant]
KR 20210019597A · 2021 [cited by applicant]
KR 20210034127A · 2021 [cited by applicant]
TW 201524192A · 2015 [cited by applicant]
WO 2011128366A1 · 2011 [cited by applicant]
WO 2014001573A1 · 2014 [cited by applicant]
WO 2015059194A1 · 2015 [cited by applicant]
Auyeung, Cheung, HLS: Non-Significant Slice Segments with Tiles for Single Layer HEVC Extensions, JCTVC-N0269 (version 2), ITU, Jul. 25, 2013, pp. 1-4, JCTVC-N0269_r1.docx. [cited by applicant]
High Efficiency Video Coding, Recommendation ITU-T H.265 (Apr. 2015), ITU, Jul. 9, 2015, pp. 244-248, 357-360. [cited by applicant]
Giovanni Ballocca: “Requirements for Frame Compatible Format”, 104. MPEG Meeting; Apr. 24, 2013, Incheon; Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11, No. m29229, XP030057761, Apr. 24, 2013. [cited by applicant]
Auyeung, Cheung, HLS: Non-significant slice segments with tiles for single layer HEVC extensions, JCTVC-NO269_r1.docx[online], Jul. 25, 2013, http://phenix.int-evry.fr/jct/doc_end_user/documents/14_Vienna/wg11/JCTVC-NO2… [cited by applicant]
Ballocca, et al., Rectangular Region Frame Packing Arrangement, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-Q0256_r1, Apr. 3, 2014. [cited by applicant]
Hattori, et al., “HLS: Extensions to Temporal Motion-Constrained Tile Sets SEI Message”, JCTVC-O0063; Joint Collaborative Team on Video Coding (JCT-VC) ofITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 15th Meeting: Gene… [cited by applicant]
Stockhammer, Thomas, “Dynamic Adaptive Streaming over HTTP—Standards and Design Principles”, In Proceedings of the ACM Multimedia Systems (MMSys11), San Jose, California, USA, Feb. 2011, pp. 133-143. [cited by applicant]
Tokumo, et al., “DASH: Signaling Tile Encapsulation Modes”, 108, MPEG meeting, Mar. 31-Apr. 4, 2014, No. M33111, Mar. 26, 2014, ISO/IEC JTC1/SC29/WG11, MPEG2014/NiLU111, 6 pages. [cited by applicant]
Wu, et al., “Motion-constrained tile sets SEI message”, JCTVC-M0235, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 13th Meeting: Incheon, KR, Apr. 18-26, 2013, v. 2. [cited by applicant]
Wu, et al., “Motion-Constrained Title Sets SEI Message”, Document: JCTVC-M0235 (v.1); Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Incheon, KR, Apr. 1… [cited by applicant]