IP Library Granted Patent US 12,230,031
Granted Patent B2
US 12,230,031 · App. 18/331,549 · Granted Feb 18, 2025

Advanced video data stream extraction and multi-resolution video transmission

Inventors: Robert Skupin (Berlin, DE); Cornelius Hellge (Berlin, DE); Benjamin Bross (Berlin, DE); Thomas Schierl (Berlin, DE); Yago Sánchez De La Fuente (Berlin, DE); Karsten Suehring (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: Dolby Video Compression, LLC
G06V20/46G06T7/12G06V20/49H04N19/59
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,230,031
App. No.
18/331,549
Granted
Feb 18, 2025
Kind
B2
Abstract

A concept for a video data stream extraction is presented which is more efficient namely which is, for example, able to more efficiently deal with video content of a type unknown to the recipient with videos of different type differing, for instance, in view-port-to-picture-plane projection, etc., or which lessens the extraction process complexity. Further, a concept is described using which a juxtaposition of different versions of a video scene, the versions differing in scene resolution, may be provided more efficiently to a recipient.

Claims (32)

1. A non-transitory digital storage medium storing a video data stream comprising:

a sequence of slice portions, each slice portion having encoded thereinto a respective slice of a plurality of slices of a picture of a video, wherein each slice portion comprises a slice address indicating a location where, in a picture area of the video, the slice is located which the respective slice portion has encoded thereinto; and

extraction information for usage in an extraction of a reduced video data stream from the video data stream, the reduced video data stream having encoded thereinto a spatially smaller video corresponding to a spatial section of the video of the video data stream, the extraction involving confining the video data stream to slice portions having encoded thereinto any slice within the spatial section and amending the slice address so as to relate to a decreased picture area of the spatially smaller video, the extraction information comprising:

first information defining the spatial section within the picture area; and

second information explicitly signalling substitute slice addresses for a replacement of the slice addresses of the slice portions of the slices within the spatial section, the substitute slice addresses indicating, within the reduced video data stream, the location where, in the decreased picture area, the slices are located,

wherein the sequence of slice portions comprises: slice portions having encoded thereinto a slice within the spatial section and slice portions having encoded thereinto a slice external to the spatial section.

2. The non-transitory digital storage medium according to claim 1 , wherein the first information defines the spatial section within the picture area as being composed of a set of at least two sectional regions within each of which the video is encoded into the video data stream independent from outside the respective sectional region, wherein none of the plurality of slices crosses borders of any of the at least two sectional regions.

3. The non-transitory digital storage medium according to claim 1 , wherein the slice address of each slice portion of the sequence of slice portions of the video data stream one-dimensionally indexes a position of a coding start of the slice encoded into the respective slice portion along a first coding scan order which traverses the picture area, and the substitute for the replacement of the slice address of the respective slice portion one-dimensionally indexes the position of the coding start of the slice encoded into the respective slice portion along a second coding scan order which traverses the decreased picture area.

4. The non-transitory digital storage medium according to claim 3 , wherein the second information sequentially provides the substitute slice addresses for the slice portions following an order at which the slice portions occur in the video data stream.

5. The non-transitory digital storage medium according to claim 1 , wherein the extraction information further defines a shape and size of the decreased picture area, and further includes third information indicating whether the slice portions of slices within the spatial section are to be re-ordered.

6. The non-transitory digital storage medium according to claim 5 , wherein the extraction information defines the shape and the size of the decreased picture area by way of a size parameter in a parameter set substitute ought to be used to substitute a corresponding parameter set in the video data stream in the extraction of the reduced video data stream from the video data stream.

7. An apparatus for generating a video data stream configured to:

provide the video data stream with a sequence of slice portions, each slice portion having encoded thereinto a respective slice of a plurality of slices of a picture of a video, wherein each slice portion comprises a slice address indicating a location where, in a picture area of the video, the slice which the respective slice portion has encoded thereinto is located; and

provide the video data stream with extraction information for usage in an extraction of a reduced video data stream from the video data stream, the reduced video data stream having encoded thereinto a spatially smaller video corresponding to a spatial section of the video of the video data stream, the extraction involving confining the video data stream to slice portions having encoded thereinto any slice within the spatial section and amending the slice address so as to relate to a decreased picture area of the spatially smaller video, the extraction information comprising:

first information defining the spatial section within the picture area within which the video is encoded into the video data stream independent from outside the spatial section; and

second information explicitly signaling substitute slice addresses for a replacement of the slice addresses of the slice portions of the slices within the spatial section, the substitute slice address indicating, within the reduced video data stream, the location where, in the decreased picture area, the slices are located,

wherein the sequence of slice portions comprises: slice portions having encoded thereinto a slice within the spatial section and slice portions having encoded thereinto a slice external to the spatial section.

8. The apparatus according to claim 7 , wherein the first information defines the spatial section within the picture area as being composed of a set of at least two sectional regions within each of which the video is provided into the video data stream independent from outside the respective sectional region, wherein none of the plurality of slices crosses borders of any of the at least two sectional regions.

9. The apparatus according to claim 7 , wherein the slice address of each slice portion of the sequence of slice portions of the video data stream one-dimensionally indexes a position of a coding start of the slice encoded into the respective slice portion along a first coding scan order which traverses the picture area, and the substitute for the replacement of the slice address of the respective slice portion one-dimensionally indexes the position of the coding start of the slice encoded into the respective slice portion along a second coding scan order which traverses the decreased picture area.

10. The apparatus according to claim 9 , wherein the second information sequentially provides the substitute slice addresses for the slice portions following an order at which the slice portions occur in the video data stream.

11. The apparatus according to claim 7 , wherein the extraction information further defines a shape and size of the decreased picture area, and further includes third information indicating whether the slice portions of slices within the spatial section are to be re-ordered.

12. The apparatus according to claim 11 , wherein the extraction information defines the shape and the size of the decreased picture area by way of a size parameter in a parameter set substitute ought to be used to substitute a corresponding parameter set in the video data stream in the extraction of the reduced video data stream from the video data stream.

13. An apparatus for extracting from a video data stream having encoded thereinto the video a reduced video data stream having encoded thereinto a spatially smaller video, the video data stream comprising a sequence of slice portions, each slice portion having encoded thereinto a respective slice of a plurality of slices of a picture of a video, wherein each slice portion comprises a slice address indicating a location where, in a picture area of the video, the slice which the respective slice portion has encoded thereinto is located, wherein the apparatus is configured to:

read extraction information from the video data stream;

derive from first information in the extraction information a spatial section within the picture area, wherein the reduced video data stream is confined to slice portions having encoded thereinto any slice within the spatial section;

derive from the extraction information second information explicit signaling substitute slice addresses; and

replace the slice addresses of the slice portions of the slices within the spatial section using the substitute slice addresses explicit signaled by the extraction information, the substitute slice addresses indicating, within the reduced video data stream, the location where, in a decreased picture area of the spatially smaller video, the slices are located,

wherein the sequence of slice portions comprises: slice portions having encoded thereinto a slice within the spatial section and slice portions having encoded thereinto a slice external to the spatial section.

14. The apparatus according to claim 13 , wherein the first information defines the spatial section within the picture area as being composed of a set of at least two sectional regions within each of which the video is provided into the video data stream independent from outside the respective sectional region, wherein none of the plurality of slices crosses borders of any of the at least two sectional regions.

15. The apparatus according to claim 13 , wherein the slice address of each slice portion of the sequence of slice portions of the video data stream one-dimensionally indexes a position of a coding start of the slice encoded into the respective slice portion along a first coding scan order which traverses the picture area, and the substitute for the replacement of the slice address of the respective slice portion one-dimensionally indexes the position of the coding start of the slice encoded into the respective slice portion along a second coding scan order which traverses the decreased picture area.

16. The apparatus according to claim 15 , wherein the second information sequentially provides the substitute slice addresses for the slice portions following an order at which the slice portions occur in the video data stream.

17. The apparatus according to claim 13 , wherein the extraction information further defines a shape and size of the decreased picture area, and further includes third information indicating whether the slice portions of slices within the spatial section are to be re-ordered.

Assignments (2)
CHANGE OF NAME Recorded Jan 30, 2026
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 074536/0781 →
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0772 →
Priority Claims (1)
EP 17161917 · Mar 20, 2017 · regional
Continuity (4)
Continuation 17840269 · Jun 14, 2022
Continuation 16576051 · Sep 19, 2019
Continuation PCTEP2018056788 · Mar 19, 2018
Related Publication 20230316752A1 · Oct 5, 2023
References Cited (39)
US 20070200923A1 · Eleftheriadis et al. · 2007 [cited by applicant]
US 20090187960A1 · Lee et al. · 2009 [cited by applicant]
US 20120171648A1 · Chen et al. · 2012 [cited by applicant]
US 20140301464A1 · Wu et al. · 2014 [cited by applicant]
US 20150016504A1 · Auyeung et al. · 2015 [cited by applicant]
US 20150023409A1 · Schierl et al. · 2015 [cited by applicant]
US 20150023434A1 · Schierl et al. · 2015 [cited by applicant]
US 20160100196A1 · Wu et al. · 2016 [cited by applicant]
US 20170163994A1 · Sanchez De La Fuente · 2017 [cited by examiner]
US 20170201766A1 · Choi · 2017 [cited by applicant]
US 20170318288A1 · Wu et al. · 2017 [cited by applicant]
CN 103299627A · 2013 [cited by applicant]
CN 104685893A · 2015 [cited by applicant]
CN 104737541A · 2015 [cited by applicant]
JP 2015507899A · 2015 [cited by applicant]
KR 20090079838A · 2009 [cited by applicant]
KR 20160019091A · 2016 [cited by applicant]
KR 20170020780A · 2017 [cited by applicant]
WO 2014001573A1 · 2014 [cited by applicant]
WO 2015009676A1 · 2015 [cited by applicant]
WO 2016026526A · 2016 [cited by applicant]
WO WO2016026526A2 · 2016 [cited by examiner]
WO 2016180486A1 · 2016 [cited by applicant]
Amon, P., Sapre, M. and Hutter, A., May 2012. Compressed domain stitching of HEVC streams for video conferencing applications. In 2012 19th International Packet Video Workshop (PV) (pp. 36-40). IEEE. [cited by examiner]
Sanchez, Y., Globisch, R., Schierl, T. and Wiegand, T., Jan. 2014. Low complexity cloud-video-mixing using HEVC. In 2014 IEEE 11th Consumer Communications and Networking Conference (CCNC) (pp. 213-218). IEEESanchez, Y.,… [cited by examiner]
Office Action issued in corresponding Chinese Patent Application 202211365637.1 dated Nov. 22, 2023. [cited by applicant]
Skupin, Robert, et al., “MCTS extraction with slice reordering”, (JCTVC-AA0029), JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 27th Meeting: Hobart, AU, (Mar. 21, 2017). [cited by applicant]
ITU-T, “[Uploaded in 16 parts] Series H: Audiovisual and Multimedia Systems”, XP055224026, CH Retrieved from the Internet: URL: https://www.itu.int/rec/dologin_pub.asp?lang=e&id= T-REC-H.265-201504-I !! PDF-E&Type=items… [cited by applicant]
Skupin, Robert, et al., “On MCTS extraction”, JCT-VC Meeting; Oct. 14, 2016; Chengdu; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG. 16) URL: http://wftp3.itu.int/av-arch/jctvc-site/, … [cited by applicant]
Deshpande, Sachin, “On Motion-Constrained Tile Sets Extraction Information Set SEI”, Joint Collaborative Team on Video Coding (JCT-VC), Jan. 15, 2017, [JCTVC-Z0037] (version 2), 7 pp. [cited by applicant]
Skupin, Robert, et al., “On MCTS extraction”, (JCTVC-X0039), JCT-VC of ITU-TSG16 WP3 and ISO/IEC JTC1/SC29/WG11 24th Meeting: Geneva, CH, (published on May 29, 2016), 5 pp. [cited by applicant]
Boyce, Jill et al., “JCTVC-Z1005”,—Hevc Additional Supplemental Enhancement Information (Draft 1 ), Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 26th Meeting: Gene… [cited by applicant]
ITU-T, “Series H: Audiovisual and Multimedia Systems”, Recommendation H.265 (Apr. 2013), Infrastructure of audiovisual services—Coding of moving video, High efficiency video coding, Online: http://www.itu.int/rec/T-REC-… [cited by applicant]
Office Action (Decision to Grant) issued in corresponding Japanese Patent Application 2021-1647 41 dated May 23, J023, with English translation. [cited by applicant]
Office Action (Notice of Allowance) issued in corresponding Chinese Patent Application 201880033197.5 dated Aug. 15, 2022. [cited by applicant]
Office Action (Notice of Preliminary Rejection) issued in corresponding Korean Patent Application 2022-7020901 dated Aug. 8, 2022, with English translation. [cited by applicant]
Office Action issued in corresponding Japanese Patent Application 2021-1647 41 dated Aug. 2, 2022, with English translation. [cited by applicant]
Notice of Allowance issued in corresponding U.S. Appl. No. 16/576,051 dated Mar. 7, 2022. [cited by applicant]
Office Action (Notice of Allowance) issued in corresponding Korean Patent Application 10-2022-7020901 dated Apr. 5, 2023. [cited by applicant]