IP Library › Granted Patent US 12,341,992
Granted Patent B2
US 12,341,992 · App. 18/638,841 · Granted Jun 24, 2025

Apparatus, a method and a computer program for video coding and decoding

Inventor: Miska Hannuksela (Tampere, FI)
Assignee: Nokia Technologies Oy
H04N19/597H04N19/119H04N19/167H04N19/177H04N19/188H04N19/46H04N19/593H04N19/70H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,341,992
App. No.
18/638,841
Granted
Jun 24, 2025
Kind
B2
Abstract

Four or more bitstream versions of a same content are encoded and divided into segments of independently coded tile sets representing multiple spatial regions. First and second bitstreams include independently coded tile sets encoded at a first quality. Third and fourth bitstreams include independently coded tile sets encoded at a second quality. First and third bitstreams have first random access picture interval. Second and fourth bitstreams have second random access picture interval. Independently coded tile sets are grouped into multiple groups of collocated sub-picture tracks, only one of said tile sets per group is intended to be received and/or decoded per any segment. Instruction(s) are generated for merging tile sets of different spatial locations into coded picture(s), causing a tile set originating from a random access picture to be decoded as a tile set originating from a non-random-access picture when merged with a tile set originating from a non-random-access picture.

Claims (39)

1. A method comprising:

encoding or obtaining at least two bitstream versions of a same video content, wherein the same video content is divided into image segments, wherein each of the at least two bitstream version of the same video content is encoded at different quality, wherein a first bitstream comprises a random access picture at a first time instance and a second bitstream comprises a non-random access picture at said first time instance;

providing an image segment of the first bitstream comprising said random access picture with a video coding layer network abstraction layer (VCL NAL) unit type indicative of a random access image segment;

providing an image segment of the second bitstream comprising said non-random access picture with a VCL NAL unit type indicative of a non-random access image segment; and

generating at least one instruction for merging said at least two bitstreams, wherein said image segments from the first and the second bitstreams at said first time instance are merged into at least one coded picture.

2. The method according to claim 1 , comprising providing said at least one coded picture with a picture-level indication to indicate a picture type.

3. The method according to claim 1 , wherein generating the at least one instruction for merging comprises an instruction for rewriting a picture-level indication to indicate a non-random-access picture.

4. An apparatus comprising at least one processor, and at least one memory storing computer program code that, when executed by the processor, cause the apparatus to perform:

encoding or obtaining at least two bitstream versions of a same video content wherein the same video content is divided into image segments, wherein each of the at least two bitstream version of the same video content is encoded at different quality, wherein a first bitstream comprises a random access picture at a first time instance and a second bitstream comprises a non-random access picture at said first time instance;

providing an image segment of the first bitstream comprising said random access picture with a video coding layer network abstraction layer (VCL NAL) unit type indicative of a random access image segment;

providing an image segment of the second bitstream comprising said non-random access picture with a VCL NAL unit type indicative of a non-random access image segment; and

generating at least one instruction for merging said at least two bitstreams, wherein said image segments from the first and the second bitstreams at said first time instance are merged into at least one coded picture.

5. The apparatus according to claim 4 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform: providing said at least one coded picture with a picture-level indication to indicate a picture type.

6. The apparatus according to claim 4 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform: generating the at least one instruction for merging further comprises generating an instruction for rewriting a picture-level indication to indicate a non-random-access picture.

7. The apparatus according to claim 5 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform: encoding said picture-level indication in a picture header, a header parameter set, and/or a picture parameter set.

8. The apparatus according to claim 6 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform: encoding said picture-level indication in a picture header, a header parameter set, and/or a picture parameter set.

9. The apparatus according to claim 4 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform:

encapsulating the at least one instruction into a collector track, wherein the collector track extracts implicitly or explicitly coded video data from video tracks, and wherein the video tracks comprise coded video data of the first and second bitstreams.

10. A method comprising:

obtaining an image segment of a first bitstream with a video coding layer network abstraction layer (VCL NAL) unit type indicative of a random access image segment at a first time instance;

obtaining an image segment of a second bitstream with a VCL NAL unit type indicative of a non-random access image segment at said first time instance; and

merging said at least the first and second bitstreams, wherein image segments from the first and second bitstreams at said first time instance are merged into at least one coded picture and a picture-level indication to indicate a picture type is provided with said at least one coded picture.

11. The method according to claim 10 , wherein the merging further comprises rewriting the picture-level indication to indicate a non-random access picture.

12. The method according to claim 10 , further comprising encoding said picture-level indication in one or more of the following: a picture header, a header parameter set, or a picture parameter set.

13. The method according to claim 10 , wherein at least one VCL NAL unit type of the image segments in said at least one coded picture is allowed to differ from the picture type of said at least one coded picture.

14. An apparatus comprising at least one processor, and at least one memory storing computer program code that, when executed by the processor, cause the apparatus to perform:

obtaining an image segment of a first bitstream with a video coding layer network abstraction layer (VCL NAL) unit type indicative of a random access image segment at a first time instance;

obtaining an image segment of a second bitstream with a VCL NAL unit type indicative of a non-random access image segment at said first time instance; and

merging said at least the first and second bitstreams, wherein image segments from the first and second bitstreams at said first time instance are merged into at least one coded picture and a picture-level indication to indicate a picture type is provided with said at least one coded picture.

15. The apparatus according to claim 14 , wherein the merging further comprises rewriting the picture-level indication to indicate a non-random access picture.

16. The apparatus according to claim 15 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform:

encoding said picture-level indication in a picture header, a header parameter set, and/or a picture parameter set.

17. The apparatus according to claim 14 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform:

encoding said picture-level indication in a picture header, a header parameter set, and/or a picture parameter set.

18. The apparatus according to claim 14 , wherein at least one VCL NAL unit type of the image segments in said at least one coded picture is allowed to differ from the picture type of said at least one coded picture.

19. The apparatus according to claim 14 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform:

obtaining or inferring at least one instruction for merging said image segments into the at least one coded picture; and

processing the at least one instruction to form the at least one coded picture.

20. The apparatus according to claim 19 , wherein the at least one memory further stores computer program code that, when executed by the processor, cause the apparatus to perform: obtaining or inferring the at least one instruction from a collector track, wherein the collector track extracts implicitly or explicitly coded video data from video tracks, and wherein the video tracks comprise coded video data of the first and second bitstreams.

Priority Claims (1)
FI 20195001 · Jan 2, 2019 · national
Continuity (3)
Continuation 18121729 · Mar 15, 2023
Continuation 17419080
Related Publication 20240267560A1 · Aug 8, 2024
References Cited (27)
US 20170094288A1 · Hennuksela · 2017 [cited by applicant]
US 20170105004A1 · Chen · 2017 [cited by applicant]
US 20180103199A1 · Hendry et al. · 2018 [cited by applicant]
US 20210203942A1 · Choi · 2021 [cited by applicant]
EP 3349467A1 · 2018 [cited by applicant]
WO WO2014106692A1 · 2014 [cited by applicant]
WO WO2017140945A1 · 2017 [cited by applicant]
“Information technology—Dynamic Adaptive Streaming over HTTP (DASH)—Part 1: Media presentation Description and Segment Formats”, ISO/IEC 23009-1, Second Edition, May 15, 2014, 152 pages. [cited by applicant]
“Advanced Video Coding for Generic Audiovisual services”, Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving Video, Recommendation ITU-T H.264, Apr. 2017, 812 pages. [cited by applicant]
“High Efficiency Video Coding”, Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding ofmoving video, Recommendationn ITU-T H.265, Feb. 2018, 692 pages. [cited by applicant]
Schulzrinne et al., “RTP: A Transport Protocol for Real-Time Applications”, RFC 3550, network Working Group, Jul. 2003, pp. 1-104. [cited by applicant]
“Information Technology—Generic coding of moving pictures and associated audio information: Systems”, Series H: Audiovisual Anti Multimedia Systems, Infratructure of audiovisual services—Transmission multiplexing and sy… [cited by applicant]
“Information Technology—Coding of Audio-Visual Objects—Part 12: ISO Base Media File Format”, ISO/IEC 14496-12, Fifth Edition, Dec. 15, 2015, 248 pages. [cited by applicant]
“Information Technology—Coding of Audio-Visual Objects—Part 14: MP4 File Format”, ISO/IEC 14496-14, First edition, Nov. 15, 2003, 18 pages. [cited by applicant]
“Information Technology—Coding of Audio-Visual Objects—Part 15: Advanced Video Coding (AVC) File Format”, ISO/IEC 14496-15, First edition, Apr. 15, 2004, 29 pages. [cited by applicant]
“3rd Generation Partnershp Project; Technical Specification Group Services and System Aspects; Transparent end-to-end packed switched streaming services (PSS); 3GPP file format (3GP) (Releae 15)”, 3GPP TS 26.244, V15.0.… [cited by applicant]
“Video Coding for Low Bit Rate Communication”, Series H: Audiovisual and Mutlimedia Systems, Infrastructure of audiovisual services—Coding of moving Video, ITU-T Recommendation H.263, Jan. 2005, 226 pages. [cited by applicant]
“Parameter Values for Ultra-High Definition Television Systems for Production and International Programme Exchange”, Recommendation ITU-R BT.2020, Aug. 2012, 7 pages. [cited by applicant]
“Parameter Values for the HDTV Standards for Production and International Programme Exchange”, Recommendation ITU-R BT.709.6, Jun. 2015, 19 pages. [cited by applicant]
“3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Transparent end-to-end Packet switched Streaming Service (PSS); Protocols and codecs (Release 15)”, 3GPP TS 26.234, V15.1.0… [cited by applicant]
“3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Transparent end-to-end Packet-switched Streaming Service (PSS); Progressive Download and Dynamic Adaptive Streaming over HT… [cited by applicant]
Moats, “URN Syntax”, RFC 2141, Network Working Group, May 1997, pp. 1-8. [cited by applicant]
Berners-Lee et al., “Uniform Resource Identifier (URI): Generic Syntax”, RFC 3986, Network Working Group, Jan. 2005, pp. 1-61. [cited by applicant]
Wang et al., “WD 3 of ISO/IEC 23090-2 OMAF 2nd edition”, Systems, ISO/IEC JTC1/SC29/WG11 N17963-v1, Oct. 2018, 226 pages. [cited by applicant]
Sanchez et al., “Random access point period optimization for viewport adaptive tile based streaming of 360 video”, IEEE International Conference on Image Processing (ICIP), Sep. 17-20, 2017, pp. 1915-1919. [cited by applicant]
“Versatile Video Coding”, Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving Video, Recommendation ITU-T H.266, Aug. 2020, 516 pages. [cited by applicant]
“Information Technology—Coded representation of immersive media (MPEG-1)—Part 2: Ominidirectional media format”, ISO/IEC, JTC 1/SC29/WG11, ISO/IEC FDIS 23090-2:201x (E), Apr. 26, 2018, 182 pages. [cited by applicant]