IP Library Granted Patent US 12,200,217
Granted Patent B2
US 12,200,217 · App. 17/622,560 · Granted Jan 14, 2025

Image data encoding and decoding

Inventors: Stephen Mark Keating (Basingstoke, GB); Karl James Sharman (Basingstoke, GB); Adrian Richard Browne (Basingstoke, GB)
Assignee: Sony Group Corporation
H04N19/13H04N19/174H04N19/188H04N19/91
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,200,217
App. No.
17/622,560
Granted
Jan 14, 2025
Kind
B2
Abstract

Image data encoding apparatus comprises an entropy encoder for selectively encoding data items representing image data to be encoded by a first, context adaptive binary arithmetic coding (CABAC), encoding system or by a second, bypass, encoding system, so as to generate encoded binarized symbols; the image data representing one or more pictures of a picture sequence, each picture comprising two or more output data units representing respective sub-sections of the picture, each sub-section being decodable and reconstructable independently of other sub-sections in that picture and picture sequence; in which each sub-section is subject to a respective minimum compression ratio.

Claims (36)

1. An image data encoding apparatus, comprising:

an entropy encoder configured to selectively encode data items representing image data to be encoded by a first context adaptive binary arithmetic coding (CABAC) encoding system or by a second, bypass, encoding system, to generate encoded binarized symbols,

wherein the image data represents one or more pictures of a picture sequence, each picture including two or more output data units representing respective sub-sections of the picture, each sub-section being decodable and reconstructable independently of other sub-sections in that picture and picture sequence, and

wherein each sub-section is subject to a respective minimum compression ratio.

2. The image data encoding apparatus of claim 1 , wherein the entropy encoder is configured to generate an output data stream subject to a constraint defining an upper limit to a number of binarized symbols that is expressible by any individual output data unit relative to a size in bytes of that output data unit, and

wherein the entropy encoder is configured to apply the constraint to each output data unit, and to provide padding data, for each output data unit which does not meet the constraint, to increase the size in bytes of that output data unit in order to meet the constraint.

3. The image data encoding apparatus of claim 1 , wherein the second encoding system is a binary arithmetic coding system using a fixed 50% probability context model.

4. The image data encoding apparatus of claim 1 , wherein the entropy encoder comprises:

a detector configured to detect, at a predetermined stage relative to the encoding of a current output data unit, whether a constraint will be met by the current output data unit; and

a padding data generator configured to generate and insert into a current output data unit sufficient padding data so that the output data unit including the inserted padding data meets a constraint.

5. The image data encoding apparatus of claim 4 , wherein the predetermined stage is an end of encoding the current output data unit.

6. The image data encoding apparatus of claim 1 , wherein the sub-sections respectively represent a sub-section from a list consisting of: sub-pictures, slices and tiles.

7. The image data encoding apparatus of claim 1 , wherein:

(i) a sub-picture represents a region of the picture,

(ii) a slice represents a part, in a raster order, of a picture, a sub-picture or a tile and is constrained to be encapsulated within a respective network abstraction layer (NAL) unit, and

(iii) a tile represents a part, defining respective horizontal and vertical boundaries in a grid arrangement, of a picture, a sub-picture or a slice and is not constrained to be encapsulated within a respective NAL unit.

8. The image data encoding apparatus of claim 1 , wherein the apparatus is operable according to an encoding profile selected from a set of encoding profiles, each encoding profile defining at least a set of constraints on the image data to be encoded and/or an output data stream, in which the image data encoding apparatus is configured to apply a respective encoding profile to each sub-section of a picture independent of the encoding profile applied to any other sub-sections of that picture.

9. The image data encoding apparatus of claim 8 , wherein the encoding profiled for a sub-section is defined by sub-section header data associated with that sub-section.

10. A video storage, capture, transmission or reception apparatus comprising apparatus according to claim 1 .

11. An image data decoder configured to decode by circuitry a data signal stored on a non-transitory computer readable medium produced by the image data encoding apparatus of claim 1 .

12. The image data encoding apparatus of claim 1 , wherein the minimum compression ratio is associated with each sub-section by allocating a proportion of a maximum luma sample rate for the whole picture by associating a level fraction value with each sub-section.

13. An image data encoding method, comprising:

selectively encoding data items representing image data to be encoded by a first context adaptive binary arithmetic coding (CABAC) encoding system or by a second, bypass, encoding system, to generate encoded binarized symbols, the image data representing one or more pictures of a picture sequence, each picture comprising two or more output data units representing respective sub-sections of the picture, each sub-section being decodable and reconstructable independently of other sub-sections in that picture and picture sequence; and

generating an output data stream, wherein

the generating is subject to a constraint defining a respective minimum compression ratio applicable to each sub-section.

14. An image decoding method comprising decoding via circuitry a data signal stored on a non-transitory computer readable medium produced by the image encoding method of claim 13 .

15. The image data encoding method of claim 13 , wherein the minimum compression ratio is associated with each sub-section by allocating a proportion of a maximum luma sample rate for the whole picture by associating a level fraction value with each sub-section.

16. An image decoding method comprising decoding via circuitry a data signal stored on a non-transitory computer readable medium produced by the image encoding method of claim 15 .

17. An image data decoding apparatus, comprising:

a decoder configured to selectively decode data items representing image data to be decoded by a first context adaptive binary arithmetic coding (CABAC) decoding system or by a second, bypass, decoding system, so as to generate decoded binarized symbols depending on encoded image data stored in a non transitory storage medium in communication with the image data decoding apparatus,

wherein the image data represents one or more pictures of a picture sequence, each picture comprising including two or more output data units representing respective sub-sections of the picture, each sub-section being decodable and reconstructable independently of other sub-sections in that picture and picture sequence, and

wherein each sub-section is subject to a respective minimum compression ratio; and

circuitry configured to output the decoded image data.

18. The image decoding apparatus as claimed in claim 17 , wherein the minimum compression ratio is associated with each sub-section by allocating a proportion of a maximum luma sample rate for the whole picture by associating a level fraction value with each sub-section.

19. The image decoding apparatus as claimed in claim 18 , wherein the level fraction value in a proportion of two-hundred-and-fifty-six.

20. A video storage, capture, transmission or reception apparatus comprising apparatus according to claim 18 .

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2024
From: CUNIO BROWNE SERVICES LIMITED
To: SONY EUROPE B.V.
Reel/Frame 068541/0713 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2024
From: SONY EUROPE B.V.
To: SONY CORPORATION
Reel/Frame 068542/0001 →
CHANGE OF NAME Recorded Sep 10, 2024
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 068917/0340 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2024
From: KEATING, STEPHEN MARK; SHARMAN, KARL JAMES
To: SONY EUROPE BV
Reel/Frame 068499/0425 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 5, 2024
From: BROWNE, ADRIAN RICHARD
To: CUNIO BROWNE SERVICES LIMITED
Reel/Frame 068499/0497 →
Priority Claims (2)
GB 1909143 · Jun 25, 2019 · national
GB 1919471 · Dec 31, 2019 · national
Continuity (1)
Related Publication 20220360782A1 · Nov 10, 2022
References Cited (43)
US 20030164975A1 · Aoyagi · 2003 [cited by applicant]
US 20080130756A1 · Sekiguchi · 2008 [cited by applicant]
US 20100135416A1 · Huang · 2010 [cited by applicant]
US 20120300839A1 · Sze · 2012 [cited by applicant]
US 20130315297A1 · Sasai · 2013 [cited by applicant]
US 20140286417A1 · Gamei · 2014 [cited by examiner]
US 20140307807A1 · Gamei · 2014 [cited by applicant]
US 20150092833A1 · Ku · 2015 [cited by applicant]
US 20150172692A1 · Yang et al. · 2015 [cited by applicant]
US 20150172693A1 · Yang et al. · 2015 [cited by applicant]
US 20160142740A1 · Sharman · 2016 [cited by applicant]
CN 101212676A · 2008 [cited by applicant]
CN 103096074A · 2013 [cited by applicant]
CN 103621088A · 2014 [cited by applicant]
CN 106537920A · 2017 [cited by applicant]
CN 109257048A · 2019 [cited by applicant]
CN 109923861A · 2019 [cited by applicant]
GB 2523993A · 2015 [cited by applicant]
JP 2022517356 · 2020 [cited by applicant]
WO WO2013068733A1 · 2013 [cited by applicant]
WO WO2018172234A2 · 2018 [cited by applicant]
H.265; Series H: Audiovisual and Multimedia Systems; Apr. 2013; “D2” (Year: 2013). [cited by examiner]
JVT-F049r1; Hannuksela et al.; Dec. 5, 2002;“D2” (Year: 2002). [cited by examiner]
International Search report issued Dec. 3, 2020 in PCT/GB2020/051538 filed Jun. 24, 2020, 7 pages. [cited by applicant]
“High efficiency video coding”, International Telecommunication Union, H.265, Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, XP044008743, 2013, 317 total pag… [cited by applicant]
Nakagami, O., et al., “AHG11: Suggested editorial improvement on TMCTS-SEI”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, XP030240513, 2014, pp. 1-7. [cited by applicant]
Sharman, K., et al., “Independent Subpictures”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, XP030223527, 2020, pp. 1-5. [cited by applicant]
Hattori, S., et al., “HLS: Extensions to Motions-constrained tile sets SEI message”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, XP030237887, 2013, pp. 1-4. [cited by applicant]
Zhou, M., “AHG4: Enable parallel decoding with tiles”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, XP030235388, 2012, pp. 1-7. [cited by applicant]
Sunna, P., et al., “Spatial substream definition”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, XP030114083, 2013, pp. [cited by applicant]
“Versatile video coding”, ITU-T, H.266, , Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, 2020, 516 total pages. [cited by applicant]
Viscito, E., “Macroblock level limit on the number of binary symbols in CABAC”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO0IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), 2002, pp. 1-6. [cited by applicant]
Wiegand, T., “Joint Final Committee Draft )JFCD) of Joint Video Specification (ITU-T Rec. H.264 I ISO/IEC 14496-10 AVC)”, Joint Video Team (JVT) of ISO-IEC MPEG and ITU-T VCEG, 2002, 207 total pages. [cited by applicant]
Hannuksela, M., “Corrections on Sub-Sequence SEI Messages Proposed changes relative to JVT-E1 46d37”, Joint Video Team (JVT) of ISO-IEC MPEG and ITU-T VCEG, 2002, 242 total pages. [cited by applicant]
Browne, A., et al., “Slice/tile level CABAC zero-word constraints”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, 2019, 4 total pages. [cited by applicant]
Yu, Y., “Limiting the Bin to Bit Expansion Ration of CABAC”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC/1/SC 29/WG 11, 2012, pp. 1-4. [cited by applicant]
Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Incheon, KR, Apr. 18-26, 2013 JCTVC-M0235-v.2,Apr. 26, 2013,Yongjun Wu et al,Motion-constrained tile sets … [cited by applicant]
Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 16th Meeting: San José, US, Jan. 9-17, 2014JCTVC-P0051 ,Jan. 16, 2014,Sally Hattori et al,HLS: Extensions to Temoral Mot… [cited by applicant]
Recommendation ITU-TH.265,Apr. 13, 2013,ITU,High efficiency video coding,H.265 04/13, ITU-, 199. [cited by applicant]
Suzuki (Sony) T: “On MinCR”, 16. JCT-VC Meeting; Jan. 9, 2014-Jan. 17, 2014; San Jose; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/ WG11 and ITU-T SG.16), No. JCTVC-P0044 Jan. 15, 2014 (Jan. 15, 2014)… [cited by applicant]
Flynn D et al: “HEVC Range Extensions Draft 5”, 15. JCT-VC Meeting; Oct. 23, 2013-Nov. 1, 2013; Geneva; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JCTVC-01005 Dec. 17, 2013… [cited by applicant]
JVT: “Text of Joint FCD for Joint Video Specification”, 4. JVT Meeting; 61. MPEG Meeting; Jul. 22, 2002-Jul. 26, 2002; Klagenfurt,AT; (Joint Video Team of ISO/ IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVT-D157, Jul. 26,… [cited by applicant]
Flynn D et al: “HEVC Range Extensions Draft 5”, 15th JCT-VC Meeting, Oct. 23, 2013-Nov. 1, 2013; (Joint Collaborative Team On Video Coding of 1S0/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JCTVC-01005, Dec. 27, 2013 (Dec.… [cited by applicant]