IP Library › Granted Patent US 12,212,752
Granted Patent B2
US 12,212,752 · App. 17/632,238 · Granted Jan 28, 2025

Image processing device and method

Inventor: Takeshi Tsukuba (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
H04N19/126H04N19/48H04N19/61H04N19/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,212,752
App. No.
17/632,238
Granted
Jan 28, 2025
Kind
B2
Abstract

Inverse quantization is performed on a quantization coefficient using a quantization parameter depending on whether a transform skip is to be applied, inverse coefficient transformation is performed on a transform coefficient generated by inverse quantization to generate a predicted residual that is a residual between an image and a predicted image of the image when the transform skip is not to be applied, and the inverse coefficient transformation is skipped when the transform skip is to be applied. This can be applied to, for example, an image processing device, an image coding device, an image decoding device, a transmission device, a reception device, a transmission/reception device, an information processing device, an imaging device, a reproduction device, an electronic apparatus, an image processing method, an information processing method, or the like.

Claims (70)

1. An image processing device comprising:

at least one processor; and

at least one memory including computer program, wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

perform inverse quantization on a quantization coefficient;

normalize a transform coefficient generated by the inverse quantization using a scaling parameter depending on whether a transform skip is to be applied, wherein the transform coefficient is normalized using the scaling parameter that reflects the decompression processing when the transform skip is to be applied, and

perform inverse coefficient transformation on the transform coefficient normalized to generate a predicted residual that is a residual between an image and a predicted image of the image when the transform skip is not to be applied, and skip the inverse coefficient transformation without performing the decompression processing when the transform skip is applied.

2. The image processing device according to claim 1 , wherein

the transform coefficient is normalized using the scaling parameter that does not reflect decompression processing in the transform skip when the transform skip is not to be applied, and

the inverse coefficient transformation is performed on the transform coefficient normalized using the scaling parameter that does not reflect the decompression processing when the transform skip is not to be applied.

3. The image processing device according to claim 2 , wherein

a component determined on the basis of a transform block size of the scaling parameter is set to “0” when the transform skip is to be applied.

4. The image processing device according to claim 2 , wherein

a component determined on the basis of the transform block size of the scaling parameter is set to a value according to a quantization matrix when the transform skip is to be applied.

5. The image processing device according to claim 4 , wherein

the value according to the quantization matrix is “−4”.

6. An image processing method comprising:

performing inverse quantization on a quantization coefficient;

normalizing a transform coefficient generated by the inverse quantization using a scaling parameter depending on whether transform skip is to be applied, wherein the transform coefficient is normalized using the scaling parameter that reflects decompression processing in the transform skip when the transform skip is to be applied; and

performing inverse coefficient transformation on the normalized transform coefficient to generate a predicted residual that is a residual between an image and a predicted image of the image when the transform skip is not to be applied and skipping the inverse coefficient transformation without performing the decompression processing when the transform skip is to be applied.

7. An image processing device comprising:

at least one processor; and

at least one memory including computer program, wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

perform coefficient transformation on a predicted residual that is a residual between an image and a predicted image of the image to generate a transform coefficient when a transform skip is not to be applied, and skip the coefficient transformation without performing decompression processing when the transform skip is to be applied;

perform quantization on the transform coefficient when the transform skip is not to be applied and perform the quantization on the predicted residual when the transform skip is to be applied; and

normalize a quantization coefficient generated using a scaling parameter depending on whether the transform skip is to be applied, wherein the quantization coefficient is normalized using the scaling parameter reflecting the decompression processing when the transform skip is to be applied.

8. The image processing device according to claim 7 , wherein

the coefficient transformation is performed on the predicted residual when the transform skip is not to be applied, and

wherein the quantization coefficient is normalized using the scaling parameter that does not reflect the decompression processing in the transform skip when the transform skip is not to be applied.

9. The image processing device according to claim 8 , wherein

a component determined on the basis of the transform block size of the scaling parameter is set to a value according to a quantization matrix when the transform skip is to be applied.

10. The image processing device according to claim 9 , wherein

the value according to the quantization matrix is “−4”.

11. An image processing method comprising:

performing coefficient transformation on a predicted residual that is a residual between an image and a predicted image of the image to generate a transform coefficient when a transform skip is not to be applied and skipping the coefficient transformation without performing decompression processing when the transform skip is to be applied;

performing quantization on the transform coefficient when the transform skip is not to be applied and performing the quantization on the predicted residual when the transform skip is to be applied; and

normalizing a quantization coefficient generated by the quantization using a scaling parameter depending on whether the transform skip is to be applied, wherein the quantization coefficient is normalized using the scaling parameter reflecting the decompression processing when the transform skip is to be applied.

12. An image processing device comprising:

at least one processor; and

at least one memory including computer program, wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

perform inverse quantization on a quantization coefficient;

normalize a transform coefficient generated by the inverse quantization using a scaling parameter depending on whether a transform skip is to be applied,

perform inverse coefficient transformation on the transform coefficient normalized to generate a predicted residual that is a residual between an image and a predicted image of the image when the transform skip is not to be applied, and skip the inverse coefficient transformation when the transform skip is applied, and

wherein the transform coefficient is normalized using the scaling parameter reflecting normalization processing for the predicted residual when the transform skip is to be applied.

13. The image processing device according to claim 12 , wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

normalize the predicted residual when the transform skip is not to be applied.

14. The image processing device according to claim 13 , wherein

the transform coefficient is normalized using the scaling parameter including a value according to inverse quantization and a scaling list and a value according to application of inverse-dependence quantization when the transform skip is to be applied.

15. The image processing device according to claim 14 , wherein

the value according to inverse quantization and the scaling list is “10”.

16. An image processing method comprising:

performing inverse quantization on a quantization coefficient;

normalizing a transform coefficient generated by the inverse quantization using a scaling parameter depending on whether transform skip is to be applied;

performing inverse coefficient transformation on the normalized transform coefficient to generate a predicted residual that is a residual between an image and a predicted image of the image when the transform skip is not to be applied and skipping the inverse coefficient transformation when the transform skip is to be applied,

wherein the quantization coefficient is normalized using the scaling parameter reflecting normalization processing for the predicted residual when the transform skip is to be applied.

17. An image processing device comprising:

at least one processor; and

at least one memory including computer program, wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

perform coefficient transformation on a predicted residual that is a residual between an image and a predicted image of the image to generate a transform coefficient when a transform skip is not to be applied, and skip the coefficient transformation when the transform skip is to be applied;

perform quantization on the transform coefficient when the transform skip is not to be applied and perform the quantization on the predicted residual when the transform skip is to be applied; and

normalize a quantization coefficient generated by the quantization using a scaling parameter depending on whether the transform skip is to be applied, wherein the quantization coefficient is normalized using the scaling parameter reflecting normalization processing for the transform coefficient when the transform skip is to be applied.

18. The image processing device according to claim 17 , wherein the at least one memory and the computer program are configured, with the at least one processor, to cause the image processing device to at least

normalize the transform coefficient when the transform skip is not to be applied.

19. The image processing device according to claim 18 , wherein

the transform coefficient is normalized using the scaling parameter including a value according to quantization and a scaling list and a value according to application of dependent quantization when the transform skip is to be applied.

20. The image processing device according to claim 19 , wherein

the value according to quantization and the scaling list is “10”.

21. An image processing method comprising:

performing coefficient transformation on a predicted residual that is a residual between an image and a predicted image of the image to generate a transform coefficient when a transform skip is not to be applied and skipping the coefficient transformation when the transform skip is to be applied;

performing quantization on the transform coefficient when the transform skip is not to be applied and performing the quantization on the predicted residual when the transform skip is to be applied; and

normalizing a quantization coefficient generated by the quantization using a scaling parameter depending on whether the transform skip is to be applied, wherein the quantization coefficient is normalized using the scaling parameter reflecting a normalization processing for the transform coefficient when the transform skip is to be applied.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2022
From: TSUKUBA, TAKESHI
To: SONY GROUP CORPORATION
Reel/Frame 058854/0617 →
Continuity (3)
Provisional Application 62901895 · Sep 18, 2019
Provisional Application 62896989 · Sep 6, 2019
Related Publication 20220224903A1 · Jul 14, 2022
References Cited (37)
US 9712830B1 · Horn · 2017 [cited by examiner]
US 11172226B2 · Francois · 2021 [cited by examiner]
US 20130272385A1 · Yu · 2013 [cited by examiner]
US 20150012708A1 · Rubinstein · 2015 [cited by examiner]
US 20150016537A1 · Karczewicz · 2015 [cited by applicant]
US 20150139304A1 · Sato · 2015 [cited by applicant]
US 20150189327A1 · Kondo · 2015 [cited by examiner]
US 20160323583A1 · Kadoto · 2016 [cited by examiner]
US 20170150182A1 · Carlavan · 2017 [cited by examiner]
US 20180167630A1 · Zhao · 2018 [cited by examiner]
US 20190104322A1 · Tsukuba · 2019 [cited by applicant]
US 20200045313A1 · Leleannec · 2020 [cited by examiner]
US 20210136415A1 · Hashimoto · 2021 [cited by examiner]
US 20210289205A1 · Saitou · 2021 [cited by examiner]
US 20210321111A1 · Tamse · 2021 [cited by examiner]
US 20220094932A1 · De Lagrange · 2022 [cited by examiner]
US 20220094983A1 · Krishnan · 2022 [cited by examiner]
US 20220182629A1 · Lai · 2022 [cited by examiner]
US 20220303567A1 · Jung · 2022 [cited by examiner]
US 20220321916A1 · Zhu · 2022 [cited by applicant]
CN 104662901A · 2015 [cited by applicant]
CN 109076226A · 2018 [cited by applicant]
JP 2014135739A · 2014 [cited by applicant]
JP 2017073602A · 2017 [cited by applicant]
WO WO2015012600A1 · 2015 [cited by applicant]
WO 2019003676A1 · 2019 [cited by applicant]
WO WO2019159697A1 · 2019 [cited by applicant]
WO WO2021107641A1 · 2021 [cited by applicant]
Karczewicz (Qualcomm) M et al: “Non-CE8: Minimum QP for Transform Skip Mode”, 15. JVET Meeting; Jul. 3, 2019-Jul. 12, 2019; Gothenburg; (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16 ), No. J… [cited by applicant]
Lou (Motorola) J et al: “Scaling list selection for transform skip mode”, 11. JCT-VC Meeting; Oct. 10, 2012-Oct. 19, 2012; Shanghai; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16 ),… [cited by applicant]
International Search Report and Written Opinion mailed on Nov. 24, 2020, received for PCT Application PCT/JP2020/033573, Filed on Sep. 4, 2020, 18 pages including English Translation. [cited by applicant]
Nguyen et al., “Non-CE8: Minimum Allowed QP for Transform Skip Mode”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-00405-v1, Jul. 3-12, 2019, 5 pages. [cited by applicant]
Karczewicz et al., “Non-CE8: Minimum QP for Transform Skip Mode”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-00919-v2, Jul. 3-12, 2019, 4 pages. [cited by applicant]
Albrecht et al., “Description of SDR, HDR, and 360°video Coding Technology Proposal by Fraunhofer HHI”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-J0014-v1, Apr. 10-20, 2018,… [cited by applicant]
Ma et al., “Lossless Coding for VVC”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O1061, Jul. 3-12, 2019, 18 pages. [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding”, Joint Video Experts Team(JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018,Oct. 3, 2018,JVET-K1001-v5,Full text. [cited by applicant]
Marta Karczewicz et al., “WD for Non-CE8: Minimum QP for Transform Skip Mode” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Jul. 3-12, 2019. [cited by applicant]