IP Library Granted Patent US 10,728,578
Granted Patent B2
US 10,728,578 · App. 15/912,825 · Granted Jul 28, 2020

Bias minimization for successive image reconstruction based on embedded codec circuitry

Inventors: Mohammed Golam Sarwer (San Jose, CA); Ali Tabatabai (San Jose, CA)
Assignee: SONY CORPORATION
H04N19/60H04N19/124H04N19/65H04N19/91
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,728,578
App. No.
15/912,825
Granted
Jul 28, 2020
Kind
B2
Abstract

An embedded decoder circuitry for bias minimization in successively reconstructed images from a decoded image obtained from an encoded input image, includes an on-chip memory and a bias control circuit communicatively coupled to the on-chip memory. The on-chip memory stores transform-domain compressed data that includes a plurality of transform blocks of an input image. The bias control circuit is configured to determine a set of conditions associated with each transform coefficient in a transform block of the encoded input image and modify a set of transform coefficients from the plurality of transform coefficients in the transform block based on determination of the set of conditions. The bias control circuit is further configured to generate a decoded image that exhibits a reduced error propagation in successively reconstructed images with respect to the input image, based on inverse transformation of each transform block of the encoded input image.

Claims (78)

1. An embedded decoder circuitry, comprising:

a memory configured to store transform-domain compressed data that comprises a plurality of transform blocks of an input image encoded by an embedded encoder circuitry; and

a bias control circuit communicatively coupled to the memory, wherein the bias control circuit is configured to:

determine a set of conditions for each transform coefficient of a plurality of transform coefficients in each transform block of the plurality of transform blocks based on a set of parameters associated with each transform coefficient, wherein

the set of condition includes a first condition,

for the first condition,

a transform coefficient of the plurality of transform coefficients in a specific transform block of the plurality of transform blocks is associated with a non-zero quantization parameter, and

an absolute value of the transform coefficient is zero;

modify a set of transform coefficients from the plurality of transform coefficients in each transform block based on the determined set of conditions associated with each transform coefficient;

generate a plurality of modified transform blocks based on the modification of the set of transform coefficients in each transform block;

generate a decoded image based on inverse transformation of each modified transform block of the plurality of modified transform blocks of the encoded input image, wherein

the decoded image exhibits a minimum value of peak signal to noise ratio (PSNR) in successively reconstructed images with respect to the input image; and

compensate for a bias in the successively reconstructed images, wherein the bias is compensated based on the decoded image.

2. The embedded decoder circuitry according to claim 1 , wherein the bias control circuit is further configured to execute a sequential decoding scheme on a bit-stream of compressed image data of the encoded input image based on header information in the bit-stream of compressed image data.

3. The embedded decoder circuitry according to claim 2 , wherein

the header information indicates application of a sequential encoding scheme at encoding of a plurality of patches of the input image to obtain the bit-stream of compressed image data of the encoded input image, and

the sequential decoding scheme includes a sequential application of entropy decoding followed by inverse discrete pulse code modulation (DPCM), inverse quantization, and inverse transformation.

4. The embedded decoder circuitry according to claim 3 , wherein the bias control circuit is further configured to:

modify the set of transform coefficients for reconstruction of the input image; and

compensate for the bias in the successively reconstructed images, wherein the bias is compensated after an inverse quantization and before an inverse transform in the sequential decoding scheme.

5. The embedded decoder circuitry according to claim 1 , wherein the bias control circuit is further configured to receive, from an inverse quantizer (IQ), the transform-domain compressed data after sequential application of entropy decoding followed by inverse discrete pulse code modulation (DPCM) and inverse quantization.

6. The embedded decoder circuitry according to claim 1 , wherein each transform block comprises a luma block and at least one chroma block of the encoded input image, wherein each transform block corresponds to a patch of the input image.

7. The embedded decoder circuitry according to claim 1 , wherein the bias control circuit is further configured to minimize a PSNR shift in the successively reconstructed images based on the modification of the set of transform coefficients.

8. The embedded decoder circuitry according to claim 1 , wherein the modification of the set of transform coefficients corresponds to a reconstruction of the specific transform block that corresponds to a patch of the input image.

9. The embedded decoder circuitry according to claim 1 , wherein the set of transform coefficients corresponds to coefficient values that require a modification for a bias minimization and image refinement in the successively reconstructed images.

10. The embedded decoder circuitry according to claim 1 , wherein the set of parameters associated with each transform coefficient comprises at least one of a quantization parameter, the absolute value of the transform coefficient, a transform type for the specific transform block, or a specific coefficient distribution pattern of a luma block and at least one chroma block.

11. The embedded decoder circuitry according to claim 1 , wherein

the bias control circuit is further configured to generate a pseudo random number for the transform coefficient based on the first condition.

12. The embedded decoder circuitry according to claim 11 , wherein the pseudo random number is generated based on a number of refinement bits for the specific transform block.

13. The embedded decoder circuitry according to claim 11 , wherein the pseudo random number is a positive number when a number of refinement bits for the specific transform block is odd, and wherein the pseudo random number is a negative number when the number of refinement bits for the specific transform block is even.

14. The embedded decoder circuitry according to claim 1 , wherein

the set of conditions further includes a second condition,

for the second condition,

the transform coefficient is associated with the non-zero quantization parameter,

the absolute value of the transform coefficient is zero, and

the specific transform block that includes the transform coefficient is obtained through column Discrete Cosine Transformation (DCT) of a patch of the input image, and

the bias control circuit is further configured to generate a pseudo random number for the transform coefficient based on the second condition.

15. The embedded decoder circuitry according to claim 1 , wherein

the set of conditions further includes a third condition,

in the third condition,

the transform coefficient is associated with the non-zero quantization parameter,

the absolute value of the transform coefficient is zero, and

at least one of a chroma block and a luma block associated with the transform coefficient exhibits a specific coefficient distribution pattern, and

the bias control circuit is further configured to generate a pseudo random number for the transform coefficient based on the third condition.

16. The embedded decoder circuitry according to claim 1 , wherein the modification of the set of transform coefficients is based on a pseudo random number associated with the set of transform coefficients.

17. The embedded decoder circuitry according to claim 1 , wherein the bias control circuit is further configured to increment the absolute value of the transform coefficient by a specific value when the transform coefficient is associated with the non-zero quantization parameter and the absolute value of the transform coefficient is a non-zero value.

18. The embedded decoder circuitry according to claim 17 , wherein the bias control circuit is further configured to retain a sign value of the transform coefficient in the incremented transform coefficient.

19. The embedded decoder circuitry according to claim 1 , wherein

the bias control circuit is further configured to compensate for a PSNR shift in the plurality of transform blocks associated with successive decompression of the encoded input image based on the modification of the set of transform coefficients in each of the plurality of transform blocks, and

the PSNR shift is compensated for each color block in a color space associated with the encoded input image.

20. A method, comprising:

in an embedded decoder circuitry that comprises a memory configured to store transform-domain compressed data and a bias control circuit communicatively coupled to the memory, wherein the transform-domain compressed data comprises a plurality of transform blocks of an input image encoded by an embedded encoder circuitry:

determining, by the bias control circuit, a set of conditions for each transform coefficient of a plurality of transform coefficients in each transform block of the plurality of transform blocks based on a set of parameters associated with each transform coefficient, wherein

the set of condition includes a first condition,

in the first condition,

a transform coefficient of the plurality of transform coefficients in a specific transform block of the plurality of transform blocks is associated with a non-zero quantization parameter, and

an absolute value of the transform coefficient is zero;

modifying, by the bias control circuit, a set of transform coefficients from the plurality of transform coefficients in each transform block based on the determined set of conditions associated with each transform coefficient,

generating, by the bias control circuit, a plurality of modified transform blocks based on the modification of the set of transform coefficients in each transform block;

generating, by the bias control circuit, a decoded image based on inverse transformation of each modified transform block of the plurality of modified transform blocks of the encoded input image, wherein

the decoded image exhibits a minimum value of peak signal to noise ratio (PSNR) in successively reconstructed images with respect to the input image; and

compensating, by the bias control circuit, for a bias in the successively reconstructed images, wherein the bias is compensated based on the decoded image.

21. The method according to claim 20 , further comprising

generating, by the bias control circuit, a pseudo random number for the transform coefficient based on the first condition.

22. The method according to claim 20 , wherein

the set of conditions further includes a second condition,

in the second condition,

the transform coefficient is associated with the non-zero quantization parameter,

the absolute value of the transform coefficient is zero, and

the specific transform block that includes the transform coefficient is obtained through column Discrete Cosine Transformation (DCT) of a patch of the input image, and

the method further comprising generating, by the bias control circuit, a pseudo random number for the transform coefficient based on the second condition.

23. The method according to claim 20 , wherein

the set of conditions further includes a third condition,

in the third condition,

the transform coefficient is associated with the non-zero quantization parameter,

the absolute value of the transform coefficient is zero, and

at least one of a chroma block and a luma block associated with the transform coefficient exhibits a specific coefficient distribution pattern, and

the method further comprising generating, by the bias control circuit, a pseudo random number for the transform coefficient based on the third condition.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2021
From: SONY CORPORATION
To: SATURN LICENSING LLC
Reel/Frame 057022/0566 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 21, 2018
From: SARWER, MOHAMMED GOLAM; TABATABAI, ALI
To: SONY CORPORATION
Reel/Frame 045299/0797 →
Continuity (2)
Provisional Application 62467421 · Mar 6, 2017
Related Publication 20180255320A1 · Sep 6, 2018