IP Library › Granted Patent US 12,033,303
Granted Patent B2
US 12,033,303 · App. 17/666,917 · Granted Jul 9, 2024

Mitigation of quantization-induced image artifacts

Inventor: Sheng Li (Irvine, CA)
Assignee: KYOCERA DOCUMENT SOLUTIONS, INC.
G06T3/4069G06T3/4046G06V10/56G06V10/762G06V10/764
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,033,303
App. No.
17/666,917
Granted
Jul 9, 2024
Kind
B2
Abstract

A system for super-sampling digital images detects artifacts in an SRGAN super-sampled image, determines blocks of the image that contribute to the artifacts, and if the artifact-contributing blocks exceed a threshold, discards the SRGAN generated output image in favor of applying a super-sampled image generated by an alternate mechanism, such as a nearest neighbor algorithm.

Claims (41)

1. A method comprising:

upsampling a digital image using a super-resolution generative adversarial network (SRGAN) to generate a first upsampled image;

upsampling the digital image using a fallback algorithm to generate a second upsampled image;

subtracting corresponding pixels of the first upsampled image and the second upsampled image;

counting pixel clusters comprising subtraction differences satisfying a first predefined threshold;

on condition that the count of the pixel clusters satisfies a second predefined threshold, selecting the second upsampled image as an applied output; and

otherwise, selecting the first upsampled image as the applied output.

2. The method of claim 1 , wherein the fallback algorithm is a floating point algorithm and the SRGAN is a fixed point network.

3. The method of claim 2 , wherein the fallback algorithm is a nearest neighbor algorithm.

4. The method of claim 1 , wherein the SRGAN comprises an 8-bit fixed point SRGAN converted from a 32-bit floating point SRGAN.

5. The method of claim 1 , wherein subtracting corresponding pixels comprises individually subtracting different color fields of the corresponding pixels.

6. The method of claim 1 , wherein the first predefined threshold comprises subtraction differences exceeding a third predefined threshold for all pixels in the pixel cluster.

7. The method of claim 1 , wherein the second predefined threshold comprises between 1% and 5% of a total number of the pixel blocks in the first upsampled image.

8. The method of claim 1 , wherein the second predefined threshold comprises a threshold specifically for high-contrast areas of the first upsampled image.

9. The method of claim 1 , wherein the pixel clusters are 2×2 pixel blocks.

10. The method of claim 1 , wherein the upsampling of the digital image by the SRGAN and by the nearest neighbor algorithm comprises a 4 X upsampling.

11. A system comprising:

at least one processor; and

logic to operate the processor to:

upsample a digital image using a super-resolution generative adversarial network (SRGAN) to generate a first upsampled image;

upsample the digital image using a nearest neighbor algorithm to generate a second upsampled image;

compare corresponding pixels of the first upsampled image and the second upsampled image;

count pixel clusters for which the comparison satisfies a first predefined threshold;

on condition that the count of the pixel clusters satisfies a second predefined threshold, select the second upsampled image as an applied output; and

otherwise, select the first upsampled image as the applied output.

12. The system of claim 11 , wherein the SRGAN comprises a fixed point SRGAN converted from a floating point SRGAN.

13. The system of claim 11 , wherein comparing corresponding pixels comprises determining an overall color difference between the corresponding pixels.

14. The system of claim 11 , wherein the pixel clusters are 2×2 pixel blocks.

15. The system of claim 14 , wherein the 2×2 pixel blocks are determined using a stride length of one (1).

16. A system comprising:

an SRGAN generated by quantizing a floating point deep network to a fixed point deep network;

a floating point digital imaging algorithm;

a digital image comparator;

wherein the SRGAN is configured to receive and convert a low resolution digital image into a first higher resolution image, the floating point digital imaging algorithm is configured to receive and convert the low resolution digital image into a second higher resolution image, the digital image comparator is configured to compare the first higher resolution image and the second higher resolution image; and

logic to:

count pixel clusters for which the comparison satisfies a first predefined threshold; and

on condition that the count of the pixel clusters satisfies a second predefined threshold, select the second higher resolution image instead of the first higher resolution image as an output image of a printer or digital display.

17. The system of claim 16 , wherein the digital image comparator compares the first higher resolution image and the second higher resolution image on a multi-pixel block by block basis.

18. The system of claim 17 , wherein the comparison satisfies the first predefined threshold when all of the pixels in a block each differ between the first higher resolution image and the second higher resolution image by a third threshold amount.

19. The system of claim 16 , wherein the multi-pixel blocks are 2×2 pixel blocks.

20. The system of claim 19 , wherein the 2×2 pixel blocks are determined using a stride length of one (1).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 22, 2022
From: LI, SHENG
To: KYOCERA DOCUMENT SOLUTIONS, INC.
Reel/Frame 061859/0221 →
Continuity (1)
Related Publication 20230252603A1 · Aug 10, 2023