IP Library › Granted Patent US 11,366,624
Granted Patent B2
US 11,366,624 · App. 16/834,772 · Granted Jun 21, 2022

Super-resolution convolutional neural network with gradient image detection

Inventors: Sheng Li (El Segundo, CA); Dongpei Su (Palos Verdes, CA)
Assignee: KYOCERA Document Solutions Inc.
G06F3/1256G06F3/1208G06K9/6232G06K9/6257G06N3/04G06T3/4046G06T3/4053
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,366,624
App. No.
16/834,772
Granted
Jun 21, 2022
Kind
B2
Abstract

An example system includes a processor and a non-transitory computer-readable medium having stored therein instructions that are executable to cause the system to perform various functions. The functions include obtaining an image associated with a print job, and providing the image as input to a convolutional neural network. The convolutional neural network includes a residual network, upscaling layers, and classification layers configured to detect whether the image is an artificial image having a computer-generated image gradient. The functions also include determining, based on an output of the classification layers, that the image is an artificial image having a computer-generated image gradient. Further, the functions include, based on determining that the image is an artificial image having a computer-generated image gradient, providing the image to an upscaling module of a print pipeline for upscaling rather than using an output of the upscaling layers for the upscaling.

Claims (42)

1. A system comprising:

a processor; and

a non-transitory computer-readable medium having stored therein instructions that are executable to cause the system to perform functions comprising:

obtaining an image associated with a print job,

providing the image as input to a convolutional neural network, wherein the convolutional neural network comprises a residual network configured to extract feature maps from the image, upscaling layers configured to increase a resolution of the image based on the feature maps extracted from the image, and classification layers configured to detect whether the image is an artificial image having a computer-generated image gradient based on the feature maps extracted from the image,

determining, based on an output of the classification layers corresponding to the image, that the image is an artificial image having a computer-generated image gradient, and

based on the determining that the image is an artificial image having a computer-generated image gradient, providing the image to an upscaling module of a print pipeline for upscaling rather than using an output of the upscaling layers corresponding to the image for the upscaling.

2. The system of claim 1 , wherein the computer-generated image gradient comprises a directional change in intensity within the image.

3. The system of claim 1 , wherein the computer-generated image gradient comprises a directional change in color within the image.

4. The system of claim 1 , wherein the functions further comprise discarding the output of the upscaling layers corresponding to the image based on the determining that the image is an artificial image having a computer-generated image gradient.

5. The system of claim 1 , wherein the upscaling layers are parallel to the classification layers.

6. The system of claim 1 , wherein the residual network comprises a plurality of residual blocks.

7. The system of claim 6 , wherein the classification layers comprise a convolution layer, a pooling layer, and a fully connected layer.

8. The system of claim 7 , wherein the classification layers comprise five convolution layers followed by three fully connected layers.

9. The system of claim 1 , wherein the system comprises a printing device, and wherein the functions further comprise:

obtaining an upscaled image from the upscaling module; and

printing the upscaled image.

10. The system of claim 1 , wherein the functions further comprise:

obtaining another image;

providing the other image as input to the convolutional neural network;

determining, based on an output of the classification layers corresponding to the other image that the image is not an artificial image having a computer-generated image gradient; and

using an output of the upscaling layers corresponding to the other image for upscaling the other image.

11. The system of claim 1 , wherein the classification layers are trained using a set of training images comprising images that are known to be artificial images having computer-generated image gradients.

12. The system of claim 11 , wherein during training of the classification layers, parameters of the residual network and the upscaling layers are not updated.

13. The system of claim 12 , wherein prior to training of the classification layers, the residual network and the upscaling layers are preloaded with pre-trained parameters.

14. A computer-implemented method comprising:

obtaining an image associated with a print job;

providing the image as input to a convolutional neural network, wherein the convolutional neural network comprises a residual network configured to extract feature maps from the image, upscaling layers configured to increase a resolution of the image based on the feature maps extracted from the image, and classification layers configured to detect whether the image is an artificial image having a computer-generated image gradient based on the feature maps extracted from the image;

determining, based on an output of the classification layers corresponding to the image, that the image is an artificial image having a computer-generated image gradient; and

based on the determining that the image is an artificial image having a computer-generated image gradient, providing the image to an upscaling module of a print pipeline for upscaling rather than using an output of the upscaling layers corresponding to the image for the upscaling.

15. The computer-implemented method of claim 14 , further comprising discarding the output of the upscaling layers corresponding to the image based on the determining that the image is an artificial image having a computer-generated image gradient.

16. The computer-implemented method of claim 14 , wherein the upscaling layers are parallel to the classification layers.

17. The computer-implemented method of claim 14 , further comprising:

obtaining an upscaled image from the upscaling module; and

printing the upscaled image.

18. A non-transitory computer-readable medium having stored therein instructions that are executable to cause a system to perform functions comprising:

obtaining an image associated with a print job;

providing the image as input to a convolutional neural network, wherein the convolutional neural network comprises a residual network configured to extract feature maps from the image, upscaling layers configured to increase a resolution of the image based on the feature maps extracted from the image, and classification layers configured to detect whether the image is an artificial image having a computer-generated image gradient based on the feature maps extracted from the image;

determining, based on an output of the classification layers corresponding to the image, that the image is an artificial image having a computer-generated image gradient; and

based on the determining that the image is an artificial image having a computer-generated image gradient, providing the image to an upscaling module of a print pipeline for upscaling rather than using an output of the upscaling layers corresponding to the image for the upscaling.

19. The non-transitory computer-readable medium of claim 18 , wherein the functions further comprise discarding the output of the upscaling layers corresponding to the image based on the determining that the image is an artificial image having a computer-generated image gradient.

20. The non-transitory computer-readable medium of claim 18 , wherein the upscaling layers are parallel to the classification layers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2020
From: LI, SHENG; SU, DONGPEI
To: KYOCERA DOCUMENT SOLUTIONS INC.
Reel/Frame 052264/0093 →
Continuity (1)
Related Publication 20210303243A1 · Sep 30, 2021