IP Library › Granted Patent US 11,272,074
Granted Patent B2
US 11,272,074 · App. 16/793,860 · Granted Mar 8, 2022

System, method, and storage medium for converting low-resolution image data into high-resolution image data using neural network

Inventors: Ken Achiwa (Kawasaki, JP); Ryo Kosaka (Tokyo, JP)
Assignee: CANON KABUSHIKI KAISHA
H04N1/40068G06N3/08H04N1/00482H04N1/409H04N1/40H04N1/401
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,272,074
App. No.
16/793,860
Granted
Mar 8, 2022
Kind
B2
Abstract

A system previously generates a plurality of neural networks by performing learning based on an image data pair generated based on identical original data and differing in resolution. Then, the system determines one neural network out of the plurality of neural networks based on a resolution of input image data and a resolution of an image to be output, acquires post-conversion image data based on the determined neural network and the input image data, and performs outputting that is based on the post-conversion image data.

Claims (57)

1. A system comprising:

at least one memory that stores a program; and

at least one processor that executes the program to perform:

acquiring first information about a first resolution of an input image;

acquiring second information about a second resolution of an image that is to be output, wherein the second resolution is higher than the first resolution;

determining, from among a plurality of neural networks that convert a low-resolution image into a high-resolution image, one neural network based on the first information and the second information, wherein the determined one neural network is a neural network suitable for converting an image having the first resolution into an image having the second resolution,

wherein the plurality of neural networks are obtained by performing learning based on an image data pair which includes first image data and second image data which is higher in resolution than the first image data,

wherein the second image data is obtained by rendering page-description language (PDL) data included in a print job, and

wherein the first image data is obtained at least by imparting noises to the obtained second image data and converting the noises-imparted second image data into an image of a low-resolution;

acquiring post-conversion image data having the second resolution into which the input image having the first resolution is converted using the determined one neural network; and

outputting the acquired post-conversion image data.

2. The system according to claim 1 , wherein the first image data is binary data and the second image data is multivalued data.

3. The system according to claim 1 , further comprising a printer,

wherein the outputting includes causing the printer to form the acquired post-conversion image data on a sheet, and

wherein the second information is information about the second resolution at which image formation is performed by the printer.

4. The system according to claim 1 , further comprising a user interface configured to receive an operation performed by a user,

wherein the second information is information about the second resolution designated by the user via the user interface.

5. The system according to claim 1 , wherein the noise imparting processing includes processing for imparting random noise to the obtained second image data.

6. The system according to claim 1 , wherein the first image data is obtained by imparting noises to the obtained second image data, converting the noises-imparted second image data into an image of a low-resolution and further applying binarization processing to the converted image of the low-resolution.

7. The system according to claim 1 , wherein the noise imparting processing includes processing for imparting respective independent random noises to a surrounding region of a foreground region of the obtained second image data and an inside region of the foreground region of obtained the second image data.

8. A method comprising:

acquiring first information about a first resolution of an input image;

acquiring second information about a second resolution of an image that is to be output, wherein the second resolution is higher than the first resolution;

determining, from among a plurality of neural networks that convert a low-resolution image into a high-resolution image, one neural network based on the first information and the second information, wherein the determined one neural network is a neural network suitable for converting an image having the first resolution into an image having the second resolution,

wherein the plurality of neural networks are obtained by performing learning based on an image data pair which includes first image data and second image data which is higher in resolution than the first image data, wherein the second image data is obtained by rendering page-description language (PDL) data included in a print job, and

wherein the first image data is obtained at least by imparting noises to the obtained second image data and converting the noises-imparted second image data into an image of a low-resolution;

acquiring post-conversion image data having the second resolution into which the input image having the first resolution is converted using the determined one neural network; and

outputting the acquired post-conversion image data.

9. The method according to claim 8 ,

wherein the outputting includes causing a printer to form the acquired post-conversion image data on a sheet, and

wherein the second information is information about the second resolution at which image formation is performed by the printer.

10. The method according to claim 8 , further comprising receiving an operation performed by a user by a user interface,

wherein the second information is information about the second resolution designated by the user via the user interface.

11. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a computer, cause the computer to perform a method comprising:

acquiring first information about a first resolution of an input image;

acquiring second information about a second resolution of an image that is to be output, wherein the second resolution is higher than the first resolution;

determining, from among a plurality of neural networks that convert a low-resolution image into a high-resolution image, one neural network based on the first information and the second information, wherein the determined one neural network is a neural network suitable for converting an image having the first resolution into an image having the second resolution,

wherein the plurality of neural networks is obtained by performing learning based on an image data pair which includes first image data and second image data which is higher in resolution than the first image data,

wherein the second image data is obtained by rendering page-description language (PDL) data included in a print job, and

wherein the first image data is obtained at least by imparting noises to the obtained second image data and converting the noises-imparted second image data into an image of a low-resolution;

acquiring post-conversion image data having the second resolution into which the input image having the first resolution is converted using the determined one neural network; and

outputting the acquired post-conversion image data.

12. The non-transitory computer-readable storage medium according to claim 11 ,

wherein the outputting includes causing a printer to form the acquired post-conversion image data on a sheet, and

wherein the second information is information about the second resolution at which image formation is performed by the printer.

13. The non-transitory computer-readable storage medium according to claim 11 , further comprising receiving an operation performed by a user by a user interface,

wherein the second information is information about the second resolution designated by the user via the user interface.

14. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a computer, cause the computer to perform a method comprising:

obtaining second image data by rendering page-description language (PDL) data included in a print job;

obtaining first image data at least by imparting noises to the obtained second image data and converting the noises-imparted second image data into an image of a low-resolution, wherein a resolution of the first image is lower than a resolution of the second image; and

generating a neural network for converting a low-resolution image into a high-resolution image by performing learning based on a pair of the obtained first image data and the obtained second image data.

15. The method according to claim 8 , wherein the noise imparting processing includes processing for imparting random noise to the obtained second image data.

16. The method according to claim 8 , wherein the first image data is obtained by imparting noises to the obtained second image data, converting the noises-imparted second image data into an image of a low-resolution and further applying binarization processing to the converted image of the low-resolution.

17. The method according to claim 8 , wherein the noise imparting processing includes processing for imparting respective independent random noises to a surrounding region of a foreground region of the obtained second image data and an inside region of the foreground region of the obtained second image data.

18. The non-transitory computer-readable storage medium according to claim 11 , wherein the noise imparting processing includes processing for imparting random noise to the obtained second image data.

19. The non-transitory computer-readable storage medium according to claim 11 , wherein the first image data is obtained by imparting noises to the obtained second image data, converting the noises-imparted second image data into an image of a low-resolution and further applying binarization processing to the converted image of the low-resolution.

20. The non-transitory computer-readable storage medium according to claim 11 , wherein the noise imparting processing includes processing for imparting respective independent random noises to a surrounding region of a foreground region of the obtained second image data and an inside region of the foreground region of the obtained second image data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 29, 2020
From: ACHIWA, KEN; KOSAKA, RYO
To: CANON KABUSHIKI KAISHA
Reel/Frame 053685/0763 →
Priority Claims (1)
JP JP2019-036854 · Feb 28, 2019 · national
Continuity (1)
Related Publication 20200280654A1 · Sep 3, 2020
Cited By (2)
US 12,198,387 US 12,211,172