IP Library Granted Patent US 11,055,816
Granted Patent B2
US 11,055,816 · App. 16/475,742 · Granted Jul 6, 2021

Image processing device, image processing method, and image processing program

Inventor: Yeongnam Chae (Tokyo, JP)
Assignee: Rakuten, Inc.
G06T3/40G06T5/003G06T7/90G06T1/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,055,816
App. No.
16/475,742
Granted
Jul 6, 2021
Kind
B2
Abstract

An image processing device according to one embodiment includes an acquisition unit, a generation unit, a calculation unit, and an estimation unit. The acquisition unit acquires an input image. The generation unit generates a plurality of comparison images by compressing a target region being at least part of the input image with each of a plurality of compression levels and expanding the compressed target region to its original size. The calculation unit calculates, for each of the plurality of comparison images, a degradation level of the comparison image with respect to the input image. The estimation unit estimates the blur level of the input image based on a plurality of calculated degradation levels.

Claims (28)

1. An image processing device comprising:

at least one memory operable to store program code; and

at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:

acquisition code configured to cause the at least one processor to acquire an input image;

generation code configured to cause the at least one processor to generate a plurality of comparison images by compressing a target region being at least part of the input image with each of a plurality of compression levels and expanding the compressed target region to its original size;

calculation code configured to cause the at least one processor to calculate, for each of the plurality of comparison images, a degradation level of the comparison image with respect to the input image;

estimation code configured to cause the at least one processor to estimate a blur level of the input image based on a plurality of calculated degradation levels, and

super-resolution code configured to cause the at least one processor to select a machine learning model corresponding to the blur level estimated by the estimation code from a plurality of machine learning models for super-resolution respectively corresponding to a plurality of blur levels, and input the input image into the selected machine learning model to generate a high-resolution image.

2. The image processing device according to claim 1 , wherein the degradation level is peak signal-to-noise ratio.

3. The image processing device according to claim 1 , wherein the estimation code is further configured to cause the at least one processor to estimate the blur level by comparing the plurality of degradation levels with a threshold.

4. The image processing device according to claim 1 , wherein the estimation code is further configured to cause the at least one processor to estimate the blur level based on a leading coefficient of a non-linear function representing a relationship between the compression level and the degradation level.

5. The image processing device according to claim 4 , wherein

a correspondence between the leading coefficient and the blur level is set in advance, and

the calculation code is further configured to cause the at least one processor to calculate the leading coefficient of the non-linear function from the plurality of compression levels and the plurality of degradation levels, and determine the blur level corresponding to the calculated leading coefficient by referring to the correspondence.

6. An image processing method performed by an image processing device including a processor, comprising:

acquiring an input image;

generating a plurality of comparison images by compressing a target region being at least part of the input image with each of a plurality of compression levels and expanding the compressed target region to its original size;

calculating, for each of the plurality of comparison images, a degradation level of the comparison image with respect to the input image;

estimating a blur level of the input image based on a plurality of calculated degradation levels;

selecting a machine learning model corresponding to the blur level from a plurality of machine learning models for super-resolution respectively corresponding to a plurality of blur levels; and

inputting the input image into the selected machine learning model to generate a high-resolution image.

7. A non-transitory recording medium storing an image processing program causing a computer to:

acquire an input image;

generate a plurality of comparison images by compressing a target region being at least part of the input image with each of a plurality of compression levels and expanding the compressed target region to its original size;

calculate, for each of the plurality of comparison images, a degradation level of the comparison image with respect to the input image;

estimate a blur level of the input image based on a plurality of calculated degradation levels,

select a machine learning model corresponding to the blur level from a plurality of machine learning models for super-resolution respectively corresponding to a plurality of blur levels; and

input the input image into the selected machine learning model to generate a high-resolution image.

Assignments (2)
CHANGE OF NAME Recorded Jul 9, 2021
From: RAKUTEN INC
To: RAKUTEN GROUP INC
Reel/Frame 056816/0068 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 3, 2019
From: CHAE, YEONGNAM
To: RAKUTEN, INC.
Reel/Frame 049662/0243 →