Granted Patent B2

US 12,283,034 · App. 18/489,117 · Granted Apr 22, 2025

Inventor: Takashi Oniki (Tochigi, JP)

Assignee: CANON KABUSHIKI KAISHA

G06T7/0002G06T11/00G06T2207/20081G06T2207/20084G06T2207/30168

Abstract

Claims (61)

1. A method of generating training data used to train a neural network, the method comprising:

acquiring an original image;

acquiring a first image as a training image generated based on first blur and the original image; and

acquiring a second image as a ground truth image generated based on second blur and the original image,

wherein the first blur and the second blur are acquired based on the same optical information.

2. The method according to claim 1 , wherein a sharpness of the training image is lower than a sharpness of the ground truth image.

3. The method according to claim 1 ,

wherein the optical information is a plurality of pieces of optical information, each of which is different depending on an image height,

wherein the first image is generated using the plurality of pieces of optical information, and

wherein the second image is generated using the plurality of pieces of optical information.

4. The method according to claim 1 ,

wherein the second blur has a different amount depending on an image height.

5. The method according to claim 4 ,

wherein the second blur has a third blur amount at a first image height of the original image and a fourth blur amount at a second image height higher than the first image height of the original image, the fourth blur amount being larger than the third blur amount.

6. The method according to claim 1 ,

wherein the optical information is a plurality of pieces of optical information each of which is different depending on an image height,

wherein the first image is generated using optical information corresponding to an image height of the first image among the plurality of pieces of optical information, and

wherein the second image generated using optical information corresponding to an image height of the second image among the plurality of pieces of optical information.

7. The method according to claim 1 ,

wherein the second blur is acquired based on a luminance value of the original image.

8. The method according to claim 7 ,

wherein when the luminance value is a first luminance value, the second image is generated based on the second blur with a third blur amount and the original image, and

wherein when the luminance value is a second luminance value lower than the first luminance value, the second image is generated based on the second blur with a fourth blur amount smaller than the third blur amount and the original image.

9. A training method for training a neural network using training data generated by the method according to claim 1 , the training method comprising:

generating a processed image by inputting the first image as the training image into the neural network; and

updating the neural network based on the second image and the processed image.

10. A non-transitory computer-readable storage medium storing a computer program that causes a computer to execute the method according to claim 1 .

11. A method of generating a neural network that is generated using training data generated by the method according to claim 1 , the method comprising:

generating a processed image by inputting the first image as the training image into the neural network; and

training the neural network based on the second image and the processed image.

12. An image processing method comprising:

acquiring the neural network obtained by the method of generating the neural network according to claim 11 ;

generating an estimated image by inputting an input image into the neural network.

13. The image processing method according to claim 12 ,

wherein the input image is an image obtained by image pickup, and

wherein the estimated image is generated by correcting blur in the input image generated in the image pickup using the neural network.

14. A non-transitory computer-readable storage medium storing a computer program that causes a computer to execute the image processing method according to claim 12 .

15. An image processing apparatus comprising:

a memory storing instructions; and

at least one processor that executes the instructions to:

acquire the neural network obtained by the method of generating the neural network according to claim 11 ; and

estimate an estimated image by inputting an input image into the neural network.

16. The method according to claim 1 , wherein the first blur and the second blur are different from each other.

17. A training data generating apparatus which generates training data used to train a neural network, the training data generating apparatus comprising:

at least one memory storing instructions; and

at least one processor that executes the instructions to:

acquire an original image;

acquire a first image generated based on first blur and the original image; and

acquire a second image generated based on second blur and the original image,

wherein the first blur and the second blur are acquired based on the same optical information.

18. A training apparatus comprising:

the training data generating apparatus according to claim 17 ; and

the at least one processor that executes the instructions to:

generate a processed image by inputting the first image as the training image to a neural network; and

update the neural network based on the second image and the processed image.

19. An image processing method comprising:

acquiring an input image; and

generating an estimated image by inputting the input image into a neural network,

wherein the neural network is generated by training using a training image and a ground truth image,

wherein the training image is generated based on first blur and an original image, and the ground truth image is generated based on second blur and the original image, and

wherein the first blur and the second blur are acquired based on the same optical information.

Priority Claims (2)

Continuity (2)

References Cited (19)