Systems and methods for training a non-blind image deblurring module

1. A method of training a non-blind deblurring module,

wherein the non-blind deblurring module includes a regularized deconvolution sub-module and a convolutional neural network sub-module,

the regularized deconvolution sub-module is configured to perform a regularized deconvolution function on a blurred input image to produce a deconvolved image potentially having image artifacts, and

the convolutional neural network sub-module is configured to receive the deconvolved image as an input to a convolutional neural network and to remove image artifacts, thereby providing a deblurred output image,

the method comprising:

receiving, via at least one processor, unblurred test images and blurred test images, wherein each of the blurred test images is related to a corresponding one of the unblurred test images by a blur kernel term and a noise term;

jointly training, via the at least one processor, the regularized deconvolution sub-module and the convolutional neural network by adjusting a regularization parameter of the regularized deconvolution function and weights of the convolution neural network in order to minimize a cost function representative of a difference between each deblurred output image and a corresponding one of the unblurred test images, thereby providing a trained regularization parameter, trained weights and a trained non-blind deblurring module;

receiving, via the at least one processor, a blurred input image from an imaging device;

deblurring, via the at least one processor, the blurred input image using the trained non-blind deblurring module; and

outputting, via the at least one processor, a deblurred output image.

2. The method of claim 1 , wherein the deconvolution function is a Wiener deconvolution function.

3. The method of claim 1 , wherein the deconvolution function is a Tikhonov-regularized deconvolution function.

4. The method of claim 1 , comprising using the trained non-blind deblurring module to deblur the blurred input image, thereby producing the deblurred output image, wherein the regularized deconvolution sub-module performs the regularized deconvolution function on the blurred input image to produce a deconvolved image potentially having image artifacts, the regularized deconvolution function including the trained regularization parameter, and the convolutional neural network sub-module processes the deconvolved image through the convolutional neural network to remove image artifacts, the convolution neural network including the trained weights.

5. The method of claim 4 , wherein the convolutional neural network outputs residuals and the trained non-blind deblurring module adds the residuals to the deconvolved image, thereby producing the deblurred output image.

6. The method of claim 1 , wherein adjusting, via the at least one processor, the regularization parameter and the weights uses a back propagation algorithm.

7. The method of claim 6 , wherein the back propagation algorithm adjusts the regularization parameter based on gradients that have been fed back from the CNN and a derivative of the deconvolved image potentially having image artifacts with respect to the regularization parameter.

8. The method of claim 1 , wherein the at least one processor receives the unblurred test images and artificially generates the blurred test images using a blur kernel function and a noise function on the unblurred test images.

9. The method of claim 1 , wherein the blurred input image is received from the imaging device mounted to a vehicle.

10. The method of claim 9 , wherein the vehicle includes a vehicle controller and the method includes controlling at least one vehicle function based on the deblurred output image.

11. A system for training a non-blind deblurring module, comprising:

a non-blind deblurring module including a regularized deconvolution sub-module and a convolutional neural network sub-module,

wherein the regularized deconvolution sub-module is configured to perform a regularized deconvolution function on a blurred input image to produce a deconvolved image potentially having image artifacts, and

wherein the convolutional neural network sub-module is configured to receive the deconvolved image as an input to a convolutional neural network and to remove image artifacts, thereby providing a deblurred output image;

an imaging device; and

at least one processor configured to execute program instructions, wherein the program instructions are configured to cause the at least one processor to:

received unblurred test images and blurred test images, wherein each of the blurred test images is related to a corresponding one of the unblurred test images by a blur kernel term and a noise term;

jointly train the regularized deconvolution sub-module and the convolutional neural network by adjusting a regularization parameter of the regularized deconvolution function and weights of the convolution neural network in order to minimize a cost function representative of a difference between each deblurred output image and a corresponding one of the unblurred test images, thereby providing a trained regularization parameter, trained weights and a trained non-blind deblurring module;

receive a blurred input image from the imaging device;

deblur the blurred input image using the trained non-blind deblurring module; and

output a deblurred output image.

12. The system of claim 11 , wherein the deconvolution function is a Wiener deconvolution function.

13. The system of claim 11 , wherein the deconvolution function is a Tikhonov-regularized deconvolution function.

14. The system of claim 11 , wherein the trained non-blind deblurring module is configured to deblur the blurred input image, thereby producing the deblurred output image, wherein the regularized deconvolution sub-module is configured to perform the regularized deconvolution function on the blurred input image to produce a deconvolved image potentially having image artifacts using the trained regularization parameter, and the convolutional neural network sub-module is configured to process the deconvolved image through the convolutional neural network to remove image artifacts using the trained weights.

15. The system of claim 14 , wherein the convolutional neural network is configured to output residuals and the trained non-blind deblurring module is configured to add the residuals to the deconvolved image, thereby producing the deblurred output image.

16. The system of claim 11 , wherein the program instructions are configured to cause the at least one processor to adjust the regularization parameter and the weights using a back propagation algorithm.

17. The system of claim 11 , wherein the program instructions are configured to cause the at least one processor to adjust the regularization parameter based on gradients that have been fed back from the CNN and a derivative of the deconvolved image potentially having image artifacts with respect to the regularization parameter.

18. The system of claim 11 , wherein the program instructions are configured to cause the at least one processor to receive the unblurred test images and artificially generate the blurred test images using a blur kernel function and a noise function on the unblurred test images.

19. The system of claim 11 comprising a vehicle, the vehicle comprising the imaging device and the non-blind deblurring module, the non-blind deblurring module configured to receive the blurred input image from the imaging device.

20. The system of claim 19 , wherein the vehicle includes a vehicle controller configured to control at least one vehicle function based on the deblurred output image.