Adaptive deformable kernel prediction network for image de-noising
Embodiments are generally directed to an adaptive deformable kernel prediction network for image de-noising. An embodiment of a method for de-noising an image by a convolutional neural network implemented on a compute engine, the image including a plurality of pixels, the method comprising: for each of the plurality of pixels of the image, generating a convolutional kernel having a plurality of kernel values for the pixel; generating a plurality of offsets for the pixel respectively corresponding to the plurality of kernel values, each of the plurality of offsets to indicate a deviation from a pixel position of the pixel; determining a plurality of deviated pixel positions based on the pixel position of the pixel and the plurality of offsets; and filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions to obtain a de-noised pixel.
1. A method comprising:
de-noising an image by a convolutional neural network implemented on a compute hardware engine, the image including a plurality of pixels,
for each of the plurality of pixels of the image, generating a convolutional kernel having a plurality of kernel values for the pixel;
generating a plurality of offsets for the pixel respectively corresponding to the plurality of kernel values, each of the plurality of offsets to indicate a deviation from a pixel position of the pixel, wherein an upper limit of the deviation is predefined;
determining a plurality of deviated pixel positions based on the pixel position of the pixel and the plurality of offsets, wherein an offset of the plurality of offsets comprises a position value to indicate the deviation from the pixel position of the pixel, wherein the position value comprises floating point values, wherein the plurality of kernel values are different for at least two pixels of the image; and
filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions to obtain a de-noised pixel.
2. The method of claim 1 , wherein the plurality of offsets are different for at least two pixels of the image, wherein the plurality of offsets are generated prior to or simultaneously with the generation of the convolutional kernel.
3. The method of claim 1 , wherein the filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions comprises:
applying the plurality of kernel values of the convolutional kernel to the pixel values of the plurality of deviated pixel positions to obtain a weighted average of the pixel values.
4. An apparatus comprising:
a data storage to store data including an image, the image including a plurality of pixels; and
a compute hardware engine coupled to the data storage, the compute hardware engine to denoise the image by a convolutional neural network, the compute hardware engine to:
for each of the plurality of pixels of the image,
generate a convolutional kernel having a plurality of kernel values for the pixel;
generate a plurality of offsets for the pixel respectively corresponding to the plurality of kernel values, each of the plurality of offsets to indicate a deviation from a pixel position of the pixel, wherein an upper limit of the deviation is predefined;
determine a plurality of deviated pixel positions based on the pixel position of the pixel and the plurality of offsets, wherein an offset of the plurality of offsets comprises a position value to indicate the deviation from the pixel position of the pixel, wherein the position value comprises floating point values, wherein the plurality of kernel values are different for at least two pixels of the image; and
filter the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions to obtain a de-noised pixel.
5. The apparatus of claim 4 , wherein the plurality of offsets are different for at least two pixels of the image, wherein the plurality of offsets are generated prior to or simultaneously with the generation of the convolutional kernel.
6. The apparatus of claim 4 , wherein the filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions comprises:
applying the plurality of kernel values of the convolutional kernel to the pixel values of the plurality of deviated pixel positions to obtain a weighted average of the pixel values.
7. At least one non-transitory computer-readable medium comprising a plurality of instructions which, when executed, cause a computing device to perform operations comprising:
de-noising an image by a convolutional neural network implemented on a compute hardware engine, the image including a plurality of pixels,
for each of the plurality of pixels of the image, generating a convolutional kernel having a plurality of kernel values for the pixel;
generating a plurality of offsets for the pixel respectively corresponding to the plurality of kernel values, each of the plurality of offsets to indicate a deviation from a pixel position of the pixel, wherein an upper limit of the deviation is predefined;
determining a plurality of deviated pixel positions based on the pixel position of the pixel and the plurality of offsets, wherein an offset of the plurality of offsets comprises a position value to indicate the deviation from the pixel position of the pixel, wherein the position value comprises floating point values, wherein the plurality of kernel values are different for at least two pixels of the image; and
filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions to obtain a de-noised pixel.
8. The non-transitory computer-readable medium of claim 7 , wherein the plurality of offsets are different for at least two pixels of the image, wherein the plurality of offsets are generated prior to or simultaneously with the generation of the convolutional kernel.
9. The non-transitory computer-readable medium of claim 7 , wherein the filtering the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions comprises:
applying the plurality of kernel values of the convolutional kernel to the pixel values of the plurality of deviated pixel positions to obtain a weighted average of the pixel values.
10. A system comprising:
a data storage to store data including an image, the image including a plurality of pixels; and
a compute hardware engine coupled to the data storage, the compute hardware engine to denoise the image by a convolutional neural network, the compute hardware engine to:
for each of the plurality of pixels of the image,
generate a convolutional kernel having a plurality of kernel values for the pixel;
generate a plurality of offsets for the pixel respectively corresponding to the plurality of kernel values, each of the plurality of offsets to indicate a deviation from a pixel position of the pixel, wherein an upper limit of the deviation is predefined;
determine a plurality of deviated pixel positions based on the pixel position of the pixel and the plurality of offsets, wherein an offset of the plurality of offsets comprises a position value to indicate the deviation from the pixel position of the pixel, wherein the position value comprises floating point values, wherein the plurality of kernel values are different for at least two pixels of the image; and
filter the pixel with the convolutional kernel and pixel values of the plurality of deviated pixel positions to obtain a de-noised pixel.
11. The system of claim 10 , wherein the plurality of offsets are different for at least two pixels of the image, wherein the plurality of offsets are generated prior to or simultaneously with the generation of the convolutional kernel.