Deep neural network for image enhancement
A method for generating a super-resolved high-resolution image based on a low-resolution image is provided. The method includes receiving a low-resolution image having a first image size from a camera, and determining an interpolated image of the low-resolution image. The interpolated image has a second image size being greater than the first image size. The method also includes determining a high-resolution super-resolved image based on the interpolated image and model data. The model data is stored on memory hardware. The method also includes displaying, on a user interface, the high-resolution super-resolved image.
1. A method for generating a super-resolved high-resolution image based on a low-resolution image, the method comprising:
during a training phase:
receiving, at data processing hardware, one or more high-resolution test images;
downsizing, at the data processing hardware, the one or more high-resolution test images into a low-resolution test image having a low-resolution image size that is less than an image size of the one or more test high-resolution images;
interpolating, at the data processing hardware, the one or more low-resolution images into one or more interpolated images having an interpolated image size that is equal to the image size of the one or more high-resolution test images;
mapping, at the data processing hardware, an interpolated sub-image of the interpolated image having a number of pixels to a pixel of a test sub-image of the high-resolution test image;
generating, at the data processing hardware, a model data comprising mapped data of the interpolated sub-image to the pixel of the test sub-image; and
storing, at memory hardware in communication with the data processing hardware, the model data; and
during an execution phase:
receiving, at data processing hardware, a low-resolution image having a first image size from a camera in communication with the data processing hardware;
determining, at the data processing hardware, an interpolated image of the low-resolution image, the interpolated image having a second image size being greater than the first image size;
determining, at the data processing hardware, a high-resolution super-resolved image based on the interpolated image and the stored model data; and
displaying, on a user interface in communication with the data processing hardware, the high-resolution super-resolved image.
2. The method of claim 1 , wherein the model data comprises neural network weights and biases along with other neural network parameters.
3. The method of claim 1 , wherein mapping the interpolated sub-image to a pixel of the test sub-image, further comprises:
dividing the interpolated image into interpolated sub-images;
dividing the high-resolution test image into test sub-images;
for each interpolated sub-image:
determining a complementary test sub-image having a location within the high-resolution test image being the same location as a location of the interpolated sub-image within the interpolated image; and
mapping the interpolated sub-image to one pixel of the complementary test sub-image.
4. The method of claim 3 , wherein the one pixel of the test sub-image is positioned in the center of the test sub-image.
5. The method of claim 1 , wherein the data processing hardware is supported by a vehicle.
6. The method of claim 1 , wherein the second image size of the interpolated image is double the first image size of the low-resolution image.
7. The method of claim 1 , wherein the data processing hardware comprises a neural network.
8. A system for generating a super-resolved high-resolution image based on a low-resolution image, the system comprising:
a user interface;
data processing hardware in communication with the user interface; and
memory hardware in communication with the processing hardware, the memory hardware storing instruction that when executed cause the data processing hardware to execute a method, the method includes:
during a training phase:
receiving one or more high-resolution test images;
downsizing the one or more high-resolution test images into a low-resolution test image having a low-resolution image size that is less than an image size of the one or more test high-resolution images;
interpolating the one or more low-resolution images into one or more interpolated images having an interpolated image size that is equal to the image size of the one or more high-resolution test images;
mapping an interpolated sub-image of the interpolated image having a number of pixels to a pixel of a test sub-image of the high-resolution test image; and
generating a model data comprising mapped data of the interpolated sub-image to the pixel of the test sub-image;
storing the model data at the hardware memory;
during an execution phase:
receiving a low-resolution image having a first image size from a camera in communication with the data processing hardware;
determining an interpolated image of the low-resolution image, the interpolated image having a second image size being greater than the first image size;
determining a high-resolution super-resolved image based on the interpolated image and the stored model data; and
transmitting a command to the user interface causing the user interface to display the high-resolution super-resolved image.
9. The system of claim 8 , wherein the model data comprises neural network weights and biases along with other neural network parameters.
10. The system of claim 8 , wherein mapping the interpolated sub-image to a pixel of the test sub-image, further comprises:
dividing the interpolated image into interpolated sub-images;
dividing the high-resolution test image into test sub-images;
for each interpolated sub-image:
determining a complementary test sub-image having a location within the high-resolution test image being the same location as a location of the interpolated sub-image within the interpolated image; and
mapping the interpolated sub-image to one pixel of the complementary test sub-image.
11. The system of claim 8 , wherein the one pixel of the test sub-image is positioned in the center of the test sub-image.
12. The system of claim 8 , wherein the data processing hardware is supported by a vehicle.
13. The system of claim 8 , wherein the second image size of the interpolated image is double the first image size of the low-resolution image.
14. The system of claim 8 , wherein the data processing hardware comprises a neural network.