Granted Patent B2

US 10,891,715 · App. 16/138,496 · Granted Jan 12, 2021

Inventor: Dalong Li (Rochester Hills, MI)

Assignee: Continental Automotive Systems, Inc.

G06T3/4007G06T3/4046G06T3/4053G06T5/50H04N5/23232H04N7/18G06T2207/20084H04N7/181

Abstract

Claims (51)

1. A method for generating a super-resolved high-resolution image based on a low-resolution image, the method comprising:

during a training phase:

receiving, at data processing hardware, one or more high-resolution test images;

downsizing, at the data processing hardware, the one or more high-resolution test images into a low-resolution test image having a low-resolution image size that is less than an image size of the one or more test high-resolution images;

interpolating, at the data processing hardware, the one or more low-resolution images into one or more interpolated images having an interpolated image size that is equal to the image size of the one or more high-resolution test images;

mapping, at the data processing hardware, an interpolated sub-image of the interpolated image having a number of pixels to a pixel of a test sub-image of the high-resolution test image;

generating, at the data processing hardware, a model data comprising mapped data of the interpolated sub-image to the pixel of the test sub-image; and

storing, at memory hardware in communication with the data processing hardware, the model data; and

during an execution phase:

receiving, at data processing hardware, a low-resolution image having a first image size from a camera in communication with the data processing hardware;

determining, at the data processing hardware, an interpolated image of the low-resolution image, the interpolated image having a second image size being greater than the first image size;

determining, at the data processing hardware, a high-resolution super-resolved image based on the interpolated image and the stored model data; and

displaying, on a user interface in communication with the data processing hardware, the high-resolution super-resolved image.

2. The method of claim 1 , wherein the model data comprises neural network weights and biases along with other neural network parameters.

3. The method of claim 1 , wherein mapping the interpolated sub-image to a pixel of the test sub-image, further comprises:

dividing the interpolated image into interpolated sub-images;

dividing the high-resolution test image into test sub-images;

for each interpolated sub-image:

determining a complementary test sub-image having a location within the high-resolution test image being the same location as a location of the interpolated sub-image within the interpolated image; and

mapping the interpolated sub-image to one pixel of the complementary test sub-image.

4. The method of claim 3 , wherein the one pixel of the test sub-image is positioned in the center of the test sub-image.

5. The method of claim 1 , wherein the data processing hardware is supported by a vehicle.

6. The method of claim 1 , wherein the second image size of the interpolated image is double the first image size of the low-resolution image.

7. The method of claim 1 , wherein the data processing hardware comprises a neural network.

8. A system for generating a super-resolved high-resolution image based on a low-resolution image, the system comprising:

a user interface;

data processing hardware in communication with the user interface; and

memory hardware in communication with the processing hardware, the memory hardware storing instruction that when executed cause the data processing hardware to execute a method, the method includes:

during a training phase:

receiving one or more high-resolution test images;

downsizing the one or more high-resolution test images into a low-resolution test image having a low-resolution image size that is less than an image size of the one or more test high-resolution images;

interpolating the one or more low-resolution images into one or more interpolated images having an interpolated image size that is equal to the image size of the one or more high-resolution test images;

mapping an interpolated sub-image of the interpolated image having a number of pixels to a pixel of a test sub-image of the high-resolution test image; and

generating a model data comprising mapped data of the interpolated sub-image to the pixel of the test sub-image;

storing the model data at the hardware memory;

during an execution phase:

receiving a low-resolution image having a first image size from a camera in communication with the data processing hardware;

determining an interpolated image of the low-resolution image, the interpolated image having a second image size being greater than the first image size;

determining a high-resolution super-resolved image based on the interpolated image and the stored model data; and

transmitting a command to the user interface causing the user interface to display the high-resolution super-resolved image.

9. The system of claim 8 , wherein the model data comprises neural network weights and biases along with other neural network parameters.

10. The system of claim 8 , wherein mapping the interpolated sub-image to a pixel of the test sub-image, further comprises:

dividing the interpolated image into interpolated sub-images;

dividing the high-resolution test image into test sub-images;

for each interpolated sub-image:

determining a complementary test sub-image having a location within the high-resolution test image being the same location as a location of the interpolated sub-image within the interpolated image; and

mapping the interpolated sub-image to one pixel of the complementary test sub-image.

11. The system of claim 8 , wherein the one pixel of the test sub-image is positioned in the center of the test sub-image.

12. The system of claim 8 , wherein the data processing hardware is supported by a vehicle.

13. The system of claim 8 , wherein the second image size of the interpolated image is double the first image size of the low-resolution image.

14. The system of claim 8 , wherein the data processing hardware comprises a neural network.

Assignments (2)

Cited By (12)

US 12,307,350 US 12,346,816 US 12,367,405 US 12,455,739 US 12,462,575 US 12,522,243 US 12,536,131 US 12,554,467 US 12,591,240 US 12,618,976 US 12,623,691 US 12,709,294