IP Library Granted Patent US 10,488,500
Granted Patent B2
US 10,488,500 · App. 15/793,284 · Granted Nov 26, 2019

System and method for enhancing image resolution

Inventors: Zhenyu Zhu (Shenzhen, CN); Cong Zhao (Shenzhen, CN); Ketan Tang (Shenzhen, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
G01S7/4865G01S17/023G01S17/102G01S17/89G06T3/4076G06T5/50
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,488,500
App. No.
15/793,284
Granted
Nov 26, 2019
Kind
B2
Abstract

A method for enhancing image resolution includes obtaining one or more images of a scene. The one or more images are associated with a first depth map. The method further includes determining a second depth map of the scene based upon the one or more images. The second depth map has a higher resolution than the first depth map.

Claims (40)

1. A method for enhancing image resolution comprising:

capturing a first image of a scene including a first depth map via a first imaging device;

capturing a second image of the scene via a second imaging device;

converting the first image into an intensity image associated with the first image and the first depth map;

converting the second image into a grayscale image associated with the second image;

calculating a relationship between the intensity image associated with the first image and the grayscale image associated with the second image; and

enhancing a resolution of the first depth map based upon the relationship to generate a second depth map,

wherein the second depth map has a higher resolution than the first depth map.

2. The method of claim 1 , wherein the first imaging device includes a Time of Flight (“ToF”) camera.

3. The method of claim 1 , wherein the second imaging device includes a Red, Green, and Blue (RGB) camera and the second image of the scene includes an RGB image.

4. The method of claim 1 , further comprising:

calibrating at least one of the first imaging device and the second imaging device by obtaining a transformation relationship between coordinates of the first and the second imaging devices.

5. The method of claim 1 , further comprising:

mapping one or more pixels of the second image to the first image.

6. The method of claim 1 , wherein calculating the relationship includes:

analyzing differences between image blocks of the intensity image associated with the first image and corresponding image blocks of the grayscale image associated with the second image; and

producing a degradation model between the image blocks of the intensity image associated with the first image and the corresponding image blocks of the grayscale image associated with the second image.

7. The method of claim 6 , wherein enhancing the resolution of the first depth map includes applying a transformation matrix to the image blocks of the first depth map, the transformation matrix being an inverse of a degradation model.

8. An apparatus for imaging resolution enhancing comprising:

a memory storing instructions; and

one or more processors individually or collectively coupled to the memory and configured to execute the instructions to:

capture a first image of a scene including a first depth map via a first imaging device;

capture a second image of the scene via a second imaging device:

convert the first image into an intensity image associated with the first image and the first depth map;

convert the second image into a grayscale image associated with the second image;

calculate a relationship between the intensity image associated with the first image and the grayscale image associated with the second image; and

enhance a resolution of the first depth map based upon the relationship to generate a second depth map,

wherein the second depth map has a higher resolution than the first depth map.

9. The apparatus of claim 8 , further comprising:

a first imaging device configured to capture the first image from one or more images of the scene; and

a second imaging device configured to capture the second image from one or more images of the scene.

10. The apparatus of claim 9 , wherein the one or more processors are configured to further execute the instructions to calibrate at least one of the first imaging device and the second imaging device by obtaining a transformation relationship between coordinates of the first and second imaging devices.

11. The apparatus of claim 9 , wherein the one or more processors are configured to further execute the instructions to map one or more pixels of the second image to one or more pixels of the first image.

12. The apparatus of claim 8 , wherein the one or more processors are configured to further execute the instructions to:

analyze a difference between image blocks of the intensity image associated with the first image and corresponding image blocks of the grayscale image associated with the second image; and

produce a degradation model between the image blocks of the intensity image associated with the first image and the corresponding image blocks of the grayscale image associated with the second image.

13. The apparatus of claim 12 , wherein the one or more processors are configured to further execute the instructions to enhance the resolution of the first depth map by applying a transformation matrix, which is an inverse of a degradation model, to the first depth map.

14. The apparatus of claim 8 , wherein either of the first imaging device or the second imaging device is a camera or a sensor.

15. The apparatus of claim 8 , wherein the first and second imaging devices are selected from a group including Red, Green, and Blue (“RGB”) cameras, laser cameras, infrared cameras, ultrasound cameras, and Time-of-Flight (“ToF”) cameras.

16. The apparatus of claim 15 , wherein the first imaging device includes one or more ToF cameras and the second imaging device includes one or more RGB cameras.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2017
From: ZHU, ZHENYU; ZHAO, CONG; TANG, KETAN
To: SZ DJI TECHNOLOGY CO., LTD.
Reel/Frame 043946/0834 →
Continuity (2)
Continuation PCTCN2015078096 · Apr 30, 2015
Related Publication 20180059225A1 · Mar 1, 2018