IP Library › Granted Patent US 10,708,486
Granted Patent B2
US 10,708,486 · App. 16/137,949 · Granted Jul 7, 2020

Generation of a depth-artificial image by determining an interpolated supplementary depth through interpolation based on the original depths and a detected edge

Inventors: Jisoo Lee (San Diego, CA); Kyuseo Han (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N5/23212G06T5/003G06T7/50H04N5/2621H04N5/272
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,708,486
App. No.
16/137,949
Granted
Jul 7, 2020
Kind
B2
Abstract

An original depth-map of a depth-natural image can be generated. The depth-natural image can include image pixels. The original depth-map can include original depths mapping to one or more of the image pixels. An edge in the depth-natural image can be detected. A supplementary depth can be interpolated based on the original depths and the detected edge. The depth-natural image can be blurred based on the interpolated supplementary depth to generate a depth-artificial image.

Claims (64)

1. A method for image processing comprising, with a processing system comprising one or more processors:

generating an original depth-map of a depth-natural image, the depth-natural image comprising image pixels, the original depth-map comprising original depths, the original depths mapping to one or more of the image pixels;

detecting an edge in the depth-natural image;

determining an interpolated supplementary depth through interpolation based on the original depths and the detected edge; and

blurring the depth-natural image based on the interpolated supplementary depth to generate a first depth-artificial image.

2. The method of claim 1 , wherein the depth-natural image exhibits a natural optical state and the blurring transforms the depth-natural image into the first depth-artificial image, so the first depth-artificial image exhibits an artificial optical state.

3. The method of claim 2 , wherein the natural optical state of the depth-natural image lacks a defocused background whereas the artificial optical state of the first depth-artificial image comprises a defocused background.

4. The method of claim 2 comprising:

receiving the artificial optical state, which comprises two optical fields; and

generating the depth-natural image such that regions of the first depth-artificial image mapping to one of the optical fields are out-of-focus while regions of the first depth-artificial image mapping to the other optical field are in-focus.

5. The method of claim 4 , wherein the received artificial optical state comprises an artificial depth-range for at least one of the artificial optical fields.

6. The method of claim 2 comprising:

receiving a command to generate a revised artificial optical state;

in response to the command, accessing the depth-original image and accessing the interpolated supplementary depth; and

blurring the depth-natural image based on the interpolated supplementary depth and the revised artificial optical state to generate a generate a second depth-artificial image different from the first depth-artificial image.

7. The method of claim 1 , wherein generating the original depth-map comprises associating a different confidence value with each of the original depths;

the method comprising determining the interpolated supplementary depth based on the different confidence values of the original depths.

8. The method of claim 1 comprising:

accessing a depth-natural mosaic;

demosaicing the depth-natural mosaic into the depth-natural image; and

generating the original depth-map of the depth-natural image using the depth-natural mosaic.

9. The method of claim 1 wherein the processing system comprises a camera with a lens, and the first depth-artificial image exhibits an artificial optical state, the method comprising:

receiving the artificial optical state;

moving the lens based on the received artificial optical state;

capturing the depth-natural image with the camera and through the lens; and

blurring the depth-natural image to generate the first depth-artificial image exhibiting the artificial optical state.

10. The method of claim 1 comprising:

assigning depth-boundaries to the depth-natural image, each depth-boundary dividing (a) a defocused foreground from a field-of-focus, (b) a defocused background from the field-of-focus, or (c) the defocused foreground from the defocused background; and

blurring the depth-natural image based on the depth-boundaries.

11. A processing system comprising one or more processors configured to:

generate an original depth-map of a depth-natural image, the depth-natural image comprising image pixels, the original depth-map comprising original depths, the original depths mapping to one or more of the image pixels;

detect an edge in the depth-natural image;

determine an interpolated supplementary depth through interpolation based on the original depths and the detected edge; and

blur the depth-natural image based on the interpolated supplementary depth to generate a first depth-artificial image.

12. The system of claim 11 , wherein the depth-natural image exhibits a natural optical state and the blurring transforms the depth-natural image into the first depth-artificial image, so the first depth-artificial image exhibits an artificial optical state.

13. The system of claim 12 , wherein the natural optical state of the depth-natural image lacks a defocused background whereas the artificial optical state of the first depth-artificial image comprises a defocused background.

14. The system of claim 12 wherein the one or more processors are configured to:

receive the artificial optical state, which comprises two optical fields; and

generate the depth-natural image such that regions of the first depth-artificial image mapping to one of the optical fields are out-of-focus while regions of the first depth-artificial image mapping to the other optical field are in-focus.

15. The system of claim 14 , wherein the received artificial optical state comprises an artificial depth-range for at least one of the artificial optical fields.

16. The system of claim 12 , wherein the one or more processors are configured to:

receive a command to generate a revised artificial optical state;

in response to the command, access the depth-original image and accessing the interpolated supplementary depth; and

blur the depth-natural image based on the interpolated supplementary depth and the revised artificial optical state to generate a generate a second depth-artificial image different from the first depth-artificial image.

17. The system of claim 11 , wherein the one or more processors are configured to:

generate the original depth-map by associating a different confidence value with each of the original depths;

determine the interpolated supplementary depth based on the different confidence values of the original depths.

18. The system of claim 11 , wherein the one or more processors are configured to:

access a depth-natural mosaic;

demosaic the depth-natural mosaic into the depth-natural image; and

generate the original depth-map of the depth-natural image using the depth-natural mosaic.

19. The system of claim 11 , wherein the processing system comprises a camera with a lens, and the first depth-artificial image exhibits an artificial optical state, the one or more processors being configured to:

receive the artificial optical state;

move the lens based on the received artificial optical state;

capture the depth-natural image with the camera and through the lens; and

blur the depth-natural image to generate the first depth-artificial image exhibiting the artificial optical state.

20. The system of claim 11 , wherein the one or more processors are configured to:

assign depth-boundaries to the depth-natural image, each depth-boundary dividing (a) a defocused foreground from a field-of-focus, (b) a defocused background from the field-of-focus, or (c) the defocused foreground from the defocused background; and

blur the depth-natural image based on the depth-boundaries.

21. A non-transitory computer-readable medium comprising program code, which, when executed by one or more processors, causes the one or more processors to perform operations, the program code comprising code for:

generating an original depth-map of a depth-natural image, the depth-natural image comprising image pixels, the original depth-map comprising original depths, the original depths mapping to one or more of the image pixels;

detecting an edge in the depth-natural image;

determining an interpolated supplementary depth through interpolation based on the original depths and the detected edge; and

blurring the depth-natural image based on the interpolated supplementary depth to generate a first depth-artificial image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2019
From: LEE, JISOO; HAN, KYUSEO
To: QUALCOMM INCORPORATED
Reel/Frame 048971/0885 →
Continuity (2)
Provisional Application 62631302 · Feb 15, 2018
Related Publication 20190253608A1 · Aug 15, 2019
Cited By (2)
US 12,334,096 US 12,457,416