IP Library Granted Patent US 11,605,174
Granted Patent B2
US 11,605,174 · App. 17/110,570 · Granted Mar 14, 2023

Depth-of-field simulation via optical-based depth extraction

Inventors: Jizhang Shan (Los Gatos, CA); Tiecheng Wu (Singapore, SG); Bo Li (Singapore, SG); Jin Xu (Singapore, SG)
Assignee: Black Sesame Technologies Inc.
G06T7/571G06T5/002G06T5/20G06T5/50G06T7/11G06T2207/20192G06T2207/20212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,605,174
App. No.
17/110,570
Granted
Mar 14, 2023
Kind
B2
Abstract

A method of depth-of-field simulation, including receiving a plurality of images, predicting a layer of interest mask of the plurality of images, determining a plurality of mean brightness anchor values of a respective plurality of layers of interest window arrays, setting a plurality of layers of interest set of binary codes, determining a hamming distance between plurality of layers of interest set of binary codes, determining a cost volume based on the hamming distance, resampling a vertical cost based on a vertical ordinal direction the cost volume, resampling a horizontal cost based on a horizontal ordinal direction the cost volume, determining an all-in-focus layer based on the vertical cost and the horizontal cost, determining an out-of-focus layer based on the vertical cost and the horizontal cost and determining a depth of the all-in-focus layer and out-of-focus layer.

Claims (41)

1. A method of depth-of-field simulation, comprising:

receiving a first image;

receiving a second image;

predicting a layer of interest mask of the first image;

determining a first mean brightness anchor value of a first window array of the layer of interest mask;

determining a second mean brightness anchor value of a second window array of the layer of interest mask;

setting a first set of binary codes of the first window array;

setting a second set of binary codes of the second window array;

determining a hamming distance between the first set of binary codes and the second set of binary codes;

determining a cost volume based on the hamming distance;

resampling a vertical cost based on a vertical ordinal direction the cost volume;

resampling a horizontal cost based on a horizontal ordinal direction the cost volume;

determining an all-in-focus layer based on the vertical cost and the horizontal cost;

determining an out-of-focus layer based on the vertical cost and the horizontal cost; and

determining a depth of the all-in-focus layer and out-of-focus layer.

2. The method of claim 1 , further comprising semantic partitioning of the cost volume.

3. The method of claim 1 , further comprising edge a filtering the all-in-focus layer.

4. The method of claim 3 , further comprising smooth filtering the out-of-focus layer.

5. The method of claim 4 , further comprising recombining the edge filtered ail-in-focus layer and the smooth filtered out-of-focus layer.

6. The method of claim 1 , further comprising decoupling a spatial domain and a depth domain of the first image and the second image.

7. The method of claim 6 , further comprising estimating an all-in-focus depth of the all-in-focus layer.

8. The method of claim 7 , further comprising segregating the all-in-focus layer and the out-of-focus layer.

9. A method of depth-of-field simulation, comprising:

receiving a plurality of images;

predicting a layer of interest mask of the plurality of images;

determining a plurality of mean brightness anchor values of a respective plurality of layers of interest window arrays of the layer of interest mask within a respective plurality of images;

setting a plurality of layers of interest set of binary codes of the respective plurality of layers of interest window arrays;

determining a hamming distance between plurality of layers of interest set of binary codes of the respective plurality of layers of interest window arrays;

determining a cost volume based on the hamming distance;

resampling a vertical cost based on a vertical ordinal direction the cost volume;

resampling a horizontal cost based on a horizontal ordinal direction the cost volume;

determining an all-in-focus layer based on the vertical cost and the horizontal cost;

determining an out-of-focus layer based on the vertical cost and the horizontal cost; and

determining a depth of the all-in-focus layer and out-of-focus layer.

10. The method of claim 9 , further comprising semantic partitioning of the cost volume.

11. The method of claim 9 , further comprising edge filtering the all-in-focus layer.

12. The method of claim 11 , further comprising smooth filtering the out-of-focus layer.

13. The method of claim 12 , further comprising recombining the edge filtered all-in-focus layer and the smooth filtered out-of-focus layer.

14. The method of claim 9 , further comprising decoupling a spatial domain and a depth domain of the plurality of images.

15. The method of claim 14 , further comprising estimating an all-in-focus depth of the all-in-focus layer.

16. The method of claim 15 , further comprising segregating the all-in-focus layer and the out-of-focus layer.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2021
From: BLACK SESAME INTERNATIONAL HOLDING LIMITED
To: BLACK SESAME TECHNOLOGIES INC.
Reel/Frame 058301/0364 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2021
From: SHAN, JIZHANG; WU, TIECHENG; LI, BO; XU, JIN
To: BLACK SESAME INTERNATIONAL HOLDING LIMITED
Reel/Frame 057903/0141 →
Continuity (1)
Related Publication 20220180544A1 · Jun 9, 2022
Cited By (1)
US 12,238,259