IP Library › Granted Patent US 11,055,826
Granted Patent B2
US 11,055,826 · App. 17/016,043 · Granted Jul 6, 2021

Method and apparatus for image processing

Inventors: Jianhua Lin (Hangzhou, CN); Li Yu (Hangzhou, CN); Feng Zhou (Hangzhou, CN)
Assignee: ArcSoft Corporation Limited
G06T5/002G06T5/50G06T7/593H04N5/2226H04N5/2258H04N5/23212H04N5/23229G06T2207/10012G06T2207/10024G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,055,826
App. No.
17/016,043
Granted
Jul 6, 2021
Kind
B2
Abstract

An apparatus for image processing includes a processor and a depth camera. The processor obtains a first two-dimensional image and focusing information of the first two-dimensional image, and obtains depth information of a part or all of a content captured in the first two-dimensional image via the depth camera. The processor determines a background area of the first two-dimensional image according to the focusing information. The processor also performs bokeh on the background area of the first two-dimensional image according to the depth information.

Claims (47)

1. An apparatus for image processing, comprising a processor and a depth camera, wherein the processor:

obtains a first two-dimensional image and focusing information of the first two-dimensional image, and obtains depth information of a part or all of a content captured in the first two-dimensional image via the depth camera;

determines a background area of the first two-dimensional image according to the focusing information;

performs bokeh on the background area of the first two-dimensional image according to the depth information,

wherein the depth information of the part or all of the content in the first two-dimensional image comprises a depth value of the content corresponding to at least one pixel in the first two-dimensional image, and

wherein the processor performs different intensity of bokeh on pixels corresponding to different depth values in the background area of the first two-dimensional image.

2. The apparatus according to claim 1 , wherein

the focusing information of the first two-dimensional image comprises position information of a focusing point or a focusing area of the first two-dimensional image; and

the processor determines a main area of the first two-dimensional image and the background area of the main area according to the position information of the focusing point or the focusing area.

3. The apparatus according to claim 1 , wherein

the position information of the focusing point or the focusing area comprises coordinate information of the focusing point or the focusing area.

4. The apparatus according to claim 1 , wherein

the processor obtains a second two-dimensional image of the content in the first two-dimensional image; and

the processor obtains the depth information of the part or all of the content in the first two-dimensional image according to the first two-dimensional image and the second two-dimensional image.

5. The apparatus according to claim 4 , wherein

the processor obtains the depth information of the part or all of the content by a stereo matching algorithm according to the first two-dimensional image and the second two-dimensional image.

6. The apparatus according to claim 1 , wherein

the processor obtains a mapping relationship between depth information of the part or all of the content and corresponding pixels in the first two-dimensional image.

7. The apparatus according to claim 6 , wherein

the depth information comprises depth map;

the processor obtains a mapping relationship between coordinates of pixels in the depth map of the part or all of the content and coordinates of the corresponding pixels in the first two-dimensional image.

8. The apparatus according to claim 7 , wherein the processor:

obtains a mapping relationship between the coordinates of the pixels in the depth map and coordinates of corresponding pixels of the content in a three-dimensional coordinate system of a depth camera;

obtains a mapping relationship between the coordinates of the pixels of the content in the three-dimensional coordinate system of the depth camera and coordinates of corresponding pixels of the content in the three-dimensional coordinate system of a non-depth camera which captured the first two-dimensional image; and

obtains a mapping relationship between the coordinates of the pixels of the content in the three-dimensional coordinate system of the non-depth camera and the coordinates of the corresponding pixels in the first two-dimensional image of the content.

9. The apparatus according to claim 8 , wherein the processor corrects the mapping relationship between the coordinates of the pixels of the content in the three-dimensional coordinate system of the non-depth camera and the coordinates of the corresponding pixels in the first two-dimensional image according to a distortion coefficient of the non-depth camera.

10. The apparatus according to claim 8 , wherein the non-depth camera is an RGB camera.

11. The apparatus according to claim 8 , wherein the non-depth camera and the depth camera are placed side by side.

12. The apparatus according to claim 7 , wherein the processor corrects the depth map according to a distortion coefficient of the depth camera.

13. The apparatus according to claim 1 ,

wherein:

the processor obtains focusing indicating information, wherein the focusing indicating information indicates whether focusing is being performed;

the processor determines that the focusing indicating information indicates focusing is being performed;

the processor obtains a third two-dimensional image and focusing information of the third two-dimensional image after focusing is completed and obtain depth information of a part or all of the content in the third two-dimensional image;

the processor determines a background area of the third two-dimensional image according to the focusing information; and

the processor performs bokeh on the background area of the third two-dimensional image according to the depth information of the part or all of the content in the third two-dimensional image,

or wherein:

the processor determines the focusing indicating information indicates the focusing is not being performed,

the processor obtains a fourth two-dimensional image which is the next frame of the first two-dimensional image and obtain depth information of a part or all of the content in the fourth two-dimensional image,

the processor determines a background area of the fourth two-dimensional image according to the focusing information of the fourth two-dimensional image, and

the processor performs bokeh on the background area of the fourth two-dimensional image according to the depth information.

14. The apparatus according to claim 13 , wherein

the depth information of the part or all of the content in the third two-dimensional image comprises a depth value of the content corresponding to at least one pixel in the third two-dimensional image; and

the processor performs different intensity of bokeh on pixels corresponding to different depth values in the background area of the third two-dimensional image.

15. The apparatus according to claim 13 , wherein

the depth information of the part or all of the content in the fourth two-dimensional image comprises a depth value of the content corresponding to at least one pixel in the fourth two-dimensional image; and

the processor performs different intensity of bokeh on pixels corresponding to different depth values in the background area of the fourth two-dimensional image.

Priority Claims (1)
CN 201710802117.5 · Sep 7, 2017 · national
Continuity (2)
Continuation 16123610 · Sep 6, 2018
Related Publication 20200410646A1 · Dec 31, 2020