IP Library › Granted Patent US 10,432,853
Granted Patent B2
US 10,432,853 · App. 15/443,012 · Granted Oct 1, 2019

Image processing for automatic detection of focus area

Inventor: Pingshan Li (Sunnyvale, CA)
Assignee: SONY CORPORATION
H04N5/23229G06K9/00228G06K9/00369G06K9/3241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,432,853
App. No.
15/443,012
Granted
Oct 1, 2019
Kind
B2
Abstract

A method and a device, such as an imaging device, for image processing for automatic detection of focus area are disclosed herein. In accordance with an embodiment, the device for image processing is the imaging device that is configured to extract a plurality of object features of a plurality of objects in a field-of-view (FOV) of the imaging device. A plurality of confidence maps are generated based on the extracted plurality of object features. The imaging device is further configured to determine a focus area corresponding to the FOV, based on the generated plurality of confidence maps by use of a pre-determined rule.

Claims (62)

1. A method for image processing, said method comprising:

extracting, by an imaging device, a plurality of object features of a plurality of objects,

wherein the plurality of objects are in a field-of-view (FOV) of said imaging device;

generating, by said imaging device, a plurality of confidence maps based on said extracted plurality of object features;

determining, by said imaging device, a focus area corresponding to said FOV based on said generated plurality of confidence maps and a specific rule.

2. The method according to claim 1 , further comprising

capturing, by said imaging device, an image of said FOV, based on said determined focus area.

3. The method according to claim 1 , further comprising

setting, by said imaging device, said determined focus area to a center portion of said FOV when said specific rule specifies that a depth-of-field (DOF) value of said FOV exceeds a DOF threshold value.

4. The method according to claim 3 , wherein said DOF value is determined based on setting parameters of said imaging device and depth information of said FOV.

5. The method according to claim 1 , further comprising

detecting, by said imaging device, said plurality of objects in said FOV, wherein said plurality of objects corresponds to at least one human being.

6. The method according to claim 5 , further comprising

setting, by said imaging device, said determined focus area to a first object of said plurality of objects when said specific rule specifies that a motion value of said first object is greater than a motion value of each of remaining objects in said plurality of objects.

7. The method according to claim 5 , further comprising

setting, by said imaging device, said determined focus area to a first object of said plurality of objects when said specific rule specifies that motion values of said plurality of objects are equal and a depth value of said first object is smaller than a depth value of each of remaining objects in said plurality of objects.

8. The method according to claim 5 , further comprising

setting, by said imaging device, said determined focus area to a front view of a facial portion of a first object among said plurality of objects when said specific rule specifies that motion values of said plurality of objects are less than a threshold value and a depth value of said first object is smaller than a depth value of a facial portion portions of each of remaining objects in said plurality of objects.

9. The method according to claim 5 , further comprising

setting, by said imaging device, said determined focus area to a first object among said plurality of objects when said specific rule specifies that facial portions of said plurality of objects are undetected,

wherein said first object is a human being, of said at least one human being, with a smallest depth value when a presence of said at least one human being is detected independent of facial detection of said at least one human being, and

wherein said first object is a non-human object with a smallest depth value when said at least one human being is absent in said FOV.

10. The method according to claim 1 , further comprising

setting, by said imaging device, said determined focus area to a first object among said plurality of objects when said specific rule specifies that a depth value of said first object is smaller than a depth value of each of remaining objects in said plurality of objects.

11. The method according to claim 1 , wherein said specific rule is based on at least one of a lens setting, depth information, or motion information of said plurality of objects in said FOV of said imaging device.

12. The method according to claim 1 , further comprising

generating, by said imaging device, a final confidence map.

13. The method according to claim 12 , wherein said determination of said focus area is within said final confidence map of said FOV.

14. An imaging device, comprising:

circuitry configured to:

extract a plurality of object features of a plurality of objects,

wherein the plurality of objects are in a field-of-view (FOV) of said imaging device;

generate a plurality of confidence maps based on said extracted plurality of object features; and

determine a focus area corresponding to said FOV based on said generated plurality of confidence maps and a specific rule.

15. The imaging device according to claim 14 , wherein said circuitry is further configured to capture an image of said FOV, based on said determined focus area.

16. The imaging device according to claim 15 , wherein said circuitry is further configured to set said determined focus area to a center portion of said FOV when said specific rule specifies that a depth-of-field (DOF) value of said FOV exceeds a DOF threshold value.

17. The imaging device according to claim 16 , wherein said DOF value is determined based on setting parameters of said imaging device and depth information of said FOV.

18. The imaging device according to claim 14 , wherein

said FOV comprises said plurality of objects, and

said plurality of objects correspond to at least one human being.

19. The imaging device according to claim 18 , wherein said circuitry is further configured to set said determined focus area to a first object of said plurality of objects when said specific rule specifies that a motion value of said first object is greater than a motion value of each of remaining objects in said plurality of objects.

20. The imaging device according to claim 18 , wherein said circuitry is further configured to set said determined focus area to a first object of said plurality of objects when said specific rule specifies that motion values of said plurality of objects are equal and a depth value of said first object is smaller than a depth value of each of remaining objects in said plurality of objects.

21. The imaging device according to claim 18 , wherein said circuitry is further configured to set said determined focus area to a front view of a facial portion of a first object among said plurality of objects when said specific rule specifies that motion values of said plurality of objects are less than a threshold value and a depth value of said first object is smaller than a depth value of facial portion of each of remaining objects in said plurality of objects.

22. The imaging device according to claim 18 , wherein said circuitry is further configured to set said determined focus area to a first object among said plurality of objects when said specific rule specifies that facial portions of said plurality of objects are undetected,

wherein said first object is a human being, of said at least one human being, with a smallest depth value when a presence of said at least one human being is detected independent of facial detection of said at least one human being, and

wherein said first object is a non-human object with a smallest depth value when said at least one human being is absent in said FOV.

23. The imaging device according to claim 14 , wherein said circuitry is further configured to set said determined focus area to a first object among said plurality of objects when said specific rule specifies that a depth value of said first object is smaller than a depth value of each of remaining objects in said plurality of objects.

24. The imaging device according to claim 14 , wherein said specific rule is based on at least one of a lens setting, depth information, or motion information of said plurality of objects in said FOV of said imaging device.

25. The imaging device according to claim 14 , wherein said circuitry is further configured to generate a final confidence map based on said plurality of confidence maps.

26. The imaging device according to claim 25 , wherein said determination of said focus area is within said FOV of said final confidence map.

27. A non-transitory computer-readable medium having stored thereon, computer-executable instructions that, when executed by a processor, cause the processor to execute operations, the operations comprising:

extracting a plurality of object features of a plurality of objects,

wherein the plurality of objects are in a field-of-view (FOV) of an imaging device;

generating a plurality of confidence maps based on said extracted plurality of object features; and

determining a focus area corresponding to said FOV based on said generated plurality of confidence maps and a specific rule.

28. A method for image processing, said method comprising:

extracting, by an imaging device, a plurality of object features of a plurality of objects,

wherein the plurality of objects are in a field-of-view (FOV) of said imaging device;

generating, by said imaging device, a plurality of confidence maps based on said extracted plurality of object features;

generating, by said imaging device, a final confidence map based on said plurality of confidence maps; and

determining, by said imaging device, a focus area corresponding to said FOV based on said generated final confidence map and a specific rule,

wherein said determination of said focus area is within said final confidence map of said FOV.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2017
From: LI, PINGSHAN
To: SONY CORPORATION
Reel/Frame 042415/0422 →
Continuity (1)
Related Publication 20180249072A1 · Aug 30, 2018