IP Library › Granted Patent US 10,999,488
Granted Patent B2
US 10,999,488 · App. 16/493,599 · Granted May 4, 2021

Control device, imaging device, and control method

Inventor: Yu Kimishima (Tokyo, JP)
Assignee: SONY CORPORATION
H04N5/23212G03B13/36H04N5/23219H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,999,488
App. No.
16/493,599
Granted
May 4, 2021
Kind
B2
Abstract

A control device including a main subject detection unit that detects a first main subject from among subjects included in a first image, and acquires main subject information indicating the first main subject, a distance information calculation unit that detects a distance of a subject included in a first image, and acquires first distance information indicating the distance, and a detection unit that detects a distance within which a first main subject is included on the basis of main subject information and first distance information, and acquires main subject distance information indicating the distance within which the first main subject is included.

Claims (110)

1. A control device, comprising:

circuitry configured to:

control acquisition of a first image;

detect a first main subject from a first plurality of subjects in the first image;

acquire specific main subject information indicating the first main subject;

detect a first plurality of distances of the first plurality of subjects in the first image, wherein

each distance of the first plurality of distances corresponds to a respective subject of the first plurality of subjects, and

the each distance is from a first point to the respective subject in a depth direction in the first image;

acquire first distance information indicating the first plurality of distances;

detect a first distance from the first plurality of distances in the first image, wherein

the first distance is of the first main subject in the first image, and

the first distance is detected based on the specific main subject information and the first distance information;

acquire main subject distance information indicating the first distance of the first main subject;

control acquisition of a second image subsequent to the first image;

detect a second plurality of distances of a second plurality of subjects in the second image, wherein

each distance of the second plurality of distances corresponds to a respective subject of the second plurality of subjects, and

the each distance of the second plurality of distances is from a second point to the respective subject, of the second plurality of subjects, in a depth direction in the second image;

acquire second distance information that indicates the second plurality of distances; and

detect a second main subject from the second plurality of subjects in the second image based on the main subject distance information and the second distance information.

2. The control device according to claim 1 , wherein the circuitry is further configured to control lens movement of a lens of an imaging system based on the detection of the second main subject.

3. The control device according to claim 1 , wherein the circuitry is further configured to control lens movement of a lens of an imaging system to:

focus on a first area, in an imaging range of the imaging system, based on presence of the second main subject in the first area;

expand the first area, to a second area in the imaging range, based on absence of the second main subject is in the first area; and

focus on the second area in the imaging range.

4. The control device according to claim 3 , wherein the circuitry is further configured to control the lens movement to:

focus on the second area based on presence of the second main subject in the second area; and

focus on the first area based on absence of the second main subject in the second area.

5. The control device according to claim 1 , wherein the circuitry is further configured to disable a stabilization process based on:

absence of the first main subject in a first area in an imaging range of an imaging system, and

presence of the second main subject in the first area.

6. The control device according to claim 1 , wherein the circuitry is further configured to enable a stabilization process based on:

presence of the first main subject in a first area in an imaging range of an imaging system, and

absence of the second main subject in the first area.

7. The control device according to claim 1 , wherein the circuitry is further configured to:

classify the first distance information into a plurality of clusters; and

acquire cluster information, indicating a cluster of the plurality of clusters, as the main subject distance information, wherein

the cluster information is acquired based on the specific main subject information, and

a proportion of the first main subject in the cluster is one of equal to or more than a threshold.

8. The control device according to claim 1 , wherein the circuitry is further configured to:

classify the first distance information into a plurality of clusters, and

acquire cluster information, indicating a cluster of the plurality of clusters, as the main subject distance information, wherein

the cluster information is acquired based on the specific main subject information, and

the cluster has a largest proportion of the first main subject among the plurality of clusters.

9. The control device according to claim 3 , wherein the circuitry is further configured to:

control a display screen to display a plurality of selectable autofocus areas including the first area; and

receive a user selection of the first area among the displayed plurality of selectable autofocus areas.

10. The control device according to claim 9 , wherein the second area is not displayed on the display screen.

11. The control device according to claim 9 , wherein the circuitry is further configured to control the display screen to temporarily display the second area.

12. The control device according to claim 5 , wherein the circuitry is further configured to control execution of the stabilization process to fix a lens position of a lens of the imaging system.

13. An imaging device, comprising:

an image sensor; and

a control device, wherein the control device is configured to:

control the image sensor to:

acquire a first image; and

acquire a second image subsequent to the first image;

detect a first main subject from a first plurality of subjects in the first image;

acquire specific main subject information indicating the first main subject;

detect a first plurality of distances of the first plurality of subjects in the first image, wherein

each distance of the first plurality of distances corresponds to a respective subject of the first plurality of subjects, and

the each distance is from a first point to the respective subject in a depth direction in the first image;

acquire first distance information indicating the first plurality of distances;

detect a first distance from the first plurality of distances in the first image, wherein

the first distance is of the first main subject in the first image, and

the first distance is detected based on the specific main subject information and the first distance information;

acquire main subject distance information indicating the first distance of the first main subject;

detect a second plurality of distances of a second plurality of subjects in the second image, wherein

each distance of the second plurality of distances corresponds to a respective subject of the second plurality of subjects, and

the each distance of the second plurality of distances is from a second point to the respective subject, of the second plurality of subjects, in a depth direction in the second image;

acquire second distance information that indicates the second plurality of distances; and

detect a second main subject from the second plurality of subjects in the second image based on the main subject distance information and the second distance information.

14. The imaging device according to claim 13 , wherein the control device is further configured to control a display screen to present a distance, of the second plurality of distances, of the second main subject.

15. The imaging device according to claim 13 , further comprising:

a lens; and

a lens drive mechanism, wherein the control device is further configured to control the lens drive mechanism to control lens movement of the lens.

16. A control method, comprising:

controlling acquisition of a first image;

detecting a first main subject from a first plurality of subjects in the first image;

acquiring specific main subject information indicating the first main subject;

detecting a first plurality of distances of the first plurality of subjects in the first image, wherein

each distance of the first plurality of distances corresponds to a respective subject of the first plurality of subjects, and

the each distance is from a first point to the respective subject in a depth direction in the first image;

acquiring first distance information indicating the first plurality of distances;

detecting a first distance from the first plurality of distances in the first image, wherein

the first distance is of the first main subject in the first image, and

the first distance is detected based on the specific main subject information and the first distance information;

acquiring main subject distance information indicating the first distance of the first main subject;

controlling acquisition of a second image subsequent to the first image;

detecting a second plurality of distances of a second plurality of subjects in the second image, wherein

each distance of the second plurality of distances corresponds to a respective subject of the second plurality of subjects, and

the each distance of the second plurality of distances is from a second point to the respective subject, of the second plurality of subjects, in a depth direction in the second image;

acquiring second distance information that indicates the second plurality of distances; and

detecting a second main subject from the second plurality of subjects in the second image based on the main subject distance information and the second distance information.

17. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:

controlling acquisition of a first image;

detecting a first main subject from a first plurality of subjects in the first image;

acquiring specific main subject information indicating the first main subject;

detecting a first plurality of distances of the first plurality of subjects in the first image, wherein

each distance of the first plurality of distances corresponds to a respective subject of the first plurality of subjects, and

the each distance is from a first point to the respective subject in a depth direction in the first image;

acquiring first distance information indicating the first plurality of distances;

detecting a first distance from the first plurality of distances in the first image, wherein

the first distance is of the first main subject in the first image, and

the first distance is detected based on the specific main subject information and the first distance information;

acquiring main subject distance information indicating the first distance of the first main subject;

controlling acquisition of a second image subsequent to the first image;

detecting a second plurality of distances of a second plurality of subjects in the second image, wherein

each distance of the second plurality of distances corresponds to a respective subject of the second plurality of subjects, and

the each distance of the second plurality of distances is from a second point to the respective subject, of the second plurality of subjects, in a depth direction in the second image;

acquiring second distance information that indicates the second plurality of distances; and

detecting a second main subject from the second plurality of subjects in the second image based on the main subject distance information and the second distance information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2019
From: KIMISHIMA, YU
To: SONY CORPORATION
Reel/Frame 050372/0227 →
Priority Claims (1)
JP JP2017-072821 · Mar 31, 2017 · national
Continuity (1)
Related Publication 20200084369A1 · Mar 12, 2020
Cited By (1)
US 12,315,179