IP Library › Granted Patent US 10,531,002
Granted Patent B2
US 10,531,002 · App. 15/761,393 · Granted Jan 7, 2020

Image processing device and image processing method

Inventors: Hiroaki Takahashi (Tokyo, JP); Yasunobu Hitomi (Kanagawa, JP); Kentaro Okamura (Kanagawa, JP); Masayuki Tachi (Kanagawa, JP)
Assignee: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
H04N5/2327G03B15/00H04N5/217H04N5/2258H04N5/2353H04N5/23222H04N5/23232H04N5/23258H04N5/243H04N5/247H04N5/347G03B19/026G03B2206/00G03B2207/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,531,002
App. No.
15/761,393
Granted
Jan 7, 2020
Kind
B2
Abstract

The present disclosure relates to an image processing device and an image processing method capable of generating a high-quality composite image. Exposure time and exposure timing of a basic imaging unit which images a basic image and a plurality of auxiliary imaging units which may image under an imaging condition independent from that of the basic imaging unit are controlled according to a motion amount. The present disclosure is applicable to an image processing device.

Claims (109)

1. An image processing device, comprising:

a motion detection unit configured to detect a first motion amount;

an imaging control unit configured to determine each of a first imaging condition and a second imaging condition based on the detected first motion amount;

a first imaging unit configured to:

capture a first image, wherein the first imaging condition is associated with the first imaging unit; and

set a first exposure time based on the detected first motion amount and a motion limit threshold, wherein

the first imaging condition is associated with the first exposure time, and

the first exposure time is a specific exposure time; and

a plurality of second imaging units configured to set the second imaging condition, wherein the second imaging condition is independent from the first imaging condition.

2. The image processing device according to claim 1 , wherein

each of the first imaging unit and the plurality of second imaging units comprises one of;

a first plurality of configurations of image sensors and optical systems or

a second plurality of configurations of a plurality of groups;

the plurality of groups is based on a division of the image sensors and the optical systems, and

each of the first plurality of configurations is independent.

3. The image processing device according to claim 1 , wherein characteristics of the first imaging unit is different from characteristics of each of the plurality of second imaging units.

4. The image processing device according to claim 3 , wherein the characteristics of the first imaging unit is different from the characteristics of each of the plurality of second imaging units based on at least one of an array of a color filter array, an optical lens characteristic, a pixel size, or a sensor size.

5. The image processing device according to claim 1 , wherein

the first exposure time elapses at a specific time, and

the specific time is based on the detected first motion amount that is larger than the motion limit threshold.

6. The image processing device according to claim 1 , wherein

the first imaging condition is associated with a first exposure start time and a first exposure finish time,

the second imaging condition is associated with a second exposure start time and a second exposure finish time, and

the plurality of second imaging units is further configured to set one of the second exposure start time equal to the first exposure start time, or the second exposure finish time equal to the first exposure finish time.

7. The image processing device according to claim 1 , wherein

the second imaging condition is a pixel addition mode,

the plurality of second imaging units is further configured to add pixel values of a plurality of surrounding pixels for each pixel of the captured first image in the pixel addition mode,

the pixel values are added based on an exposure value that is smaller than a specific exposure value, and

the exposure value corresponds to the specific exposure time.

8. The image processing device according to claim 1 , wherein

the motion detection unit is further configured to:

calculate at least one of an angular displacement amount or a translational displacement amount; and

detect a second motion amount of a camera based on the at least one of the angular displacement amount or the translational displacement amount, and

the motion detection unit comprises at least one of a gyro sensor or a six-axis sensor.

9. The image processing device according to claim 1 , wherein

the motion detection unit is further configured to detect motion based on at least one of

motion of at least one of the first imaging unit or the plurality of second imaging units, or

motion of a subject in a specific image of a plurality of second images,

the plurality of second images corresponds to one of the first imaging unit or the plurality of second imaging units, and

each of the plurality of second images is associated with a different time.

10. The image processing device according to claim 1 , further comprising:

a composition unit configured to generate a composite image, wherein

the plurality of second imaging units is further configured to capture a plurality of second images, and

the generated composite image is a combination of the captured first image and the captured plurality of second images.

11. An image processing device, comprising:

a motion detection unit configured to detect a motion amount;

an imaging control unit configured to determine each of a first imaging condition and a second imaging condition based on the detected motion amount;

a first imaging unit configured to capture an image, wherein the first imaging condition is associated with the first imaging unit; and

a plurality of second imaging units configured to:

set the second imaging condition, wherein the second imaging condition is independent from the first imaging condition; and

set a exposure time based on the detected motion amount and a motion limit threshold, wherein

the second imaging condition is associated with the exposure time, and

the exposure time is a specific exposure time.

12. The image processing device according to claim 11 , wherein the

exposure time elapses at a specific time, and

the specific time is based on the detected motion amount that is larger than the motion limit threshold.

13. The image processing device according to claim 11 , wherein

the plurality of second imaging units is further configured to set the exposure time based on an exposure time ratio, and

the exposure time ratio comprises an exposure time associated with the first imaging unit.

14. An image processing device, comprising:

a motion detection unit configured to detect a motion amount;

an imaging control unit configured to determine each of a first imaging condition and a second imaging condition based on the detected motion amount;

a first imaging unit configured to capture an image, wherein the first imaging condition is associated with the first imaging unit;

a plurality of second imaging units configured to:

set the second imaging condition, wherein

the second imaging condition is independent from the first imaging condition,

the second imaging condition is associated with an exposure time, and

the exposure time is a specific exposure time, wherein the specific exposure time is based on the detected motion amount and a motion limit threshold determined; and

calculate the specific exposure time based on the exposure time that is elapsed at a time at which the detected motion amount is larger than the motion limit threshold.

15. An image processing method, comprising:

in an image processing device:

capturing an image;

detecting a motion amount; and

determining each of a first imaging condition of a first imaging unit and a second imaging condition of a plurality of second imaging units based on the detected motion amount;

setting an exposure time based on the detected motion amount and a motion limit threshold, wherein

the first imaging condition is associated with the exposure time, and

the exposure time is a specific exposure time; and

setting the second imaging condition for the plurality of second imaging units, wherein the second imaging condition is independent from the first imaging condition.

16. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:

capturing an image;

detecting a motion amount; and

determining each of a first imaging condition of a first imaging unit and a second imaging condition of a plurality of second imaging units based on the detected motion amount;

setting an exposure time based on the detected motion amount and a motion limit threshold, wherein

the first imaging condition is associated with the exposure time, and

the exposure time is a specific exposure time; and

setting the second imaging condition for the plurality of second imaging units, wherein the second imaging condition is independent from the first imaging condition.

17. An image processing device, comprising:

a motion detection unit configured to detect a motion amount;

an imaging control unit configured to determine each of a first imaging condition and a second imaging condition based on the detected motion amount;

a first imaging unit configured to capture an image, wherein

the first imaging condition is associated with the first imaging unit, and

the first imaging condition is further associated with a first exposure start time and a first exposure finish time;

a plurality of second imaging units configured to:

set the second imaging condition, wherein

the second imaging condition is independent from the first imaging condition, and

the second imaging condition is associated with a second exposure start time and a second exposure finish time; and

set one of the second exposure start time equal to the first exposure start time, or the second exposure finish time equal to the second exposure finish time based on the captured image being a still image.

18. An image processing device, comprising:

a motion detection unit configured to detect a motion amount;

an imaging control unit configured to determine each of a first imaging condition and a second imaging condition based on the detected motion amount;

a first imaging unit configured to:

capture an image, wherein the first imaging condition is associated with the first imaging unit,

a plurality of second imaging units configured to:

set the second imaging condition, wherein

the second imaging condition is independent from the first imaging condition, and

the second imaging condition is a pixel addition mode; and

add pixel values of a plurality of surrounding pixels for each pixel of the captured image in the pixel addition mode, wherein

the pixel values are added based on an exposure value that is smaller than a specific exposure value, and

the exposure value corresponds to a specific exposure time.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2018
From: TAKAHASHI, HIROAKI; HITOMI, YASUNOBU; OKAMURA, KENTARO; TACHI, MASAYUKI
To: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
Reel/Frame 045278/0747 →
Priority Claims (1)
JP 2015-197210 · Oct 2, 2015 · national
Continuity (1)
Related Publication 20180278822A1 · Sep 27, 2018
Cited By (1)
US 12,395,729