IP Library › Granted Patent US 11,643,014
Granted Patent B2
US 11,643,014 · App. 17/263,116 · Granted May 9, 2023

Image capturing device and vehicle control system

Inventor: Ryoji Eki (Kanagawa, JP)
Assignee: Sony Semiconductor Solutions Corporation
H04N5/23218G06K9/6232G06N3/08G06V10/462H04N5/2252H04N5/23206H04N5/341H04N5/335H04N2005/2255
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,643,014
App. No.
17/263,116
Granted
May 9, 2023
Kind
B2
Abstract

Fabrication processing is executed in a chip of an image sensor. An image capturing device includes an image capturing unit ( 11 ) mounted on a vehicle and configured to generate image data by performing image capturing of a peripheral region of the vehicle, a scene recognition unit ( 214 ) configured to recognize a scene of the peripheral region based on the image data, and a drive control unit ( 12 ) configured to control drive of the image capturing unit based on the scene recognized by the scene recognition unit.

Claims (57)

1. An image capturing device comprising:

circuitry configured to function as:

an image capturing unit mounted on a vehicle and configured to generate image data by performing image capturing of a peripheral region of the vehicle;

a scene recognition unit configured to recognize a scene of the peripheral region based on the image data; and

a drive control unit configured to control drive of the image capturing unit based on the scene recognized by the scene recognition unit; and

a scene recognition control unit configured to generate a control signal for controlling the scene recognition unit,

wherein the scene recognition unit recognizes the scene of the peripheral region based on the image data and the control signal.

2. The image capturing device according to claim 1 , further comprising a signal processing unit configured to execute signal processing on the image data.

3. The image capturing device according to claim 2 , further comprising an output unit configured to output the image data on which the signal processing is executed.

4. The image capturing device according to claim 1 , wherein

the scene recognition unit determines whether the scene is a scene for which image data needs to be acquired at high resolution, and

the drive control unit

controls drive of the image capturing unit to generate image data having a first resolution when the scene recognition unit determines that the scene is a scene for which image data needs to be acquired at high resolution, and

controls drive of the image capturing unit to generate image data having a second resolution lower than the first resolution when the scene recognition unit determines that the scene is a scene for which image data does not need to be acquired at high resolution.

5. The image capturing device according to claim 4 , wherein the scene recognition unit calculates illuminance in the peripheral region based on the image data and controls drive of the image capturing unit to generate image data having the second resolution when the calculated illuminance exceeds a predetermined threshold.

6. The image capturing device according to claim 4 , wherein

the image capturing unit includes a pixel array unit constituted by a plurality of pixels arrayed in a matrix of rows and columns, and

the scene recognition unit instructs the drive control unit to read image data from a specific region that is a partial region of the pixel array unit when determining that image data needs to be acquired at high resolution.

7. The image capturing device according to claim 6 , wherein the scene recognition unit determines whether the scene is a scene in which a curve is included in the peripheral region, and shifts a position of the specific region in the pixel array unit when determining that the scene is a scene in which a curve is included in the peripheral region.

8. The image capturing device according to claim 4 , further comprising a signal processing unit configured to execute signal processing on the image data, wherein

when determining that image data needs to be acquired at high resolution, the scene recognition unit causes the signal processing unit to execute processing of extracting, from image data read from the image capturing unit, image data of a specific region that is a part of the read image data.

9. The image capturing device according to claim 8 , wherein the scene recognition unit determines whether the scene is a scene in which a curve is included in the peripheral region, and shifts a position of the specific region in the image data when determining that the scene is a scene in which a curve is included in the peripheral region.

10. The image capturing device according to claim 7 , wherein when determining that the scene is a scene in which a curve is included in the peripheral region, the scene recognition unit shifts the specific region based on a shape of the curve.

11. The image capturing device according to claim 4 , wherein

the image capturing unit includes a pixel array unit constituted by a plurality of pixels arrayed in a matrix of rows and columns, and

the scene recognition unit determines whether the scene is a scene in which a slope is included in the peripheral region, and instructs the drive control unit to read image data from a specific region that is a partial region of the pixel array unit when determining that the scene is a scene in which a slope is included in the peripheral region.

12. The image capturing device according to claim 4 , further comprising a signal processing unit configured to execute signal processing on the image data, wherein

the scene recognition unit determines whether the scene is a scene in which a slope is included in the peripheral region, and when determining that the scene is a scene in which a slope is included in the peripheral region, the scene recognition unit causes the signal processing unit to execute processing of extracting, from image data read from the image capturing unit, image data of a specific region that is a part of the read image data.

13. The image capturing device according to claim 1 , wherein

the image capturing unit includes

a first photoelectrical conversion unit,

a second photoelectrical conversion unit,

a first transfer gate connected with the first photoelectrical conversion unit,

a second transfer gate connected with the second photoelectrical conversion unit, and

a floating diffusion connected with the first transfer gate and the second transfer gate, and

the drive control unit controls drive of the first and the second transfer gates based on the scene recognized by the scene recognition unit.

14. The image capturing device according to claim 13 , wherein

the image capturing unit includes a pixel array unit constituted by a plurality of pixels arrayed in a matrix of rows and columns,

the drive control unit includes

a vertical drive circuit configured to control drive of the pixels for each row, and

a horizontal drive circuit configured to control drive of the pixels for each column,

the vertical drive circuit controls drive of the pixels for each row based on the scene recognized by the scene recognition unit, and

the horizontal drive circuit controls drive of the pixel units for each column based on the scene recognized by the scene recognition unit.

15. A vehicle control system comprising:

circuitry configured to function as:

an image capturing unit mounted on a vehicle and configured to generate image data by performing image capturing of a peripheral region of the vehicle;

a scene recognition unit configured to recognize the scene of the peripheral region based on the image data;

a drive control unit configured to control drive of the image capturing unit based on the scene recognized by the scene recognition unit;

an object detection unit configured to detect an object in the peripheral region based on the image data;

a vehicle control signal generation unit configured to generate, based on a result of the detection by the object detection unit, a vehicle control signal for controlling the vehicle;

a vehicle control unit configured to control a vehicle drive unit based on the vehicle control signal; and

a scene recognition control unit configured to generate a control signal for controlling the scene recognition unit,

wherein the scene recognition unit recognizes the scene of the peripheral region based on the image data and the control signal.

16. The vehicle control system according to claim 15 , further comprising a vehicle drive sensor configured to detect a state of the vehicle drive unit, wherein

the scene recognition control unit generates the control signal based on a result of the detection by the vehicle drive sensor.

17. The vehicle control system according to claim 15 , further comprising a map information storage unit configured to store map information, wherein

the scene recognition control unit generates the control signal based on the map information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2021
From: EKI, RYOJI
To: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
Reel/Frame 056205/0127 →
Priority Claims (3)
JP JP2018-144173 · Jul 31, 2018 · national
JP JP2019-139196 · Jul 29, 2019 · national
JP JP2019-140197 · Jul 30, 2019 · national
Continuity (1)
Related Publication 20210152732A1 · May 20, 2021