IP Library Granted Patent US 11,582,425
Granted Patent B2
US 11,582,425 · App. 17/303,917 · Granted Feb 14, 2023

Driver monitoring system using camera with adjustable mirror

Inventor: Zhongyao Liu (Troy, MI)
Assignee: MAGNA ELECTRONICS INC.
H04N7/18B60R1/04B60R1/08B60R11/04B60R16/037G06V20/597B60R2011/0003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,582,425
App. No.
17/303,917
Granted
Feb 14, 2023
Kind
B2
Abstract

A vehicular driver monitoring system includes a camera disposed within a vehicle and having a field of view interior of the vehicle, and an adjustable mirror disposed within the vehicle. The adjustable mirror includes a mirror reflector that reflects light incident thereat. The camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror toward the camera. With a driver of the vehicle sitting in a driver seat of the vehicle, light reflected off a portion of the driver is reflected by the mirror reflector of the adjustable mirror toward the camera. An electronic control unit (ECU), responsive to processing by an image processor of image data captured by the camera, adjusts the adjustable mirror to reflect the portion of the driver toward a region of the lens of the camera.

Claims (51)

1. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin at an instrument cluster of a vehicle equipped with the vehicular driver monitoring system, the camera viewing upward within the interior cabin of the vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an adjustable mirror disposed within the interior cabin of the vehicle above the camera, the adjustable mirror comprising a mirror reflector that reflects light incident thereat;

wherein the camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror toward the lens of the camera;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light reflected off a portion of the driver is reflected downward by the mirror reflector of the adjustable mirror toward the lens of the camera;

wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror so that a center region of the mirror reflector reflects the portion of the driver toward the lens of the camera;

wherein the ECU, via processing of captured image data, determines a deficiency in captured image data representative of the portion of the driver, and wherein the ECU, responsive to determining the deficiency in the captured image data, adjusts the adjustable mirror to reflect the portion of the driver from a first region of the lens toward a second region of the lens; and

wherein the determined deficiency comprises an occlusion at the lens of the camera, and wherein the portion of the driver, when reflected toward the second region of the lens, is not occluded.

2. The vehicular driver monitoring system of claim 1 , wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror to reflect the portion of the driver toward a center region the lens of the camera, and wherein the center region of the lens images light at the image sensor with reduced distortion as compared to perimeter regions of the lens.

3. The vehicular driver monitoring system of claim 1 , wherein the adjustable mirror comprises a microelectromechanical system (MEMS) mirror.

4. The vehicular driver monitoring system of claim 1 , wherein the adjustable mirror is adjustable about at least two pivot axes.

5. The vehicular driver monitoring system of claim 1 , wherein the adjustable mirror is adjustable at least about a horizontal pivot axis.

6. The vehicular driver monitoring system of claim 1 , wherein the portion of the driver comprises the eyes of the driver.

7. The vehicular driver monitoring system of claim 6 , wherein the vehicular driver monitoring system adjusts the adjustable mirror so that the center region of the mirror reflector reflects a different portion of the driver toward the lens of the camera.

8. The vehicular driver monitoring system of claim 7 , wherein the different portion of the driver comprises a hand of the driver.

9. The vehicular driver monitoring system of claim 8 , wherein the vehicular driver monitoring system adjusts the adjustable mirror to reflect the hand of the driver toward the center region of the lens responsive to determination of movement of the hand of the driver.

10. The vehicular driver monitoring system of claim 1 , wherein the adjustable mirror comprises a pivotable mirror that is pivotable by the ECU to adjust a vertical and/or lateral position of the reflection of the portion of the driver at the lens of the camera.

11. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin at an instrument cluster of a vehicle equipped with the vehicular driver monitoring system, the camera viewing upward within the interior cabin of the vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an adjustable mirror disposed within the interior cabin of the vehicle above the camera, the adjustable mirror comprising a mirror reflector that reflects light incident thereat;

wherein the camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror toward the lens of the camera;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light reflected off the head of the driver is reflected downward by the mirror reflector of the adjustable mirror toward the lens of the camera;

wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror so that a center region of the mirror reflector reflects the head of the driver toward the lens of the camera;

wherein the adjustable mirror is pivotable at least about a horizontal pivot axis;

wherein the ECU, via processing of captured image data, determines a deficiency in captured image data representative of a portion of the driver, and wherein the ECU, responsive to determining the deficiency in the captured image data, adjusts the adjustable mirror to reflect the portion of the driver from a first region of the lens toward a second region of the lens; and

wherein the determined deficiency comprises an occlusion at the lens of the camera, and wherein the portion of the driver, when reflected toward the second region of the lens, is not occluded.

12. The vehicular driver monitoring system of claim 11 , wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror to reflect the head of the driver toward a center region the lens of the camera, and wherein the center region of the lens images light at the image sensor with reduced distortion as compared to perimeter regions of the lens.

13. The vehicular driver monitoring system of claim 12 , wherein the vehicular driver monitoring system adjusts the adjustable mirror to reflect a different portion of the driver toward the center region of the lens.

14. The vehicular driver monitoring system of claim 13 , wherein the different portion of the driver comprises a hand of the driver.

15. The vehicular driver monitoring system of claim 14 , wherein the vehicular driver monitoring system adjusts the adjustable mirror to reflect the hand of the driver toward the center region of the lens responsive to determination of movement of the hand of the driver.

16. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin at an instrument cluster of a vehicle equipped with the vehicular driver monitoring system, the camera viewing upward within the interior cabin of the vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an adjustable mirror disposed within the interior cabin of the vehicle above the camera, the adjustable mirror comprising a mirror reflector that reflects light incident thereat;

wherein the camera views the adjustable mirror and captures image data based on light reflected downward by the mirror reflector of the adjustable mirror toward the lens of the camera;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light reflected off a hand of the driver is reflected downward by the mirror reflector of the adjustable mirror toward the lens of the camera;

wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror so that a center region of the mirror reflector reflects the hand of the driver toward the lens of the camera;

wherein the adjustable mirror is vertically and laterally adjustable by the ECU;

wherein the ECU, via processing of captured image data, determines a deficiency in captured image data representative of a portion of the driver, and wherein the ECU, responsive to determining the deficiency in the captured image data, adjusts the adjustable mirror to reflect the portion of the driver from a first region of the lens toward a second region of the lens; and

wherein the determined deficiency comprises an occlusion at the lens of the camera, and wherein the portion of the driver, when reflected toward the second region of the lens, is not occluded.

17. The vehicular driver monitoring system of claim 16 , wherein the ECU, responsive to processing by the image processor of image data captured by the camera, adjusts the adjustable mirror to reflect the head of the driver toward a center region the lens of the camera, and wherein the center region of the lens images light at the image sensor with reduced distortion as compared to perimeter regions of the lens.

18. The vehicular driver monitoring system of claim 17 , wherein the vehicular driver monitoring system adjusts the adjustable mirror to reflect a different portion of the driver toward the center region of the lens.

19. The vehicular driver monitoring system of claim 18 , wherein the vehicular driver monitoring system adjusts the adjustable mirror to reflect the different portion of the driver toward the center region of the lens responsive to determination of movement of the different portion of the driver.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 10, 2021
From: LIU, ZHONGYAO
To: MAGNA ELECTRONICS INC.
Reel/Frame 056499/0954 →
Continuity (2)
Provisional Application 62705075 · Jun 10, 2020
Related Publication 20210392297A1 · Dec 16, 2021
Cited By (22)
US 12,225,278 US 12,246,647 US 12,325,357 US 12,333,770 US 12,340,546 US 12,342,106 US 12,393,089 US 12,403,834 US 12,420,708 US 12,436,611 US 12,447,899 US 12,508,988 US 12,509,003 US 12,534,021 US 12,552,316 US 12,592,171 US 12,633,138 US 12,668,197 US 12,691,822 US 12,691,894 US 12,703,303 US 12,709,222