IP Library › Granted Patent US 11,856,330
Granted Patent B2
US 11,856,330 · App. 18/167,925 · Granted Dec 26, 2023

Vehicular driver monitoring system

Inventor: Zhongyao Liu (Troy, MI)
Assignee: Magna Electronics Inc.
H04N7/18B60R1/04B60R1/08B60R11/04B60R16/037G06V20/597B60R2011/0003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,856,330
App. No.
18/167,925
Granted
Dec 26, 2023
Kind
B2
Abstract

A vehicular driver monitoring system includes a camera disposed within and viewing within an interior cabin of a vehicle. The camera includes a lens and an image sensor. The camera is operable to capture image data. Electronic circuitry of an electronic control unit (ECU) includes an image processor for processing image data captured by the camera. With a driver of the vehicle sitting in a driver seat of the vehicle, light is reflected off a portion of the driver to impinge at the lens of the camera. The vehicular driver monitoring system, via processing at the ECU of image data captured by the camera, determines a deficiency in captured image data arising from light impinging at the lens. The determined deficiency in captured image data arises from an occlusion at the lens of the camera.

Claims (55)

1. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin of a vehicle equipped with the vehicular driver monitoring system, the camera viewing within the interior cabin of the equipped vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light is reflected off a portion of the driver to impinge at the lens of the camera;

wherein the vehicular driver monitoring system, via processing at the ECU of image data captured by the camera, determines a deficiency in captured image data arising from light impinging at the lens; and

wherein the determined deficiency in captured image data arises from an occlusion, and wherein the occlusion is at the lens of the camera.

2. The vehicular driver monitoring system of claim 1 , wherein the vehicular driver monitoring system, responsive to processing at the ECU of image data captured by the camera, adjusts light reflection off the portion of the driver that impinges at the lens of the camera.

3. The vehicular driver monitoring system of claim 1 , wherein the occlusion is at a first region of the lens of the camera, and wherein a second region of the lens is not occluded.

4. The vehicular driver monitoring system of claim 3 , wherein image data arising from light reflected off the portion of the driver impinges at the second region of the lens of the camera, and wherein image data arising from the light reflected off the portion of the driver that impinges at the second region of the lens of the camera is processed at the ECU.

5. The vehicular driver monitoring system of claim 1 , wherein a center region of the lens of the camera images light at the image sensor with reduced distortion as compared to perimeter regions of the lens.

6. The vehicular driver monitoring system of claim 5 , wherein light reflected off the portion of the driver impinges at the center region the lens of the camera.

7. The vehicular driver monitoring system of claim 1 , wherein the portion of the driver comprises the eyes of the driver.

8. The vehicular driver monitoring system of claim 1 , wherein the portion of the driver comprises a hand of the driver of the equipped vehicle.

9. The vehicular driver monitoring system of claim 8 , wherein a center region of the lens of the camera images light at the image sensor with reduced distortion as compared to perimeter regions of the lens, and wherein, responsive to determination of movement of the hand of the driver, the vehicular driver monitoring system adjusts light reflection off the portion of the driver so that light reflected off the hand of the driver impinges at the center region of the lens of the camera.

10. The vehicular driver monitoring system of claim 1 , wherein the light reflected off the portion of the driver that impinges at the lens of the camera is reflected off a mirror reflector of an adjustable mirror disposed within the interior cabin of the equipped vehicle.

11. The vehicular driver monitoring system of claim 10 , wherein the camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror to impinge at the lens of the camera.

12. The vehicular driver monitoring system of claim 11 , wherein the adjustable mirror comprises a pivotable mirror that is pivotable to adjust a position of the reflection of the portion of the driver impinging at the lens of the camera.

13. The vehicular driver monitoring system of claim 11 , wherein the adjustable mirror comprises a microelectromechanical system (MEMS) mirror.

14. The vehicular driver monitoring system of claim 11 , wherein the adjustable mirror is adjustable about at least two axes.

15. The vehicular driver monitoring system of claim 11 , wherein the adjustable mirror is adjustable at least about a horizontal axis to adjust a vertical position of the reflection of the portion of the driver impinging at the lens of the camera.

16. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin of a vehicle equipped with the vehicular driver monitoring system, the camera viewing within the interior cabin of the equipped vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light is reflected off a portion of the driver to impinge at the lens of the camera;

wherein a center region of the lens of the camera images light at the image sensor with reduced distortion as compared to perimeter regions of the lens;

wherein the vehicular driver monitoring system, via processing at the ECU of image data captured by the camera, determines a deficiency in captured image data arising from light impinging at the lens;

wherein the determined deficiency in captured image data arises from an occlusion, and wherein the occlusion is at the lens of the camera; and

wherein the occlusion is at a first region of the lens of the camera, and wherein a second region of the lens is not occluded.

17. The vehicular driver monitoring system of claim 16 , wherein image data arising from light reflected off the portion of the driver impinges at the second region of the lens of the camera, and wherein image data arising from the light reflected off the portion of the driver that impinges at the second region of the lens of the camera is processed at the ECU.

18. The vehicular driver monitoring system of claim 16 , wherein the vehicular driver monitoring system, responsive to processing at the ECU of image data captured by the camera, adjusts light reflection off the portion of the driver that impinges at the lens of the camera.

19. The vehicular driver monitoring system of claim 16 , wherein light reflected off the portion of the driver impinges at the center region the lens of the camera.

20. The vehicular driver monitoring system of claim 16 , wherein the portion of the driver comprises the eyes of the driver.

21. The vehicular driver monitoring system of claim 16 , wherein the portion of the driver comprises a hand of the driver of the equipped vehicle.

22. The vehicular driver monitoring system of claim 21 , wherein, responsive to determination of movement of the hand of the driver, the vehicular driver monitoring system adjusts light reflection off the portion of the driver so that light reflected off the hand of the driver impinges at the center region of the lens of the camera.

23. The vehicular driver monitoring system of claim 16 , wherein the light reflected off the portion of the driver that impinges at the lens of the camera is reflected off a mirror reflector of an adjustable mirror disposed within the interior cabin of the equipped vehicle.

24. The vehicular driver monitoring system of claim 23 , wherein the camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror to impinge at the lens of the camera.

25. A vehicular driver monitoring system, the vehicular driver monitoring system comprising:

a camera disposed within an interior cabin of a vehicle equipped with the vehicular driver monitoring system, the camera viewing within the interior cabin of the equipped vehicle, the camera comprising a lens and an image sensor, wherein the camera is operable to capture image data;

wherein the image sensor comprises a two-dimensional array of at least one million photosensor elements arranged in a plurality of rows and columns;

an electronic control unit (ECU) comprising electronic circuitry and associated software;

wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;

wherein, with a driver of the equipped vehicle sitting in a driver seat of the equipped vehicle, light is reflected off a portion of the driver to impinge at the lens of the camera;

wherein the portion of the driver comprises the eyes of the driver;

wherein the vehicular driver monitoring system, via processing at the ECU of image data captured by the camera, determines a deficiency in captured image data arising from light impinging at the lens;

wherein the determined deficiency in captured image data arises from an occlusion, and wherein the occlusion is at the lens of the camera; and

wherein the occlusion is at a first region of the lens of the camera, and wherein a second region of the lens is not occluded.

26. The vehicular driver monitoring system of claim 25 , wherein image data arising from light reflected off the portion of the driver impinges at the second region of the lens of the camera, and wherein image data arising from the light reflected off the portion of the driver that impinges at the second region of the lens of the camera is processed at the ECU.

27. The vehicular driver monitoring system of claim 25 , wherein the vehicular driver monitoring system, responsive to processing at the ECU of image data captured by the camera, adjusts light reflection off the portion of the driver that impinges at the lens of the camera.

28. The vehicular driver monitoring system of claim 25 , wherein the light reflected off the portion of the driver that impinges at the lens of the camera is reflected off a mirror reflector of an adjustable mirror disposed within the interior cabin of the equipped vehicle.

29. The vehicular driver monitoring system of claim 28 , wherein the camera views the adjustable mirror and captures image data based on light reflected by the mirror reflector of the adjustable mirror to impinge at the lens of the camera.

30. The vehicular driver monitoring system of claim 29 , wherein the adjustable mirror comprises a pivotable mirror that is pivotable to adjust a position of the reflection of the portion of the driver impinging at the lens of the camera.

Continuity (3)
Continuation 17303917 · Jun 10, 2021
Provisional Application 62705075 · Jun 10, 2020
Related Publication 20230224435A1 · Jul 13, 2023
Cited By (2)
US 12,340,546 US 12,501,128