Vehicular driver monitoring system with event camera
A vehicular driver monitoring system includes an event camera disposed in a cabin of a vehicle and viewing the head of a driver of the vehicle. The dynamic range of the event camera is at least 100 dB. The response time of the event camera is less than 10,000 microseconds. The event camera has a quantum efficiency of at least 15% for near-infrared light having a wavelength of 940 nm. The system, responsive to processing of data output by the event camera, monitors the driver of the vehicle. Responsive at least in part (a) to monitoring of the driver of the vehicle and (b) to vehicle kinematic data provided to the ECU, at least one selected from the group consisting of (i) a driver attentiveness alert is generated, (ii) a driver drowsiness alert is generated, (iii) steering of the vehicle is controlled and (iv) braking of the vehicle is controlled.
1. A vehicular driver monitoring system, the vehicular driver monitoring system comprising: an event camera disposed in a cabin of a vehicle equipped with the vehicular driver monitoring system, the event camera viewing the head of a driver of the equipped vehicle; the event camera comprising a photosensing array having photosensor pixels arranged in rows and columns; wherein the dynamic range of the event camera is at least 100 dB; wherein the response time of the event camera is less than 10,000 microseconds; wherein the event camera has a quantum efficiency (QE) of at least 15% for near-infrared (near-IR) light having a wavelength of 940 nm; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein data output by the event camera is provided to the ECU; wherein the electronic circuitry of the ECU comprises a processor for processing data output by the event camera and provided to the ECU; wherein the vehicular driver monitoring system, responsive to processing at the ECU of data output by the event camera and provided to the ECU, monitors the driver of the equipped vehicle; wherein vehicle kinematic data is provided to the ECU as the equipped vehicle is driven along a road; wherein the vehicle kinematic data provided to the ECU comprises at least one data selected from the group consisting of (i) speed data, (ii) steering angle data, (iii) yaw data, (iv) pitch data and (v) roll data; and wherein, responsive at least in part to monitoring of the driver of the equipped vehicle and responsive at least in part to the vehicle kinematic data provided to the ECU, at least one is selected from the group consisting of (i) a driver attentiveness alert is generated, (ii) a driver drowsiness alert is generated, (iii) steering of the equipped vehicle is controlled and (iv) braking of the equipped vehicle is controlled.
2. The vehicular driver monitoring system of claim 1 , wherein the data output by the event camera is fused with data provided to the ECU from at least one other sensor of the equipped vehicle.
3. The vehicular driver monitoring system of claim 2 , wherein the at least one other sensor comprises a radar sensor disposed at the equipped vehicle.
4. The vehicular driver monitoring system of claim 2 , wherein the at least one other sensor comprises a lidar sensor disposed at the equipped vehicle.
5. The vehicular driver monitoring system of claim 1 , wherein the event camera is disposed at an in-cabin side of a windshield of the equipped vehicle.
6. The vehicular driver monitoring system of claim 5 , wherein the event camera is disposed at an interior rearview mirror assembly of the equipped vehicle.
7. The vehicular driver monitoring system of claim 6 , wherein the interior rearview mirror assembly includes a mirror head that is adjustably disposed at a mounting base that is configured to attach the interior rearview mirror assembly at the in-cabin side of the windshield of the equipped vehicle, the mirror head having a mirror reflective element.
8. The vehicular driver monitoring system of claim 7 , wherein the event camera is disposed within the mirror head.
9. The vehicular driver monitoring system of claim 8 , wherein the event camera views through the mirror reflective element of the mirror head.
10. The vehicular driver monitoring system of claim 9 , wherein the mirror reflective element comprises a mirror transflector, and wherein the mirror transflector transmits near-infrared (near-IR) light incident thereon, transmits visible light incident thereon and reflects visible light incident thereon.
11. The vehicular driver monitoring system of claim 10 , wherein the event camera views through the mirror transflector of the mirror reflective element of the mirror head.
12. The vehicular driver monitoring system of claim 11 , wherein the event camera is covertly disposed behind the mirror transflector of the mirror reflective element of the mirror head.
13. The vehicular driver monitoring system of claim 11 , comprising an illumination source disposed within the mirror head and operable to emit light that passes through the mirror transflector of the mirror reflective element to illuminate the head of the driver of the equipped vehicle.
14. The vehicular driver monitoring system of claim 13 , wherein the illumination source, when operated to emit light, emits near-infrared (near-IR) light.
15. The vehicular driver monitoring system of claim 7 , wherein the ECU is disposed at the interior rearview mirror assembly.
16. The vehicular driver monitoring system of claim 15 , wherein the ECU is disposed within the mirror head of at the interior rearview mirror assembly.
17. The vehicular driver monitoring system of claim 1 , comprising an illumination source operable to emit light to illuminate the head of the driver of the equipped vehicle.
18. The vehicular driver monitoring system of claim 17 , wherein the illumination source, when operated to emit light, emits near-infrared (near-IR) light.
19. The vehicular driver monitoring system of claim 18 , wherein the event camera comprises a photosensing array having at least one million photosensor pixels arranged in rows and columns.
20. The vehicular driver monitoring system of claim 18 , wherein the event camera has a quantum efficiency (QE) of at least 20% for near-infrared (near-IR) light having a wavelength of 940 nm.
21. The vehicular driver monitoring system of claim 18 , wherein the event camera has a quantum efficiency (QE) of at least 30% for near-infrared (near-IR) light having a wavelength of 940 nm.
22. A vehicular driver monitoring system, the vehicular driver monitoring system comprising: an event camera disposed in an interior rearview mirror assembly attached at an in-cabin side of a windshield of a vehicle equipped with the vehicular driver monitoring system, the event camera viewing the head of a driver of the equipped vehicle; the event camera comprising a photosensing array having photosensor pixels arranged in rows and columns; wherein the dynamic range of the event camera is at least 100 dB; wherein the response time of the event camera is less than 10,000 microseconds; wherein the event camera has a quantum efficiency (QE) of at least 15% for near-infrared (near-IR) light having a wavelength of 940 nm; wherein the interior rearview mirror assembly includes a mirror head that is adjustably disposed at a mounting base that is configured to attach the interior rearview mirror assembly at the in-cabin side of the windshield of the equipped vehicle, the mirror head having a mirror reflective element; wherein the event camera is disposed within the mirror head; wherein the interior rearview mirror assembly includes a plurality of near-IR illumination sources operable to emit near-infrared (near-IR) light to illuminate the head of the driver of the equipped vehicle; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein data output by the event camera is provided to the ECU; wherein the electronic circuitry of the ECU comprises a processor for processing data output by the event camera and provided to the ECU; wherein the vehicular driver monitoring system, responsive to processing at the ECU of data output by the event camera and provided to the ECU, monitors the driver of the equipped vehicle; wherein vehicle kinematic data is provided to the ECU as the equipped vehicle is driven along a road; wherein the vehicle kinematic data provided to the ECU comprises at least one data selected from the group consisting of (i) speed data, (ii) steering angle data, (iii) yaw data, (iv) pitch data and (v) roll data; and wherein, responsive at least in part to monitoring of the driver of the equipped vehicle and responsive at least in part to the vehicle kinematic data provided to the ECU, at least one is selected from the group consisting of (i) a driver attentiveness alert is generated, (ii) a driver drowsiness alert is generated, (iii) steering of the equipped vehicle is controlled and (iv) braking of the equipped vehicle is controlled.
23. The vehicular driver monitoring system of claim 22 , wherein the mirror reflective element comprises a mirror transflector, and wherein the mirror transflector transmits near-infrared (near-IR) light incident thereon, transmits visible light incident thereon and reflects visible light incident thereon.
24. The vehicular driver monitoring system of claim 23 , wherein the event camera views through the mirror transflector of the mirror reflective element.
25. The vehicular driver monitoring system of claim 23 , wherein the illumination sources of the plurality of near-IR illumination sources, when operated, emit near-infrared (near-IR) light that passes through the mirror transflector of the mirror reflective element.
26. The vehicular driver monitoring system of claim 23 , wherein the event camera is covertly disposed behind the mirror transflector of the mirror reflective element of the mirror head, and wherein the illumination sources of the plurality of near-IR illumination sources, when not operated, are covertly disposed behind the mirror transflector of the mirror reflective element.
27. The vehicular driver monitoring system of claim 26 , wherein the event camera has a quantum efficiency (QE) of at least 20% for near-infrared (near-IR) light having a wavelength of 940 nm.
28. The vehicular driver monitoring system of claim 26 , wherein the event camera has a quantum efficiency (QE) of at least 30% for near-infrared (near-IR) light having a wavelength of 940 nm.
29. A vehicular driver monitoring system, the vehicular driver monitoring system comprising: an event camera disposed in an interior electrochromic rearview mirror assembly attached at an in-cabin side of a windshield of a vehicle equipped with the vehicular driver monitoring system, the event camera viewing the head of a driver of the equipped vehicle; the event camera comprising a photosensing array having photosensor pixels arranged in rows and columns; wherein the dynamic range of the event camera is at least 100 dB; wherein the response time of the event camera is less than 10,000 microseconds; wherein the event camera has a quantum efficiency (QE) of at least 15% for near-infrared (near-IR) light having a wavelength of 940 nm; wherein the interior electrochromic rearview mirror assembly includes a mirror head that is adjustably disposed at a mounting base that is configured to attach the interior electrochromic rearview mirror assembly at the in-cabin side of the windshield of the equipped vehicle, the mirror head having an electrochromic mirror reflective element; wherein the electrochromic mirror reflective element comprises a front glass substrate and a rear glass substrate with an electrochromic medium sandwiched therebetween; wherein the front glass substrate comprises a planar first side and a planar second side separated by a thickness of the front glass substrate, and wherein a transparent electrically conductive coating is disposed at the planar second side of the front glass substrate; wherein the rear glass substrate comprises a planar third side and a planar fourth side separated by a thickness of the rear glass substrate, and wherein a mirror transflector is disposed at the planar third side of the rear glass substrate; wherein an electrochromic medium is disposed in an interpane cavity between the planar second side of the front glass substrate and the planar third side of the rear glass substrate; wherein the interpane cavity is established by a seal that spaces the planar second side of the front glass substrate from the planar third side of the rear glass substrate; wherein the electrochromic medium contacts the transparent electrically conductive coating disposed at the planar second side of the front glass substrate and contacts the mirror transflector disposed at the planar third side of the rear glass substrate; wherein the mirror transflector transmits near-infrared (near-IR) light incident thereon, transmits visible light incident thereon and reflects visible light incident thereon; wherein the event camera is covertly disposed behind the mirror transflector of the electrochromic mirror reflective element of the mirror head; wherein the event camera views through the mirror transflector of the electrochromic mirror reflective element; wherein the event camera is disposed within the mirror head; wherein the interior electrochromic rearview mirror assembly includes a plurality of near-IR light emitting diodes (LEDs) operable to emit near-infrared (near-IR) light to illuminate the head of the driver of the equipped vehicle; wherein the LEDs of the plurality of near-IR LEDs, when not operated, are covertly disposed behind the mirror transflector of the electrochromic mirror reflective element; wherein the LEDs of the plurality of near-IR LEDs, when operated, emit near-infrared (near-IR) light that passes through the mirror transflector of the electrochromic mirror reflective element; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein data output by the event camera is provided to the ECU; wherein the ECU is disposed with the mirror head of the interior electrochromic rearview mirror assembly; wherein the electronic circuitry of the ECU comprises a processor for processing data output by the event camera and provided to the ECU; wherein the vehicular driver monitoring system, responsive to processing at the ECU of data output by the event camera and provided to the ECU, monitors the driver of the equipped vehicle; wherein vehicle kinematic data is provided to the ECU as the equipped vehicle is driven along a road; wherein the vehicle kinematic data provided to the ECU comprises at least one data selected from the group consisting of (i) speed data, (ii) steering angle data, (iii) yaw data, (iv) pitch data and (v) roll data; and wherein, responsive at least in part to monitoring of the driver of the equipped vehicle and responsive at least in part to the vehicle kinematic data provided to the ECU, at least one is selected from the group consisting of (i) a driver attentiveness alert is generated, (ii) a driver drowsiness alert is generated, (iii) steering of the equipped vehicle is controlled and (iv) braking of the equipped vehicle is controlled.
30. The vehicular driver monitoring system of claim 29 , wherein a metallic perimeter band circumscribes a perimeter border region of the planar second side of the front glass substrate of the electrochromic mirror reflective element, the metallic perimeter band masking the seal, that spaces the planar second side of the front glass substrate from the planar third side of the rear glass substrate, from view by the driver of the equipped vehicle viewing the electrochromic mirror reflective element when seated within an interior cabin of the equipped vehicle at a driver-side front seating area of the equipped vehicle.
31. The vehicular driver monitoring system of claim 30 , wherein no part of the rear glass substrate extends beyond any part of the front glass substrate.
32. The vehicular driver monitoring system of claim 31 , wherein the mirror head of the interior electrochromic rearview mirror assembly comprises a mirror casing, and wherein a circumferential perimeter glass edge circumscribes a periphery of the front glass substrate of the electrochromic mirror reflective element, the circumferential perimeter glass edge spanning a thickness dimension of the front glass substrate separating the planar first side from the planar second side, and wherein no portion of the mirror casing overlaps the planar first side of the front glass substrate of the electrochromic mirror reflective element.
33. The vehicular driver monitoring system of claim 32 , wherein a portion of the mirror casing circumscribes the circumferential perimeter glass edge circumscribing the periphery of the front glass substrate of the electrochromic mirror reflective element.
34. The vehicular driver monitoring system of claim 33 , wherein the electrochromic mirror reflective element nests in the mirror casing.
35. The vehicular driver monitoring system of claim 29 , wherein the event camera has a quantum efficiency (QE) of at least 20% for near-infrared (near-IR) light having a wavelength of 940 nm.
36. The vehicular driver monitoring system of claim 29 , wherein the event camera has a quantum efficiency (QE) of at least 30% for near-infrared (near-IR) light having a wavelength of 940 nm.