IP Library › Granted Patent US 12,634,587
Granted Patent B2
US 12,634,587 · App. 18/682,422 · Granted May 19, 2026

Event-based auto-exposure for digital photography

Inventors: Jia Pan (Hong Kong, CN); Shijie Lin (Hong Kong, CN)
Assignee: The University of Hong Kong
H04N23/73H04N23/71H04N23/72H04N23/74H04N25/47
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,634,587
App. No.
18/682,422
Granted
May 19, 2026
Kind
B2
Abstract

An event-guided auto-exposure (EG-AE) device leverages the high dynamic range and low latency properties of a dynamic vision sensor to guide the exposure setting of an active pixel sensor. The method utilizes physical connections of images and events to estimate the irradiance variations, which are fed into the EG-AE device for calculating the exposure setting. The method shows outstanding performance in tackling highly challenging lighting conditions and benefits multiple downstream applications, including visual tag detection and feature matching.

Claims (16)

1 . An event-guided auto-exposure control method for a digital camera, comprising the steps of:

scanning a scene to create a scene radiance signal;

linearly mapping the scene radiance signal to a sensor irradiance signal;

generating events and images for irradiance estimation in response to the sensor irradiance signal by means of an active pixel sensor (APS) that accumulates the sensor irradiance signal during the exposure time and transform it to digital images in a non-linear manner, and a dynamic vision sensor (DVS) that works asynchronously with the APS to encode the relative change of the log of the sensor irradiance signal and generate a stream of timestamped address events; and

utilizing an event-based irradiance estimation (I-IE) to compute a desired exposure time.

2 . The event-guided auto-exposure control method of claim 1 wherein the steps of scanning and linearly mapping the scene field involve scanning a light source over the scene so that reflectance of the light from objects in the scene produce a scene radiance signal that is transformed by a lens into a sensor irradiance signal.

3 . The event-guided auto-exposure control method of claim 1 wherein events and images are generated by APS and DVS in an events camera.

4 . The event-guided auto-exposure control method of claim 3 wherein the events camera is a DAVIS 346 acoustic emission (AE) event camera.

5 . The event-guided auto-exposure control method of claim 1 wherein the timestamped address events and each event is a 4-dimensional tuple.

6 . The event-guided auto-exposure control method of claim 1 wherein the event-based irradiance estimation (I-IE) is determined by unifying both events and images by the steps of:

formulating an estimated irradiance in the form of a reference irradiance times the average of relative irradiance, where the average of relative irradiance is achieved by nonuniformly sampling the relative irradiance of pixels at all event timestamps;

using the estimated irradiance to compute the desired exposure time for image capturing; and

reconstructing the intensity images using a camera response function (CRF) to form the estimated irradiance, image deblurring and high-rate frame reconstruction.

7 . The event-guided auto-exposure control method of claim 6 wherein the reference irradiance is a constant.

8 . The event-guided auto-exposure control method of claim 1 , wherein a high dynamic range (HDR) sensing capability of DVS is leveraged to guide exposure setting for APS irradiance accumulation.

9 . The event-guided auto-exposure control method of claim 6 wherein the latest unsaturated pixels are used to recover the reference irradiance, and the recovered reference irradiance is combined with the irradiance estimated from previous unsaturated pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2024
From: PAN, JIA; LIN, SHIJIE
To: THE UNIVERSITY OF HONG KONG
Reel/Frame 066422/0370 →
Continuity (2)
Provisional Application 63236375 · Aug 24, 2021
Related Publication 20240348934A1 · Oct 17, 2024
References Cited (55)
US 4423936A · Johnson · 1984 [cited by applicant]
US 5592256A · Muramatsu · 1997 [cited by applicant]
US 11768919B2 · Ryan · 2023 [cited by examiner]
US 20080226192A1 · Silverstein · 2008 [cited by examiner]
US 20180176442A1 · Okada · 2018 [cited by examiner]
US 20200226377A1 · Macias et al. · 2020 [cited by applicant]
US 20200342607A1 · Hoshino · 2020 [cited by examiner]
US 20230232112A1 · Nishio · 2023 [cited by examiner]
US 20230297158A1 · Zha · 2023 [cited by examiner]
US 20240259702A1 · Moeys · 2024 [cited by examiner]
CN 112801027A · 2021 [cited by applicant]
Davis 346 event camera, Aug. 15, 2019 (Year: 2019). [cited by examiner]
Attridge, et al. “The Manual of Photography.” Focal press, 1988, 46 pages. [cited by applicant]
Bay, et al. “SURF: Speeded-up Robust Features,” Computer Vision and Image Understanding, 2008, vol. 110, Issue 3, 14 pages. [cited by applicant]
Brandli, et al. Real-Time, High-Speed Video Decompression Using a Frame- and Event-Based DAVIS Sensor, IEEE International Symposium on Circuits and Systems (ISCAS), 2014, pp. 686-689. [cited by applicant]
Brandli, et al. “A 240 180 130 dB 3 is latency global shutter spatiotemporal vision sensor,” IEEE Journal of Solid-State Circuits, 2014, vol. 49, No. 10, pp. 2333-2341. [cited by applicant]
Chen, et al. “Fast retinomorphic event-driven representations for video gameplay and action recognition,” IEEE Transactions on Computational Imaging, 2019, 6: 14 pages. [cited by applicant]
Debevec, et al. “Recovering high dynamic range radiance maps from photographs.” Proceeding SIGGRAPH'08 ACM SIGGRAPH, 2008 classes Article 31 (2008), 10 pages. [cited by applicant]
Delbruck, et al. “32-bit Configurable Bias Current Generator with sub-off-current capability.” Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE, 2010, pp. 1647-1650. [cited by applicant]
Gehrig, et al. “EKLT: Asynchronous photometric feature tracking using events and frames.” International Journal of Computer Vision 128.3, 2020, pp. 601-618. [cited by applicant]
Gehrig, et al. “End-to-end learning of representations for asynchronous event-based data.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. pages. [cited by applicant]
GoPro hero, Official Camera of Fun, May 17, 2024, 7 pages, https://gopro.com/en/us. 7, 8. [cited by applicant]
Grossberg, et al. “Modeling the space of camera response functions.” IEEE transactions on pattern analysis and machine intelligence Oct. 26, 2004, pp. 1272-1282. [cited by applicant]
Harris, et al. “A combined corner and edge detector.” In Proc. 4th Alvey Vision Conference, Manchester, U.K., Aug. 1988, pp. 147-151. [cited by applicant]
Hu et al. “v2e: From video frames to realistic DVS events.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, 10 pages. [cited by applicant]
Huawei mate 30. May 1, 2024, 8 pages https://consumer.huawei.com/ en/phones/mate30/. 7, 8. [cited by applicant]
Kim, et al, “Younggun Cho, and Ayoung Kim. Exposure control using bayesian optimization based on entropy weighted image gradient.” In 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 857-8… [cited by applicant]
Kremens, et al. “System implications of implementing auto-exposure on consumer digital cameras.” Sensors, Cameras, and Applications for Digital Photography, SPIE, 1999, 9 pages. [cited by applicant]
Kuno, et al, “A new automatic exposure system for digital still cameras.” IEEE Transactions on Consumer Electronics 44.1, 1998, pp. 192-199. [cited by applicant]
Leutenegger, et al, “BRISK: Binary robust invariant scalable keypoints.” 2011 International conference on computer vision. IEEE, 2011, pp. 2548-2555. [cited by applicant]
Lin, et al, “Efficient spatial-temporal normalization of sae representation for event camera.” IEEE Robotics and Automation Letters 5.3, 2020, pp. 4265-4272. [cited by applicant]
Liu, et al, “No-reference image quality assessment based on spatial and spectral entropies.” Signal processing: Image communication 29.8, 2014, pp. 856-863. [cited by applicant]
Liu, et al. “Deep learning for generic object detection: A survey.” International journal of computer vision 128, 2019, 35 pages. [cited by applicant]
Lu, et al. “Camera parameters auto-adjusting technique for robust robot vision.” 2010 IEEE International Conference on Robotics and Automation, 2010, 6 pages. [cited by applicant]
Matas, et al. “Robust wide-baseline stereo from maximally stable extremal regions.” Image and vision computing 22.10 (2004): 761-767. [cited by applicant]
Meiguang, et al, “Learning to extract a video sequence from a single motion-blurred image.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6334-6342. [cited by applicant]
Mittal, et al. “No-reference image quality assessment in the spatial domain.” IEEE Transactions on image processing, Dec. 2012, vol. 21, No. 12, pp. 4695-4708. [cited by applicant]
Montalvo, et al. “Acquisition of agronomic images with sufficient quality by automatic exposure time control and histogram matching.” Advanced Concepts for Intelligent Vision Systems: 15th International Conference, ACIV… [cited by applicant]
Neves, et al. “Autonomous configuration of parameters in robotic digital cameras.” Pattern Recognition and Image Analysis: 4th Iberian Conference, 2009, IbPRIA 2009, LNCS 5524, pp. 80-87. [cited by applicant]
Nouourani-Vatani, et al. “Automatic camera exposure control.” Proceedings of the Australasian Conference on Robotics and Automation 2007. Australian Robotics and Automation Association (ARAA), 7 pages. [cited by applicant]
Nozaki, et al. “Temperature and parasitic photocurrent effects in dynamic vision sensors.” IEEE Transactions on Electron Devices, 2017, 7 pages. [cited by applicant]
Olson, “AprilTag: A robust and flexible visual fiducial system.” 2011 IEEE international conference on robotics and automation. IEEE, May 9-13, 2011, pp. 3400-3407. [cited by applicant]
“Pan, et al. “Bringing a blurry frame alive at high frame-rate with an event camera.”” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14 pages arXiv:1811.10180v2 [cs.CV] Nov. 27, 2018. [cited by applicant]
Pan, et al. “Deblurring images via dark channel prior.” IEEE transactions on pattern analysis and machine intelligence, 2017, vol. 40., No. 10, pp. 2315-2328. [cited by applicant]
Rosten, et al. “Fusing points and lines for high performance tracking.” Tenth IEEE International Conference on Computer Vision (ICCV'05), 2005, 8 pages. [cited by applicant]
Shim, et al. “Auto-adjusting camera exposure for outdoor robotics using gradient information.” IEEE, 2014, 7 pages arXiv:1708.07338v2 [cs.CV] Oct. 10, 2017. [cited by applicant]
Shim, et al. “Gradient-based camera exposure control for outdoor mobile platforms.” IEEE Transactions on Circuits and Systems for Video Technology, 2018, 14 pages arXiv:1708.07338v3 [cs.CV] Jun. 13, 2018. [cited by applicant]
Venkatanath, et al. “Blind image quality evaluation using perception based features.” IEEE, 2015, 6 pages. [cited by applicant]
Vidal, et al. “Ultimate SLAM? Combining events, images, and IMU for robust visual SLAM in HDR and high-speed scenarios.” IEEE Robotics and Automation Letters, Dec. 2017, 10 pages. [cited by applicant]
Wang, et al. “Event camera calibration of per-pixel biased contrast threshold.” 11 pages arXiv:2012.09378v1 [cs.CV] Dec. 17, 2020. [cited by applicant]
Wang, et al. “Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco… [cited by applicant]
Wang, et al. “Joint filtering of intensity images and neuromorphic events for high-resolution noise-robust imaging.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1609-1619. [cited by applicant]
“ZHANG, et al. ““Active exposure control for robust visual odometry in HDR environments.”” 2017 IEEE international conference on robotics and automation (ICRA).” IEEE, 2017, 9 pages. [cited by applicant]
International Search Report and Written Opinion of corresponding International Application No. PCT/CN2022/114456 mailed Nov. 22, 2022. 5 pages. [cited by applicant]
Delbruck “SensorsINI/jaer: Java tools for Address-Event Representation (AER) neuromorphic vision and audio sensor processing” Github, 2024, 7 pages. [cited by applicant]