IP Library › Granted Patent US 11,431,918
Granted Patent B1
US 11,431,918 · App. 17/181,922 · Granted Aug 30, 2022

High dynamic range image capture using single photon avalanche diodes

Inventors: Raymond Kirk Price (Redmond, WA); Christopher Douglas Edmonds (Carnation, WA); Michael Bleyer (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
H04N5/2355G06T5/003G06T5/50H04N5/2353G06T2207/10144G06T2207/20201G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,431,918
App. No.
17/181,922
Granted
Aug 30, 2022
Kind
B1
Abstract

A system for HDR image capture is configurable to perform a split long exposure operation by applying a first set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection and applying a second set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection. A time period intervenes between the first and second sets of long exposure shutter. The system is configurable to perform a short exposure operation by applying a set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection. The short exposure operation occurs during the time period that intervenes between the first and second sets of long exposure shutter operations. The system is also configurable to generate an image based on the split long exposure operation and the short exposure operation.

Claims (58)

1. A system for facilitating high dynamic range (HDR) image capture, comprising:

a single photon avalanche diode (SPAD) array comprising a plurality of SPAD pixels;

one or more processors; and

one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to:

perform a split long exposure operation, comprising:

applying a first set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection; and

applying a second set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, wherein a time period intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations;

perform a short exposure operation, comprising:

applying a set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, the set of short exposure shutter operations being applied during the time period that intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations; and

generate an image based at least on a number of photons detected during the split long exposure operation and a number of photons detected during the short exposure operation.

2. The system of claim 1 , wherein long exposure shutter operations of the first and second sets of long exposure shutter operations have a longer gate time period than short exposure shutter operations of the set of short exposure shutter operations.

3. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to:

perform a subsequent short exposure operation subsequent to the applying of the second set of long exposure shutter operations, the subsequent short exposure operation comprising:

applying a subsequent set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection; and

perform a subsequent long exposure operation subsequent to the performing of the subsequent short exposure operation, comprising:

applying a subsequent set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection.

4. The system of claim 3 , wherein the instructions are executable by the one or more processors to further configure the system to generate a subsequent image based on (i) a number of photons detected during the applying of the second set of long exposure shutter operations of the split long exposure operation, (ii) a number of photons detected during the subsequent short exposure operation, and (iii) a number of photons detected during the subsequent long exposure operation.

5. The system of claim 4 , wherein short exposure shutter operations of the subsequent set of short exposure shutter operations of have a same gate time period as short exposure shutter operations of the set of short exposure shutter operations.

6. The system of claim 5 , wherein long exposure shutter operations of the subsequent set of long exposure shutter operations of have a same gate time period as long exposure shutter operations of the first and second sets of long exposure shutter operations.

7. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to apply a motion compensation operation to generate the image.

8. The system of claim 7 , wherein the motion compensation operation comprises:

associating the first set of long exposure shutter operations with first motion data captured during the applying of the first set of long exposure shutter operations;

associating the set of short exposure shutter operations with second motion data captured during the applying of the set of short exposure shutter operations;

associating the second set of long exposure shutter operations with third motion data captured during the applying of the second set of long exposure shutter operations; and

using the first, second, and third motion data to reduce motion blur in the image.

9. The system of claim 7 , wherein the motion compensation operation comprises a computer vision-based motion compensation operation.

10. The system of claim 1 , wherein an effective exposure time of the second set of long exposure shutter operations is determined based on a readout from the first set of long exposure shutter operations.

11. A method for facilitating high dynamic range (HDR) image capture, comprising:

performing a split long exposure operation, comprising:

applying a first set of long exposure shutter operations to configure each single photon avalanche diode (SPAD) pixel of SPAD array to enable photon detection; and

applying a second set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, wherein a time period intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations;

performing a short exposure operation, comprising:

applying a set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, the set of short exposure shutter operations being applied during the time period that intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations; and

generating an image based at least on a number of photons detected during the split long exposure operation and a number of photons detected during the short exposure operation.

12. The method of claim 11 , wherein long exposure shutter operations of the first and second sets of long exposure shutter operations have a longer gate time period than short exposure shutter operations of the set of short exposure shutter operations.

13. The method of claim 11 , further comprising:

performing a subsequent short exposure operation subsequent to the applying of the second set of long exposure shutter operations, the subsequent short exposure operation comprising:

applying a subsequent set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection; and

performing a subsequent long exposure operation subsequent to the performing of the subsequent short exposure operation, comprising:

applying a subsequent set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection.

14. The method of claim 13 , further comprising:

generating a subsequent image based on (i) a number of photons detected during the applying of the second set of long exposure shutter operations of the split long exposure operation, (ii) a number of photons detected during the subsequent short exposure operation, and (iii) a number of photons detected during the subsequent long exposure operation.

15. The method of claim 14 , wherein short exposure shutter operations of the subsequent set of short exposure shutter operations of have a same gate time period as short exposure shutter operations of the set of short exposure shutter operations.

16. The method of claim 15 , wherein long exposure shutter operations of the subsequent set of long exposure shutter operations of have a same gate time period as long exposure shutter operations of the first and second sets of long exposure shutter operations.

17. The method of claim 11 , further comprising applying a motion compensation operation to generate the image.

18. The method of claim 17 , wherein the motion compensation operation comprises:

associating the first set of long exposure shutter operations with first motion data captured during the applying of the first set of long exposure shutter operations;

associating the set of short exposure shutter operations with second motion data captured during the applying of the set of short exposure shutter operations;

associating the second set of long exposure shutter operations with third motion data captured during the applying of the second set of long exposure shutter operations; and

using the first, second, and third motion data to reduce motion blur in the image.

19. One or more hardware storage devices storing instructions that are executable by one or more processors of a system to configure the system to:

perform a split long exposure operation, comprising:

applying a first set of long exposure shutter operations to configure each single photon avalanche (SPAD) pixel of a SPAD array to enable photon detection; and

applying a second set of long exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, wherein a time period intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations;

perform a short exposure operation, comprising:

applying a set of short exposure shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection, the set of short exposure shutter operations being applied during the time period that intervenes between the applying of the first set of long exposure shutter operations and the applying of the second set of long exposure shutter operations; and

generate an image based at least on a number of photons detected during the split long exposure operation and a number of photons detected during the short exposure operation.

20. The one or more hardware storage devices of claim 19 , wherein long exposure shutter operations of the first and second sets of long exposure shutter operations have a longer gate time period than short exposure shutter operations of the set of short exposure shutter operations.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2021
From: PRICE, RAYMOND KIRK; EDMONDS, CHRISTOPHER DOUGLAS; BLEYER, MICHAEL
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 055361/0556 →
Cited By (2)
US 12,192,661 US 12,586,159