IP Library › Granted Patent US 11,074,704
Granted Patent B2
US 11,074,704 · App. 16/240,596 · Granted Jul 27, 2021

Error reduction of depth maps

Inventors: Michael Bleyer (Seattle, WA); Christopher D. Edmonds (Carnation, WA); Yuri Pekelny (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T7/521G06T5/40G06T7/0002G06T2207/10028G06T2207/30168
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,074,704
App. No.
16/240,596
Granted
Jul 27, 2021
Kind
B2
Abstract

The described implementations relate to reducing distance errors in depth maps. One example can detect blooming in a depth map. This example can identify a depth range affected by the blooming and take an action relating to the identified depth range to address the blooming.

Claims (30)

1. A method, comprising:

receiving a depth map of a scene and a corresponding active brightness image of the scene;

detecting blooming in the depth map at least in part from saturated pixels in the active brightness image;

identifying a depth range of the saturated pixels;

invalidating the depth range of the saturated pixels from the depth map; and,

generating another depth map that includes invalidated depths for the saturated pixels.

2. The method of claim 1 , wherein the receiving comprises receiving the depth map from a time-of-flight camera.

3. The method of claim 1 , wherein the invalidating the depth range of the saturated pixels comprises invalidating only saturated pixels.

4. The method of claim 1 , wherein the invalidating the depth range of the saturated pixels comprises invalidating saturated pixels and unsaturated pixels.

5. The method of claim 1 , wherein the detecting blooming in the depth map comprises making a binary determination whether blooming occurs in the depth map.

6. The method of claim 5 , wherein the making a binary determination is based at least in part upon a number of saturated pixels in the active brightness image.

7. The method of claim 6 , wherein the making a binary determination is based at least in part upon a number of saturated pixels in the active brightness image that are adjacent to one another.

8. The method of claim 1 , wherein the detecting comprises computing a set of all valid depth pixels that have a spatial neighbor that is saturated.

9. The method of claim 8 , wherein the identifying comprises generating a histogram of depth values for the set.

10. The method of claim 9 , further comprising smoothing the histogram.

11. The method of claim 10 , wherein the smoothing comprises replacing a value of individual bins of the histogram with an average value.

12. The method of claim 11 , wherein the replacing comprises calculating the average value as the value of an individual bin averaged with at least two adjacent neighboring bins.

13. A system, comprising:

a depth camera configured to capture a depth map and a corresponding active brightness image of a scene; and,

a processor configured to:

detect blooming in the depth map at least in part from saturated pixels in the active brightness image,

identify a depth range of the saturated pixels, invalidate the depth range of the saturated pixels from the depth map, and generate another depth map that includes invalidated depths for the depth range.

14. The system of claim 13 , wherein the depth camera and the processor are implemented on a single device.

15. The system of claim 13 , wherein the depth camera is manifest on a first device and the processor is implemented on a second device.

16. The system of claim 13 , wherein the processor is configured to identify the depth range by identifying saturated pixels and adjacent pixels of the depth map.

17. The system of claim 13 , wherein the processor is configured to perform the invalidating only on regions of the depth range that are aligned with saturated pixels in the active brightness image that corresponds to the depth map.

18. The system of claim 13 , wherein the depth camera and the processor are implemented on separate devices.

19. A device comprising:

a depth camera configured to generate a depth map and a corresponding active brightness image of a scene; and,

a bloom reduction computation component configured to detect blooming in the depth map based at least in part on saturated pixels of the active brightness image, identify a depth range of the saturated pixels, invalidate the depth range of the saturated pixels from the depth map, and generate another depth map that includes invalidated depths for the saturated pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2019
From: BLEYER, MICHAEL; EDMONDS, CHRISTOPHER D.; PEKELNY, YURI
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 048113/0125 →
Continuity (1)
Related Publication 20200219276A1 · Jul 9, 2020
Cited By (3)
US 12,561,768 US 12,586,163 US 12,731,228