IP Library › Granted Patent US 11,877,088
Granted Patent B2
US 11,877,088 · App. 17/125,889 · Granted Jan 16, 2024

Contrast enhanced images composited with artificial colorization

Inventors: Raymond Kirk Price (Redmond, WA); Michael Bleyer (Seattle, WA); Christopher Douglas Edmonds (Carnation, WA); Benjamin Brown (Seattle, WA); Casey Lee Miller (Fort Collins, CO)
Assignee: Microsoft Technology Licensing, LLC
H04N5/57G06T5/008G06T5/40H04N5/33H04N9/646
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,877,088
App. No.
17/125,889
Granted
Jan 16, 2024
Kind
B2
Abstract

A method for enhancing digital imagery. The method comprises receiving a linear, intensity-based image of an environment. A histogram of intensity values is generated for a plurality of pixels within the linear, intensity-based image. Based on the histogram of intensity values, local contrast enhancement is applied to the linear, intensity-based image to generate a contrast enhanced version of the linear, intensity-based image, and artificial colorization is applied to the linear, intensity-based image to generate an artificially colorized version of the linear, intensity-based image. A composite image of the environment is then generated based on at least a portion of the contrast enhanced version of the linear, intensity-based image and at least a portion of the artificially colorized version of the linear, intensity-based image.

Claims (49)

1. A method for enhancing digital imagery, comprising:

receiving a linear, intensity-based image of an environment;

generating one or more histograms of intensity values for a set of pixels within the linear, intensity-based image;

based on the one or more histograms of intensity values, applying local contrast enhancement to the linear, intensity-based image to generate a contrast enhanced version of the linear, intensity-based image;

in parallel to applying the local contrast enhancement to the linear, intensity-based image, based on the one or more histograms of intensity values, applying artificial colorization to the linear, intensity-based image to generate an artificially colorized version of the linear, intensity-based image; and

generating a composite image of the environment based on at least a portion of the contrast enhanced version of the linear, intensity-based image and at least a portion of the artificially colorized version of the linear, intensity-based image.

2. The method of claim 1 , wherein the linear, intensity-based image is a thermal image, and wherein the intensity values indicate an amount of thermal radiation for each of the set of pixels.

3. The method of claim 1 , wherein generating one or more histograms of intensity values for a set of pixels within the linear, intensity-based image comprises:

dividing the linear, intensity-based image into a plurality of tiles; and

generating one or more histograms of intensity values for each set of pixels in each of the plurality of tiles.

4. The method of claim 3 , wherein local contrast enhancement and artificial colorization are applied on a tile-by-tile basis based on the one or more histograms of intensity values associated with each tile.

5. The method of claim 1 , wherein applying artificial colorization to the linear, intensity-based image based on each histogram of intensity values includes applying a decision tree algorithm to each histogram of intensity values to determine a secondary peak, and applying artificial colorization to pixels having an intensity greater than a shoulder of the secondary peak.

6. The method of claim 1 , wherein applying artificial colorization to the linear, intensity-based image based on each histogram of intensity values includes applying artificial colorization to pixels having an intensity above a threshold.

7. The method of claim 1 , further comprising:

receiving a low-level light image of the environment; and

generating the composite image of the environment based on at least a portion of the contrast enhanced version of the linear, intensity-based image, at least a portion of the artificially colorized version of the linear, intensity-based image, and at least a portion of the low-level light image of the environment.

8. The method of claim 1 , further comprising:

presenting the composite image of the environment on a see-through display.

9. A system for a head-mounted display, comprising:

a see-through display;

one or more thermal cameras;

one or more low-level light cameras; and

a storage machine holding instructions executable by a logic machine to:

receive a linear, thermal image of an environment from the one or more thermal cameras;

receive a low-level light image of the environment from the one or more low-level light cameras;

generate one or more histograms of intensity values for a set of pixels within the linear, thermal image;

based on the one or more histograms of intensity values, apply local contrast enhancement to the linear, thermal image to generate a contrast enhanced version of the linear, thermal image;

in parallel to applying the local contrast enhancement to the linear, intensity-based image, based on the one or more histograms of intensity values, apply artificial colorization to the linear, thermal image to generate an artificially colorized version of the linear, thermal image; and

generate a composite image of the environment based on at least a portion of the contrast enhanced version of the linear, thermal image, at least a portion of the artificially colorized version of the linear, thermal image, and at least a portion of the low-level light image of the environment.

10. The system of claim 9 , wherein applying artificial colorization to the linear, thermal image based on each histograms of intensity values includes applying a decision tree algorithm to each histogram of intensity values to determine a secondary peak, and applying artificial colorization to pixels having an intensity greater than a shoulder of the secondary peak.

11. The system of claim 9 , wherein applying artificial colorization to the linear, thermal image based on each histogram of intensity values includes applying artificial colorization to pixels having an intensity above a threshold.

12. The system of claim 9 , wherein the storage machine further holds instructions executable by the logic machine to:

present the composite image of the environment on the see-through display.

13. The system of claim 12 , wherein presenting the composite image of the environment on the see-through display further comprises overlaying the composite image of the environment in real-time for a user viewing the environment via the see-through display.

14. The system of claim 13 , wherein overlaying the composite image of the environment in real-time includes using the artificially colorized version of the linear, thermal image as a mask to remove non-colorized imagery from the see-through display.

15. A method for enhancing digital imagery, comprising:

receiving a linear, thermal image of an environment from one or more thermal cameras;

receiving a low-level light image of the environment from one or more low-level light cameras;

dividing the linear, thermal image into a plurality of tiles;

generating one or more histograms of intensity values for each set of pixels in each of the plurality of tiles;

based on the one or more histograms of intensity values, applying local contrast enhancement to the linear, thermal image to generate a contrast enhanced version of the linear, thermal image;

in parallel to applying the local contrast enhancement to the linear, intensity-based image, based on the one or more histograms of intensity values, applying artificial colorization to the linear, thermal image to generate an artificially colorized version of the linear, thermal image;

generating a composite image of the environment based on at least a portion of the contrast enhanced version of the linear, thermal image, at least a portion of the artificially colorized version of the linear, thermal image, and at least a portion of the low-level light image of the environment; and

overlaying the composite image of the environment in real-time for a user viewing the environment via a see-through display.

16. The method of claim 15 , wherein applying artificial colorization to the linear, thermal image based on each histogram of intensity values includes applying a decision tree algorithm to each histogram of intensity values to determine a secondary peak, and applying artificial colorization to pixels having an intensity greater than a shoulder of the secondary peak.

17. The method of claim 15 , wherein applying artificial colorization to the linear, thermal image based on each histogram of intensity values includes applying artificial colorization to pixels having an intensity above a threshold.

18. The method of claim 15 , wherein overlaying the composite image of the environment in real-time includes using the artificially colorized version of the linear, thermal image as a mask to remove non-colorized imagery from the see-through display.

19. The method of claim 15 , wherein the see-through display is included in a head-mounted display system.

20. The method of claim 15 , wherein the see-through display is included in an automotive windshield.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 17, 2020
From: PRICE, RAYMOND KIRK; BLEYER, MICHAEL; EDMONDS, CHRISTOPHER DOUGLAS; BROWN, BENJAMIN; MILLER, CASEY LEE
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 054687/0287 →
Continuity (1)
Related Publication 20220201239A1 · Jun 23, 2022