IP Library Granted Patent US 11,107,204
Granted Patent B2
US 11,107,204 · App. 15/757,154 · Granted Aug 31, 2021

System and method for real-time tone-mapping

Inventors: Jonas Unger (Norrköping, SE); Gabriel Eilertsen (Norrköping, SE); Rafal Mantiuk (Cambridge, GB)
Assignee: Faurecia Irystec, Inc.
G06T5/009G06T2207/10024G06T2207/20076G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,107,204
App. No.
15/757,154
Granted
Aug 31, 2021
Kind
B2
Abstract

A method and system for tone-mapping an image includes determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image and tone-mapping the input image according to the determined tone-curve. Determining the tone curve includes analytically calculating values of the tone-curve that reduce the image contrast distortion within the model of image contrast distortion. A tone-mapping operator includes a noise model generator and a tone-mapping module operable to receive one or more contextual parameters. The tone-mapping module includes an edge-stopping filtering submodule for extracting a base layer and detail layer of the input image, a tone-curve generating submodule and a combining submodule for combining the base layer and detail layer. At least one of the edge-stopping filtering submodule, the tone-curve generating submodule and combining submodule is adjustable based on the contextual parameters.

Claims (76)

1. A method for tone-mapping an input image to generate a tone-mapped output image, the method comprising:

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image; and

tone-mapping the input image according to the determined tone-curve; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion;

wherein determining the tone-curve comprises:

defining a plurality of luminance level segments corresponding to portions of luminance levels of the input image; and

determining, for each given luminance level segment, a piece-wise linear slope representing a portion of the tone-curve for the given luminance level segment.

2. The method of claim 1 , wherein the input image is one of a standard dynamic range image and a high dynamic range image.

3. The method of claim 1 , wherein the values of the tone-curve are analytically calculated for minimizing the image contrast distortion within the model of image contrast distortion.

4. The method of claim 1 , wherein determining the piece-wise linear slope representing the portion of the tone-curve for each given luminance level segment is subject to the piece-wise linear slope being non-decreasing and to the output image tone-mapped according to the piece-wise linear slope being within the available dynamic range of a display device for displaying the tone-mapped output image.

5. The method of claim 1 , wherein the piece-wise linear slope representing a portion of the tone-curve for the luminance level segment is determined for reducing the sum over all luminance level segments (k=1 . . . N) of the product of at least two of:

i) a probability of any region of the input image having a luminance level falling within a given (k-th) luminance level segment;

ii) an image saliency of the (k-th) luminance level segment; and

iii) a function of the piece-wise linear slope for the given (k-th) luminance level segment.

6. The method of claim 5 , wherein the image saliency is determined based on image contrast for the given (k-th) luminance level segment of a plurality of regions of the input image.

7. The method of claim 6 , wherein the image saliency is a function of an amount of regions of the input image having an image contrast for the given (k-th) luminance level segment greater than a noise level of a noise model of the input image.

8. The method of claim 5 , wherein the linear slope for each luminance level segment is determined based on minimizing:

ε′( s k )=Σ k N p ( l k )(1− s k ) 2

wherein p(l k ) is the probability of any region of the input image having a luminance level falling with the given (k-th) luminance level segment, s k is the piece-wise linear slope of the given (k-th) luminance level and (1−s k ) 2 is the differential value of the piece-wise linear slope for the given (k-th) luminance level segment.

9. The method of claim 8 , wherein the minimizing comprises analytically solving ε′(s k )=Σ k N p(l k )(1−s k ) 2 to determine the value of the piece wise linear slope (s k ) for each (k-th) luminance level segment.

10. The method of claim 5 , wherein the probability of any region of the input image having a luminance level falling with a given (k-th) luminance level segment is adjusted based on the input image having regions of contrast greater than a noise level of the input image.

11. The method of claim 10 , wherein the probability of any region of the input image having a luminance level falling with a given (k-th) luminance level segment corresponds to the probability of any pixel of the input image falling with a given (k-th) luminance level segment being further weighted by a number of pixels within the input image having an image contrast value greater than the noise level of the input image.

12. The method of claim 1 , wherein the input image is subdivided into a plurality of local regions, and wherein a local tone-curve is determined for each of the local regions.

13. The method of claim 1 , further comprising:

decomposing the input image into a base layer and a detail layer, wherein the tone-curve is determined for the base layer and wherein the tone-mapping is applied to the base layer; and

combining the detail layer with the tone-mapped base layer.

14. The method of claim 1 , wherein the determination of the tone-curve is adjustable based on one or more contextual parameters.

15. The method of claim 14 , wherein the contextual parameters comprises one or more of a viewer characteristic, ambient light, peak luminance of an output display device, ambient reflectance, dynamic range of the output display device, user-defined parameters, speed and exposure.

16. The method of claim 14 , wherein the one or more contextual parameters comprises one or more of ambient light, peak luminance of an output display device, dynamic range of the output display device and exposure; and

wherein the determination of the tone-curve comprises determining an effective output dynamic range based on the one or more of the ambient light, the peak luminance and dynamic range and generates the tone-curve based on the effective output dynamic range.

17. A computer-implemented system for generating a tone-mapped output image from an input image, the system comprising:

at least one data storage device; and

at least one processor coupled to the at least one storage device, the at least one processor being configured for:

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image; and

tone-mapping the input image according to the determined tone-curve; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion;

wherein determining the tone-curve comprises:

defining a plurality of luminance level segments corresponding to portions of luminance levels of the input image; and

determining, for each given luminance level segment, a piece-wise linear slope representing a portion of the tone-curve for the given luminance level segment.

18. A method for tone-mapping an input image to generate a tone-mapped output image, the method comprising:

decomposing the input image into a base layer and a detail layer;

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image, wherein the tone-curve is determined for the base layer; and

tone-mapping the input image according to the determined tone-curve, wherein the tone-mapping is applied to the base layer; and

combining the detail layer with the tone-mapped base layer; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion.

19. The method of claim 18 , wherein the combining the detail layer with the tone-mapped base layer comprises modulating the detail layer based on at least one of a viewer characteristic and at least one viewer preference.

20. The method of claim 18 , wherein the combining the detail layer with the tone-mapped base layer comprises modulating the detail layer based on a viewer characteristic.

21. The method of claim 18 , wherein decomposing the input image comprises applying a spatial filter to the input image to generate the base layer and the detail layer, the filtering comprising for each of a plurality of pixels:

detecting the presence of an edge of the input image within a region surrounding the pixel; and

selectively applying a filtering kernel to the region according to the presence of the edge within the region.

22. The method of claim 18 , further comprising modulating the detail layer based on a visibility threshold of the tone-mapped base layer and a model of noise of the input image.

23. A method for tone-mapping an input image to generate a tone-mapped output image, the method comprising:

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image; and

tone-mapping the input image according to the determined tone-curve; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion;

wherein the determination of the tone-curve is adjustable based on one or more contextual parameters;

wherein the one or more contextual parameters comprises one or more of ambient light, peak luminance of an output display device, dynamic range of the output display device and exposure; and

wherein the determination of the tone-curve comprises determining an effective output dynamic range based on the one or more of the ambient light, the peak luminance and dynamic range and generates the tone-curve based on the effective output dynamic range.

24. A computer-implemented system for generating a tone-mapped output image from an input image, the system comprising:

at least one data storage device; and

at least one processor coupled to the at least one storage device, the at least one processor being configured for:

decomposing the input image into a base layer and a detail layer;

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image,

wherein the tone-curve is determined for the base layer; and

tone-mapping the input image according to the determined tone-curve, wherein the tone-mapping is applied to the base layer; and

combining the detail layer with the tone-mapped base layer; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion.

25. A computer-implemented system for generating a tone-mapped output image from an input image, the system comprising:

at least one data storage device; and

at least one processor coupled to the at least one storage device, the at least one processor being configured for:

determining a tone-curve based on a model of image contrast distortion between the input image and a tone-mapped image; and

tone-mapping the input image according to the determined tone-curve; and

wherein determining the tone-curve comprises analytically calculating values of the tone-curve for reducing image contrast distortion within the model of image contrast distortion;

wherein the determination of the tone-curve is adjustable based on one or more contextual parameters;

wherein the one or more contextual parameters comprises one or more of ambient light, peak luminance of an output display device, dynamic range of the output display device and exposure; and

wherein the determination of the tone-curve comprises determining an effective output dynamic range based on the one or more of the ambient light, the peak luminance and dynamic range and generates the tone-curve based on the effective output dynamic range.

Assignments (3)
CHANGE OF NAME Recorded Aug 21, 2020
From: IRYSTEC SOFTWARE INC.
To: FAURECIA IRYSTEC INC.
Reel/Frame 053564/0477 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: UNGER, JONAS; EILERTSEN, GABRIEL; MANTIUK, RAFAL
To: TANDEMLAUNCH INC.
Reel/Frame 047274/0096 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: TANDEMLAUNCH INC.
To: IRYSTEC SOFTWARE INC.
Reel/Frame 047274/0298 →
Continuity (2)
Provisional Application 62213290 · Sep 2, 2015
Related Publication 20200286213A1 · Sep 10, 2020
Cited By (5)
US 12,249,054 US 12,524,858 US 12,561,774 US 12,608,783 US 12,671,909