IP Library Granted Patent US 10,152,781
Granted Patent B2
US 10,152,781 · App. 15/461,284 · Granted Dec 11, 2018

Method for image processing using local statistics convolution

Inventor: Nils Kokemohr (Hamburg, DE)
Assignee: Google LLC
G06T5/40G06K9/4661G06K9/6201G06T5/001G06T5/002G06T5/008G06T5/009G06T5/20G06T7/11G06T7/60G06K2009/4666G06T2207/20016G06T2207/20021G06T2207/20024G06T2207/20028G06T2207/20112G06T2207/20172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,152,781
App. No.
15/461,284
Granted
Dec 11, 2018
Kind
B2
Abstract

A method for filtering a digital image, comprising segmenting the digital image into a plurality of tiles; computing tile histograms corresponding to each of the plurality of tiles; deriving a plurality of tile transfer functions from the tile histograms preferably using 1D convolutions; interpolating a tile transfer function from the plurality of tile transfer functions; and filtering the digital image with the interpolated tile transfer function. Many filters otherwise difficult to conceive or to implement are possible with this method, including an edge-preserving smoothing filter, HDR tone mapping, edge invariant gradient or entropy detection, image upsampling, and mapping coarse data to fine data.

Claims (42)

1. A method to filter a digital image comprising a plurality of pixels, the method comprising:

segmenting the plurality of pixels into a first plurality of regions;

computing a first plurality of regional histograms by calculating a regional histogram corresponding to each region in the first plurality of regions, wherein the regional histogram is a three-dimensional histogram;

computing a first plurality of regional transfer functions by calculating for each region in the first plurality of regions a regional transfer function comprising a convolution of the corresponding regional histogram with a first convolution kernel;

deriving an image transfer function by interpolation of two or more of the first plurality of regional transfer functions;

filtering the digital image by applying the image transfer function to each of the plurality of pixels to generate a filtered digital image; and

causing the filtered digital image to be displayed.

2. The method of claim 1 , wherein the plurality of pixels comprises all pixels of the digital image.

3. The method of claim 2 , wherein each pixel of the plurality of pixels is contained within a single region of the first plurality of regions.

4. The method of claim 1 , wherein the three-dimensional histogram includes a mapping of pixel counts again a luminance and two chrominances.

5. The method of claim 1 , further comprising performing a bilinear interpolation on results retrieved from the three-dimensional histogram.

6. The method of claim 1 , wherein the filtering includes performing a filtering operation specific to one color.

7. The method of claim 1 , further comprising determining an entropy of a given object in the digital image, wherein the entropy is determined independent of similar luminosity objects in the digital image having a different color than the given object.

8. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform operations to transform a digital image comprising a plurality of pixels, said operations comprising:

segmenting the plurality of pixels into a plurality of regions;

computing a plurality of regional histograms by calculating a regional histogram corresponding to each region in the plurality of regions;

computing a first plurality of regional transfer functions by calculating for each region in the plurality of regions a regional transfer function comprising a convolution of the corresponding regional histogram with a first convolution kernel;

deriving an image transfer function by interpolation of two or more of the first plurality of regional transfer functions; and

filtering the digital image by applying the image transfer function to each of the plurality of pixels.

9. The program storage device of claim 8 , wherein deriving the image transfer function by the interpolation of the two or more of the first plurality of regional transfer functions comprises:

selecting a pixel having a luminosity value from a first region of the plurality of regions;

computing a plurality of single value results by inputting the luminosity value of the selected pixel into each of a second plurality of regional transfer functions, said second plurality of regional transfer functions including regional transfer functions corresponding to regions other than the first region; and

interpolating a resulting value for the selected pixel based on two of the plurality of single value results.

10. The program storage device of claim 8 , wherein the first convolution kernel is a Gaussian function.

11. The program storage device of claim 8 , where the regional transfer function further comprises a second convolution comprising the convolution of the corresponding regional histogram with a second convolution kernel.

12. The program storage device of claim 11 , where the first convolution kernel is a Gaussian function, and the second convolution kernel is a product of a Gaussian function and a step function.

13. The program storage device of claim 11 , where the second convolution is weighted by pixel luminosity.

14. The program storage device of claim 8 , where the plurality of regional histograms are a function of one or more of luminosity, pixel difference, color, depth along a z-axis, and saturation.

15. A method to filter a digital image, the method comprising:

segmenting the digital image into a plurality of tiles;

computing tile histograms corresponding to each of the plurality of tiles, wherein each tile histogram includes a three-dimensional histogram;

deriving a plurality of tile transfer functions from the tile histograms;

interpolating a tile transfer function from the plurality of tile transfer functions; and

filtering the digital image with the interpolated tile transfer function.

16. The method of claim 15 , where the tiles are non-overlapping, and each pixel of the digital image is contained within a single tile.

17. The method of claim 15 , wherein the three-dimensional histogram includes a mapping of pixel counts again a luminance and two chrominances.

18. The method of claim 15 , wherein the filtering includes performing a filtering operation specific to one color.

19. The method of claim 15 , further comprising determining an entropy of a given object within the digital image without being affected by a presence of one or more similar luminosity objects within the digital image having a different color than the given object.

20. The method of claim 15 , wherein the interpolating comprises:

selecting a pixel having a luminosity value from a first tile of the plurality of tiles;

computing a plurality of single value results by inputting the luminosity value of the selected pixel into each of a second plurality of regional transfer functions, said second plurality of regional transfer functions including regional transfer functions corresponding to tiles not comprising the selected pixel; and

interpolating a resulting value for the selected pixel as a function of the plurality of single value results.

Assignments (3)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2017
From: KOKEMOHR, NILS
To: NIK SOFTWARE, INC.
Reel/Frame 042680/0215 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2017
From: NIK SOFTWARE, INC.
To: GOOGLE INC.
Reel/Frame 042680/0232 →
Continuity (5)
Continuation 15076404 · Mar 21, 2016
Division 14094405 · Dec 2, 2013
Continuation 12566782 · Sep 25, 2009
Provisional Application 61100604 · Sep 26, 2008
Related Publication 20170186142A1 · Jun 29, 2017