IP Library › Granted Patent US 9,338,349
Granted Patent B2
US 9,338,349 · App. 14/012,534 · Granted May 10, 2016

Generation of ghost-free high dynamic range images

Inventors: Piyush Sharma (Stanford, CA); Nitesh Shroff (San Diego, CA); Ramin Rezaiifar (Del Mar, CA); Chiachi Sung (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N5/23229G06T5/007G06T5/50G06T2207/10144G06T2207/20208G06T2207/20212H04N5/2355H04N5/23238
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,338,349
App. No.
14/012,534
Granted
May 10, 2016
Kind
B2
Abstract

Apparatuses and methods for reading a set of images to merge together into a high dynamic range (HDR) output image are described. Images have a respective HDR weight and a respective ghost-free weight. Images are merged together using the weighted average of the set of input images using the ghost-free weight. A difference image is determined based on a difference between each pixel within a HDR output image and each respective pixel within a reference image used to create the HDR output image.

Claims (58)

1. A machine-implemented method for high dynamic range (HDR) image processing, the machine-implemented method comprising:

receiving a set of input images, wherein each input image in the set of input images has a respective HDR merge weight;

determining, based on the set of input images, one or more moving components each having a boundary isolating a region where motion is present;

selecting, for each moving component of the one or more moving components, one or more reference images from the set of input images, wherein the one or more reference images are independently selected for each moving component;

determining a respective ghost optimized merge weight for each of one or more reference images; and

creating an HDR output image by merging the set of input images, wherein the merging uses the respective HDR merge weight for pixels outside the boundary of each moving component and uses the respective ghost optimized merge weight instead of the respective HDR merge weight for pixels inside the boundary of a respective moving component.

2. The machine-implemented method of claim 1 , wherein merging the set of input images uses a weighted average of the set of input images and reduces motion ghost artifacts of the HDR output image while increasing dynamic range from any one of the set of input images.

3. The machine-implemented method of claim 1 , wherein the respective HDR merge weight is a baseline merge weight optimized for merging input images into the HDR output image, and wherein the respective ghost optimized merge weight modifies the respective HDR merge weight to reduce motion ghost artifacts upon merging into the HDR output image.

4. The machine-implemented method of claim 1 , further comprising:

matching one or both of chroma and illumination of an input image from the set of input images to the HDR output image.

5. The machine-implemented method of claim 4 , wherein matching further comprises:

creating an offset image by blurring a difference image of the HDR output image and the input image; and

adding the offset image to the input image to create an artifact-free combination of images.

6. The machine-implemented method of claim 5 , wherein the difference image is created by determining a difference between each pixel within the HDR output image and each respective pixel within the input image.

7. An apparatus for image processing comprising:

means for receiving a set of input images, wherein each input image in the set of input images has a respective high dynamic range (HDR) merge weight;

means for determining, based on the set of input images, one or more moving components each having a boundary isolating a region where motion is present;

means for selecting, for each moving component of the one or more moving components, one or more reference images from the set of input images, wherein the one or more reference images are independently selected for each moving component;

means for determining a respective ghost optimized merge weight for each of the one or more reference images; and

means for creating an HDR output image by merging the set of input images, wherein the merging uses the respective HDR merge weight for pixels outside the boundary of each moving component and uses the respective ghost optimized merge weight instead of the respective HDR merge weight for pixels inside the boundary of a respective moving component.

8. The apparatus of claim 7 , wherein merging the set of input images uses a weighted average of the set of input images and reduces motion ghost artifacts of the HDR output image while increasing dynamic range from any one image of the set of input images.

9. The apparatus of claim 7 , wherein the respective HDR merge weight is a baseline merge weight optimized for merging input images into the HDR output image, and wherein the respective ghost optimized merge weight modifies the respective HDR merge weight to reduce motion ghost artifacts upon merging into the HDR output image.

10. The apparatus of claim 7 , further comprising:

means for matching one or both of chroma and illumination of an input image from the set of input images to the HDR output image.

11. The apparatus of claim 10 , wherein the means for matching further comprises:

means for creating an offset image by blurring a difference image of the HDR output image and the input image; and

means for adding the offset image to the input image to create an artifact-free combination of images.

12. The apparatus of claim 11 , wherein the difference image is created by determining a difference between each pixel within the HDR output image and each respective pixel within the input image.

13. A data processing device for image processing comprising:

a processor; and

a storage device coupled to the processor and configurable for storing instructions, which, when executed by the processor cause the processor to:

receive a set of input images, wherein each input image in the set of input images has a respective high dynamic range (HDR) merge weight;

determine, based on the set of input images, one or more moving components each having a boundary isolating a region where motion is present;

select, for each moving component of the one or more moving components, one or more reference images from the set of input images, wherein the one or more reference images are independently selected for each moving component;

determine a respective ghost optimized merge weight for each of the one or more reference images; and

create an HDR output image by merging the set of input images, wherein the merging uses the respective HDR merge weight for pixels outside the boundary of each moving component and uses the respective ghost optimized merge weight instead of the respective HDR merge weight for pixels inside the boundary of a respective moving component.

14. The data processing device of claim 13 , wherein merging the set of input images uses a weighted average of the set of input images and reduces motion ghost artifacts of the HDR output image while increasing dynamic range from any one image of the set of input images.

15. The data processing device of claim 13 , wherein the respective HDR merge weight is a baseline merge weight optimized for merging input images into the HDR output image, and wherein the respective ghost optimized merge weight modifies the respective HDR merge weight to reduce motion ghost artifacts upon merging into the HDR output image.

16. The data processing device of claim 13 , wherein the instructions further comprise instructions to:

match one or both of chroma and illumination of an input image from the set of input images to the HDR output image.

17. The data processing device of claim 16 , wherein the instructions to match further comprises instructions to:

create an offset image by blurring a difference image of the HDR output image and the input image; and

add the offset image to the input image to create an artifact-free combination of images.

18. The data processing device of claim 17 , wherein the difference image is created by determining a difference between each pixel within the HDR output image and each respective pixel within the input image.

19. A machine readable non-transitory storage medium containing executable program instructions which cause a data processing device to perform high dynamic range (HDR) image processing, the executable program instructions comprising:

receiving a set of input images, wherein each input image in the set of input images has a respective HDR merge weight;

determining, based on the set of input images, one or more moving components each having a boundary isolating a region where motion is present;

selecting, for each moving component of the one or more moving components, one or more reference images from the set of input images, wherein the one or more reference images are independently selected for each moving component;

determining a respective ghost optimized merge weight for each of the one or more reference images; and

creating an HDR output image by merging the set of input images, wherein the merging uses the respective HDR merge weight for pixels outside the boundary of each moving component and uses the respective ghost optimized merge weight instead of the respective HDR merge weight for pixels inside the boundary of a respective moving component.

20. The machine readable non-transitory storage medium of claim 19 , wherein merging the set of input images uses a weighted average of the set of input images and reduces motion ghost artifacts of the HDR output image while increasing dynamic range from any one image of the set of input images.

21. The machine readable non-transitory storage medium of claim 19 , wherein the respective HDR merge weight is a baseline merge weight optimized for merging input images into the HDR output image, and wherein the respective ghost optimized merge weight modifies the respective HDR merge weight to reduce motion ghost artifacts upon merging into the HDR output image.

22. The machine readable non-transitory storage medium of claim 19 , the executable program instructions further comprises:

matching one or both of chroma and illumination of an input image from the set of input images to the HDR output image.

23. The machine readable non-transitory storage medium of claim 22 , wherein matching further comprises:

creating an offset image by blurring a difference image of the HDR output image and the input image; and

adding the offset image to the input image to create an artifact-free combination of images.

24. The machine readable non-transitory storage medium of claim 23 , wherein the difference image is created by determining a difference between each pixel within the HDR output image and each respective pixel within the input image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2014
From: SHARMA, PIYUSH; SHROFF, NITESH; REZAIIFAR, RAMIN; SUNG, CHIACHI
To: QUALCOMM INCORPORATED
Reel/Frame 032429/0605 →
Continuity (3)
Provisional Application 61812151 · Apr 15, 2013
Provisional Application 61812148 · Apr 15, 2013
Related Publication 20140307960A1 · Oct 16, 2014