IP Library Granted Patent US 11,620,738
Granted Patent B2
US 11,620,738 · App. 17/373,362 · Granted Apr 4, 2023

Hue preservation post processing with early exit for highlight recovery

Inventors: Touraj Tajbakhsh (San Jose, CA); Chihsin Wu (San Jose, CA); Frederic Cao (Santa Clara, CA)
Assignee: Apple Inc.
G06T5/009G06T7/90H04N9/643G06T2207/10024G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,620,738
App. No.
17/373,362
Granted
Apr 4, 2023
Kind
B2
Abstract

Embodiments relate to hue preservation post processing for highlight recovery of an input image. Intensity values for multiple color channels of a plurality of color channels of a pixel of the input image is determined using corresponding ratios of target hues for the plurality of color channels of the pixel, wherein the pixel has at least one color channel with an intensity above a predetermined threshold. A hue preserved value for a color channel of the plurality of color channels of the pixel is determined using intensity values determined for the plurality of color channels of the pixel and the target hues. A recovered version of the input image is generated by adjusting hue information of the pixel, using the hue preserved value for the channel of the plurality of color channels of the pixel.

Claims (58)

1. An apparatus for processing image signal data, comprising:

a hue target circuit configured to generate a plurality of target hues for a plurality of color channels of a pixel in an image using candidate hue values for the plurality of color channels of the pixel;

an intensity value calculator circuit coupled to the hue target circuit, the intensity value calculator circuit configured to determine an intensity value for a color channel of the plurality of color channels of the pixel, using corresponding ratios of the plurality of target hues for the plurality of color channels of the pixel; and

a hue processor coupled to the hue target circuit and the intensity value calculator circuit, the hue processor configured to select the intensity value as a recovered hue value for the color channel of the pixel, when a target hue of the plurality of target hues for the color channel is the largest among the plurality of target hues.

2. The apparatus of claim 1 , further comprising a hue preservation post processing circuit coupled to the hue processor and the intensity value calculator circuit, the hue preservation post processing circuit being disabled for performing a hue preservation post processing for the color channel of the pixel upon selecting the intensity value as the recovered hue value.

3. The apparatus of claim 1 , wherein the hue processor is further configured to:

compare, for each color channel of the plurality of color channels, the plurality of target hues for the plurality of color channels of the pixel to select the intensity value as the recovered hue value.

4. The apparatus of claim 1 , wherein the hue processor is further configured to:

determine values of the plurality of target hues for the plurality of color channels of the pixel,

generate a signal for the color channel, in response to determining that the target hue for the color channel is the largest among the plurality of target hues, and

configure a demultiplexer circuit coupled to the hue processor using the signal as a select signal of the demultiplexer circuit to output the intensity value for the color channel of the pixel as the recovered hue value for the color channel of the pixel, the demultiplexer circuit coupled to the hue processor and the intensity value calculator circuit.

5. The apparatus of claim 1 , wherein the hue processor is further configured to:

generate a signal for another color channel of the plurality of color channels of the pixel, in response to determining that the target hue for the color channel is the largest among the plurality of target hues, and

configure a demultiplexer circuit coupled to the hue processor using the signal as a select signal of the demultiplexer circuit to pass another intensity value for the other color channel of the pixel from the intensity value calculator circuit to a hue preservation post processing circuit coupled to the intensity value calculator circuit for computing another recovered hue value for the other color channel of the pixel.

6. The apparatus of claim 1 , wherein the intensity value calculator circuit is further configured to:

determine a first ratio between a first of the plurality of target hues for the color channel and a second of the plurality of target hues for another color channel of the plurality of color channels,

determine a second ratio between the first target hue for the color channel and a third of the plurality of target hues for a remaining color channel of the plurality of color channels, and

determine the intensity value for the color channel of the pixel using the first ratio and the second ratio.

7. The apparatus of claim 1 , wherein the intensity value calculator circuit is further configured to:

determine the intensity value for the color channel of the pixel by applying a clipping to a processed version of a pixel value for the color channel of the pixel.

8. The apparatus of claim 7 , wherein the intensity value calculator circuit is further configured to:

limit the processed version of the pixel value to be within a defined maximum hue level of the color channel when applying the clipping.

9. The apparatus of claim 7 , wherein the intensity value calculator circuit is further configured to:

apply a piecewise linear curve to the processed version of the pixel value when applying the clipping, the piecewise linear curve represented as values in a look-up table stored at a computer readable storage medium coupled to the intensity value calculator circuit.

10. A method, comprising:

generating, by a hue target circuit of an image signal processor, a plurality of target hues for a plurality of color channels of a pixel in an image using candidate hue values for the plurality of color channels of the pixel;

determining, by an intensity value calculator circuit of the image signal processor coupled to the hue target circuit, an intensity value for a color channel of the plurality of color channels of the pixel, using corresponding ratios of the plurality of target hues for the plurality of color channels of the pixel; and

selecting, by the hue processor of the image signal processor coupled to the hue target circuit and the intensity value calculator circuit, the intensity value as a recovered hue value for the color channel of the pixel, when a target hue of the plurality of target hues for the color channel is the largest among the plurality of target hues.

11. The method of claim 10 , further comprising:

disabling, by the hue processor, a hue preservation post processing for the color channel of the pixel upon selecting the intensity value as the recovered hue value.

12. The method of claim 10 , further comprising:

comparing, by the hue processor for each color channel of the plurality of color channels, the plurality of target hues for the plurality of color channels of the pixel to select the intensity value as the recovered hue value.

13. The method of claim 10 , further comprising:

determining, by the hue processor, values of the plurality of target hues for the plurality of color channels of the pixel;

generating, by the hue processor, a signal for the color channel, in response to determining that the target hue for the color channel is the largest among the plurality of target hues; and

using, by a demultiplexer circuit of the image signal processor coupled to the hue processor, the signal to output the intensity value for the color channel of the pixel as the recovered hue value for the color channel of the pixel.

14. The method of claim 10 , further comprising:

generating, by the hue processor, a signal for another color channel of the plurality of color channels of the pixel, in response to determining that the target hue for the color channel is the largest among the plurality of target hues; and

using, by a demultiplexer circuit of the image signal processor coupled to the hue processor, the signal to pass another intensity value for the other color channel of the pixel for a hue preservation post processing to compute another recovered hue value for the other color channel of the pixel.

15. The method of claim 10 , further comprising:

determining, by the intensity value calculator circuit, a first ratio between a first of the plurality of target hues for the color channel and a second of the plurality of target hues for another color channel of the plurality of color channels;

determining, by the intensity value calculator circuit, a second ratio between the first target hue for the color channel and a third of the plurality of target hues for a remaining color channel of the plurality of color channels; and

determining, by the intensity value calculator circuit, the intensity value for the color channel of the pixel using the first ratio and the second ratio.

16. The method of claim 10 , further comprising:

determining, by the intensity value calculator circuit, the intensity value for the color channel of the pixel by applying a clipping to a processed version of a pixel value for the color channel of the pixel.

17. The method of claim 16 , further comprising:

limiting, by the intensity value calculator circuit, the processed version of the pixel value to be within a defined maximum hue level of the color channel when applying the clipping.

18. An electronic device, comprising:

an image sensor configured to capture an image comprising a pixel with hue information having at least one color channel of a plurality of color channels with an intensity above a predetermined threshold; and

an image signal processor coupled to the image sensor, the image signal processor configured to perform raw processing of the image to obtain a raw processed version of the image having the plurality of color channels, the image signal processor including:

a hue target circuit configured to generate a plurality of target hues for a plurality of color channels of a pixel in the image using candidate hue values for the plurality of color channels of the pixel,

an intensity value calculator circuit coupled to the hue target circuit, the intensity value calculator circuit configured to determine an intensity value for a color channel of the plurality of color channels of the pixel, using corresponding ratios of the plurality of target hues for the plurality of color channels of the pixel, and

a hue processor coupled to the hue target circuit and the intensity value calculator circuit, the hue processor configured to select the intensity value as a recovered hue value for the color channel of the pixel, when a target hue of the plurality of target hues for the color channel is the largest among the plurality of target hues.

19. The electronic device of claim 18 , wherein the image signal processor further comprises a hue preservation post processing circuit coupled to the hue processor and the intensity value calculator circuit, the hue preservation post processing circuit being disabled for performing a hue preservation post processing for the color channel of the pixel upon selecting the intensity value as the recovered hue value.

20. The electronic device of claim 18 , wherein the hue processor is further configured to:

determine values of the plurality of target hues for the plurality of color channels of the pixel,

generate a signal for the color channel, in response to determining that the target hue for the color channel is the largest among the plurality of target hues, and

configure a demultiplexer circuit using the signal as a select signal of the demultiplexer circuit to output the intensity value for the color channel of the pixel as the recovered hue value for the color channel of the pixel, the demultiplexer circuit coupled to the hue processor and the intensity value calculator circuit.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 14, 2021
From: TAJBAKHSH, TOURAJ; WU, CHIHSIN; CAO, FREDERIC
To: APPLE INC.
Reel/Frame 056857/0952 →
Continuity (2)
Continuation 16121389 · Sep 4, 2018
Related Publication 20210342989A1 · Nov 4, 2021
Cited By (1)
US 12,604,104