Granted Patent B2

US 12,536,615 · App. 18/068,916 · Granted Jan 27, 2026

Inventors: Madhuri Suthar (Pasadena, CA); Tyler Luu (Richardson, TX); Nguyen Thang Long Le (Garland, TX); John W. Glotzbach (Allen, TX); Hamid R. Sheikh (Allen, TX)

Assignee: Samsung Electronics Co., Ltd.

G06T5/30G06T3/4015G06T5/50G06T2207/20024G06T2207/20036G06T2207/20212

Abstract

Claims (61)

1 . A method comprising:

obtaining multiple input image frames, at least two of the input image frames captured using different capture conditions;

separating color channels of each of the input image frames;

generating at least one bad pixel map for each color channel of each input image frame, wherein each bad pixel map is generated by identifying one or more outliers in pixel intensity values in a corresponding color channel based on a difference between a first maximum pixel intensity value and an N th maximum pixel intensity value or a difference between a first minimum pixel intensity value and an N th minimum pixel intensity value of an intensity distribution of the pixel intensity values in a specified operation window within the corresponding color channel, wherein N represents two or three, and wherein the specified operation window has a window size based on at least one of: the capture condition associated with a corresponding image frame or local image content in the corresponding image frame;

combining the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;

performing a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and

using one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

2 . The method of claim 1 , wherein the morphological operation comprises removing clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

3 . The method of claim 1 , wherein the morphological operation comprises spatial filtering of the at least one combined bad pixel map to produce the at least one refined bad pixel map.

4 . The method of claim 1 , wherein the morphological operation comprises performing erode operations and dilate operations, the erode operations replacing the pixel intensity values in the specified operation window with a minimum value, the dilate operations replacing the pixel intensity values in the specified operation window with a maximum value.

5 . The method of claim 1 , wherein generating the at least one bad pixel map for each color channel of each input image frame comprises:

for each color channel of each input image frame:

identifying the first and N th maximum and minimum pixel intensity values in the color channel of the image frame, the first and N th maximum and minimum pixel intensity values based on pixel values within the specified operation window;

identifying adaptive thresholds based on differences between the first and N th maximum and minimum pixel intensity values; and

applying the adaptive thresholds to identify the one or more outliers in the pixel intensity values of the color channel.

6 . The method of claim 5 , wherein identifying the adaptive thresholds comprises:

linearly updating maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel intensity values in the color channel of the image frame reach constant values.

7 . The method of claim 1 , wherein the different capture conditions comprise different exposure settings.

8 . An electronic device comprising:

at least one imaging sensor configured to capture multiple input image frames such that at least two of the input image frames are captured using different capture conditions; and

at least one processing device configured to:

separate color channels of each of the input image frames;

generate at least one bad pixel map for each color channel of each input image frame, wherein, to generate each bad pixel map, the at least one processing device is configured to identify one or more outliers in pixel intensity values in a corresponding color channel based on a difference between a first maximum pixel intensity value and an N th maximum pixel intensity value or a difference between a first minimum pixel intensity value and an N th minimum pixel intensity value of an intensity distribution of the pixel intensity values in a specified operation window within the corresponding color channel, wherein N represents two or three, and wherein the specified operation window has a window size based on at least one of: the capture condition associated with a corresponding image frame or local image content in the corresponding image frame;

combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;

perform a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and

use one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

9 . The electronic device of claim 8 , wherein, to perform the morphological operation, the at least one processing device is configured to remove clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

10 . The electronic device of claim 8 , wherein, to perform the morphological operation, the at least one processing device is configured to spatially filter the at least one combined bad pixel map to produce the at least one refined bad pixel map.

11 . The electronic device of claim 8 , wherein:

to perform the morphological operation, the at least one processing device is configured to perform erode operations and dilate operations;

to perform the erode operations, the at least one processing device is configured to replace the pixel intensity values in the specified operation window with a minimum value; and

to perform the dilate operations, the at least one processing device is configured to replace the pixel intensity values in the specified operation window with a maximum value.

12 . The electronic device of claim 8 , wherein, to generate the at least one bad pixel map for each color channel of each input image frame, the at least one processing device is configured to:

for each color channel of each input image frame:

identify the first and N th maximum and minimum pixel intensity values in the color channel of the image frame, the first and N th maximum and minimum pixel intensity values based on pixel values within the specified operation window;

identify adaptive thresholds based on differences between the first and N th maximum and minimum pixel intensity values; and

apply the adaptive thresholds to identify the one or more outliers in the pixel intensity values of the color channel.

13 . The electronic device of claim 12 , wherein, to identify the adaptive thresholds, the at least one processing device is configured to linearly update maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel intensity values in the color channel of the image frame reach constant values.

14 . The electronic device of claim 8 , wherein the different capture conditions comprise different exposure settings.

15 . A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:

obtain multiple input image frames, at least two of the input image frames captured using different capture conditions;

separate color channels of each of the input image frames;

generate at least one bad pixel map for each color channel of each input image frame, wherein the instructions that when executed cause the at least one processor to generate each bad pixel map comprise instructions that when executed cause the at least one processor to identify one or more outliers in pixel intensity values in a corresponding color channel based on a difference between a first maximum pixel intensity value and an N th maximum pixel intensity value or a difference between a first minimum pixel intensity value and an N th minimum pixel intensity value of an intensity distribution of the pixel intensity values in a specified operation window within the corresponding color channel, wherein N represents two or three, and wherein the specified operation window has a window size based on at least one of: the capture condition associated with a corresponding image frame or local image content in the corresponding image frame;

combine the bad pixel maps associated with the color channels to produce at least one combined bad pixel map;

perform a morphological operation to refine the at least one combined bad pixel map and produce at least one refined bad pixel map; and

use one or more coordinates of one or more bad pixels identified in the at least one refined bad pixel map to update one or more pixel values of at least one of the input image frames.

16 . The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to perform the morphological operation comprise:

instructions that when executed cause the at least one processor to remove clusters of bad pixels identified in the at least one combined bad pixel map to produce the at least one refined bad pixel map.

17 . The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to perform the morphological operation comprise:

instructions that when executed cause the at least one processor to spatially filter the at least one combined bad pixel map to produce the at least one refined bad pixel map.

18 . The non-transitory computer readable medium of claim 15 , wherein:

the instructions that when executed cause the at least one processor to perform the morphological operation comprise instructions that when executed cause the at least one processor to perform erode operations and dilate operations;

the instructions that when executed cause the at least one processor to perform the erode operations comprise instructions that when executed cause the at least one processor to replace the pixel intensity values in the specified operation window with a minimum value; and

the instructions that when executed cause the at least one processor to perform the dilate operations comprise instructions that when executed cause the at least one processor to replace the pixel intensity values in the specified operation window with a maximum value.

19 . The non-transitory computer readable medium of claim 15 , wherein the instructions that, when executed, cause the at least one processor to generate the at least one bad pixel map for each color channel of each input image frame comprise:

instructions that when executed cause the at least one processor, for each color channel of each input image frame, to:

identify the first and N th maximum and minimum pixel intensity values in the color channel of the image frame, the first and N th maximum and minimum pixel intensity values based on pixel values within the specified operation window;

identify adaptive thresholds based on differences between the first and N th maximum and minimum pixel intensity values; and

apply the adaptive thresholds to identify the one or more outliers in the pixel intensity values of the color channel.

20 . The non-transitory computer readable medium of claim 19 , wherein the instructions that when executed cause the at least one processor to identify the adaptive thresholds comprise:

instructions that when executed cause the at least one processor to linearly update maximum and minimum thresholds using predefined slopes until the maximum and minimum pixel intensity values in the color channel of the image frame reach constant values.

Assignments (1)

Continuity (1)

References Cited (16)