Foreground image separation method
A foreground image separation method is disclosed to separate dynamic foreground and static background in a sequence of input images which have been processed either with automatic white balance or brightness control by the camera. First, an input image is received from the camera, and then a white balance or brightness compensation is performed to the input image according to a reference image to generate a compensated image with background color and background brightness which are approximately similar to that of the reference image. Finally, a background subtraction algorithm is performed to the compensated image to generate a background separation result. The background subtraction algorithm could be a Gaussian Mixture Model based algorithm. The method could process successive images received from the camera to continuously generate background separation results and update the reference image accordingly, such that video surveillance system could adapt to the change of illumination.
1. A foreground image separation method for separating dynamic foreground and static background in continuously inputted images with automatic white balance adjustment and brightness variation, comprising:
providing a reference image as a basis for color gain compensation;
receiving an input image from the camera;
performing a color gain compensation step to the input image according to the reference image to generate a compensated image with color and brightness which are approximately similar to that of at least part of regions of the reference image corresponding to the compensated image; and
performing a background subtraction algorithm to generate a background separation result,
wherein the step of providing the reference image further comprises:
processing the input image with the background subtraction algorithm to generate a background mask;
calculating a ratio of foreground portions to whole size of the input image; and
selecting the input image as the reference image if the calculated ratio is less than a predetermined value.
2. The foreground image separation method of claim 1 , wherein the camera continuously generates the input image and the color gain compensation step is continuously performed to continuously generate the compensated image.
3. The foreground image separation method of claim 2 , wherein the background subtraction algorithm is a Gaussian Mixture Model based algorithm that utilizes the Gaussian Mixture Models as a statistical representation of static background image and thereby a background separation result can be obtained based on the decision of foreground or background for each pixel in the input image.
4. The foreground image separation method of claim 2 , wherein the color gain compensation step further comprises:
calculating color characteristics of the reference image and the input image;
comparing the color characteristics of the reference image to that of the input image to generate at least one compensation value; and
multiplying pixel color values of the input image by the compensation value to generate the compensated image.
5. The foreground image separation method of claim 4 , wherein the step of calculating color characteristics of the reference image and the input image further comprises:
separately dividing the reference image and the input image into blocks, wherein an amount and a size of each of the blocks in the reference image are same as those in the input image, each block of the reference image having a one-to-one position relationship to one of the blocks of the input image;
calculating a characteristic value of each block according to pixel color value of each block;
dividing the characteristic value of each block in the reference image by the characteristic value of a corresponding block in the input image to obtain a ratio corresponding to each of the blocks; and
performing a distribution analysis of the ratios to obtain the compensation value.
6. The foreground image separation method of claim 5 , wherein the step of performing a distribution analysis of the ratios further comprises:
dividing the ratios of all block pairs by a quantization width to obtain a plurality of quantized values and counting the occurrence for each quantized value as a histogram of the ratios; and
selecting a quantized value which is with a greatest occurrence number in the histogram to be the compensation value.
7. The foreground image separation method of claim 6 , wherein the step of selecting a quantized value as the compensation value further comprises:
setting the compensation value as one if all of the occurrence numbers of quantized values are not greater than a reliable ratio of a total number of all of the blocks such that the compensated image is equal to the input image.
8. The foreground image separation method of claim 7 , wherein the reliable ratio is defined as five percent.
9. The foreground image separation method of claim 6 , wherein:
the color characteristic values are processed with three primary colors that are red, green and blue separately such that each of the color characteristic values includes a red characteristic value, a green characteristic value and a blue characteristic value;
the compensation value includes a red compensation value, a green compensation value and a blue compensation value;
the ratio value includes a ratio value for red color, a ratio value for green color and a ratio value for blue color;
the histogram includes a histogram for red color, a histogram for green color and a histogram for blue color; and
the compensated image is obtained by multiplying the red, the green and the blue color pixel values by the red compensation value, the green compensation value and the blue compensation value respectively.
10. The foreground image separation method of claim 9 , wherein: the red characteristic value, the green characteristic value and the blue characteristic value of each block is an average value of all of the red color pixel values, an average value of all of the green color pixel values and an average value of all of the blue color pixel values within the block, respectively.
11. The foreground image separation method of claim 9 , wherein: the red characteristic value, the green characteristic value and the blue characteristic value of each block is a maximum value of all of the red color pixel values, a maximum value of all of the green color pixel values and a maximum value of all of the blue color pixel values within the block, respectively.
12. The foreground image separation method of claim 1 , wherein the predetermined value is defined as five percent.
13. The foreground image separation method of claim 6 , wherein:
the reference image and the input image are color images, and brightness characteristic values thereof are obtained by converting color value of each pixel within the reference image and the input image to a grayscale value and calculating characteristic values thereof; and
the compensated image is obtained by multiplying red, green and blue color pixel values of the input image with the compensation value.
14. The foreground image separation method of claim 6 , wherein:
the reference image and the input image are grayscale images, and brightness characteristic values thereof are calculated based on a grayscale value of each pixel within the reference image and the input image; and
the compensated image is obtained by multiplying the grayscale value of each pixel within the input image with the compensation value.