Background foreground model with dynamic absorption window and incremental update for background model thresholds
Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.
1. A computer-implemented method, comprising:
receiving image data for a current video frame captured by a video camera, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and
for each pixel in the current video frame classified as depicting scene foreground:
determining an absorption factor based on a frequency at which that pixel is classified as depicting foreground over a recent-history window, and
updating corresponding pixel data in a background model based on one or more color channel values of that pixel and the absorption factor,
wherein the absorption factor is a rate at which each of the one or more color channel values of the pixel in the current video frame classified as depicting scene foreground is reclassified as background.
2. The method of claim 1 , wherein the background model includes, for each pixel in video frames captured by the video camera, a distribution modeling each of the one or more channel values for the pixel in the sequence of video frames.
3. The method of claim 1 , further comprising, for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixels classified as depicting background in the current video frame, wherein updating the corresponding pixel data comprises updating a mean and a variance associated with a distribution modeling each of the one or more color channel values.
4. The method of claim 1 , wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model.
5. The method of claim 4 , the method further comprising reclassifying one or more of the pixels initially classified as depicting either foreground or background based on the classification of at least one other pixel as depicting either foreground or background.
6. The method of claim 1 , further comprising performing a morphological operation on each pixel in the current video frame classified as depicting scene foreground.
7. The method of claim 6 , wherein the morphological operation includes one of dilating the pixel or eroding the pixel.
8. The method of claim 6 , further comprising identifying a contiguous region of foreground in the current video frame, and comparing the contiguous region of foreground with an associated region of pixels in a background image.
9. A non-transitory processor-readable storage medium storing processor-executable instructions to:
receive image data for a current video frame captured by a video camera, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and
for each pixel in the current video frame classified as depicting scene foreground:
determine an absorption factor based on a frequency at which that pixel is classified as depicting foreground over a recent-history window, and
update corresponding pixel data in a background model based on one or more color channel values of that pixel and the absorption factor,
wherein the absorption factor is a rate at which each of the one or more color channel values of the pixel in the current video frame classified as depicting scene foreground are reclassified as background.
10. A system, comprising:
a video input source configured to provide a sequence of video frames;
a processor;
a graphics processing unit (GPU); and
a memory operably coupled to the processor and the GPU, the memory storing instructions that, when executed by at least one of the processor or the GPU, cause the at least one of the processor or the GPU to:
receive image data for a current video frame captured by a video camera, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and
for each pixel in the current video frame classified as depicting scene foreground:
determine an absorption factor based on a frequency at which that pixel is classified as depicting foreground over a recent-history window, and
update corresponding pixel data in a background model based on one or more color channel values of that pixel and the absorption factor,
wherein the absorption factor is a rate at which each of the one or more color channel values of the pixel in the current video frame classified as depicting scene foreground are reclassified as background.