Background foreground model with dynamic absorption window and incremental update for background model thresholds
Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.
1. A computer-implemented method, comprising:
receiving image data for a current video frame; and
for each pixel classified as depicting scene foreground in the current video frame:
determining an absorption factor based on a frequency at which that pixel is classified as depicting scene foreground over a recent-history window, the absorption factor being a rate at which each of at least one color channel value of that pixel is reclassified as scene background, and
updating, for that pixel, pixel data in a background model based on the absorption factor.
2. The computer-implemented method of claim 1 , wherein the background model includes a distribution modeling each of the at least one color channel value for each pixel in the current video frame.
3. The computer-implemented method of claim 1 , wherein updating the pixel data comprises updating a mean and a variance associated with a distribution modeling each of the at least one color channel value.
4. The computer-implemented method of claim 1 , wherein each pixel in the current video frame is initially classified as depicting either scene foreground or scene background based on an evaluation comparing color channel values of each pixel with color channel values of that pixel in the background model.
5. The computer-implemented method of claim 1 , the method further comprising reclassifying one or more of the pixels initially classified as depicting either scene foreground or scene background based on the classification of at least one other pixel as depicting either scene foreground or scene background.
6. The computer-implemented method of claim 1 , further comprising performing a morphological operation on each pixel in the current video frame classified as depicting scene foreground.
7. The computer-implemented method of claim 1 , further comprising performing a morphological operation on each pixel in the current video frame classified as depicting scene foreground, the morphological operation including one of dilating that pixel or eroding that pixel.
8. The computer-implemented method of claim 1 , further comprising identifying a contiguous region of scene foreground in the current video frame, and comparing the contiguous region of scene foreground with an associated region of pixels in a background image.
9. A non-transitory processor-readable storage medium storing processor-executable instructions to:
receive image data for a current video frame; and
for each pixel classified as depicting scene foreground in the current video frame:
determine an absorption factor based on a frequency at which that pixel is classified as depicting scene foreground over a recent-history window, the absorption factor being a rate at which each of at least one color channel value of that pixel is reclassified as scene background, and
update, for that pixel, pixel data in a background model based on the absorption factor.
10. The non-transitory processor-readable storage medium of claim 9 , wherein the background model includes a distribution modeling each of the one or more channel values for each pixel in the current video frame.
11. The non-transitory processor-readable storage medium of claim 9 , wherein the instructions to update the pixel data include instructions to update a mean and a variance associated with a distribution modeling each of the at least one color channel value.
12. The non-transitory processor-readable storage medium of claim 9 , further storing instructions to perform a morphological operation on each pixel in the current video frame classified as depicting scene foreground.
13. The non-transitory processor-readable storage medium of claim 9 , further storing instructions to perform a morphological operation on each pixel in the current video frame classified as depicting scene foreground, wherein the morphological operation includes one of dilating the pixel or eroding the pixel.
14. The non-transitory processor-readable storage medium of claim 9 , further storing instructions to identify a contiguous region of scene foreground in the current video frame, and comparing the contiguous region of scene foreground with an associated region of pixels in a background image.
15. A system, comprising:
a video input source configured to provide a sequence of video frames;
a processor;
a graphics processing unit (GPU); and
a memory operably coupled to the processor and the GPU, the memory storing instructions that, when executed by at least one of the processor or the GPU, cause the at least one of the processor or the GPU to:
receive image data for a current video frame; and
for each pixel classified as depicting scene foreground in the current video frame:
determine an absorption factor based on a frequency at which that pixel is classified as depicting scene foreground over a recent-history window, the absorption factor being a rate at which each of at least one color channel value of that pixel is reclassified as scene background, and
update, for that pixel, pixel data in a background model based on the absorption factor.
16. The system of claim 15 , wherein the background model includes a distribution modeling each of the one or more channel values for each pixel in the current video frame.
17. The system of claim 15 , wherein the instructions to update the pixel data include instructions to update a mean and a variance associated with a distribution modeling each of the at least one color channel value.
18. The system of claim 15 , wherein the memory further stores instructions to perform a morphological operation on each pixel in the current video frame classified as depicting scene foreground.
19. The system of claim 15 , wherein the memory further stores instructions to perform a morphological operation on each pixel in the current video frame classified as depicting scene foreground, and the morphological operation includes one of dilating the pixel or eroding the pixel.
20. The system of claim 15 , wherein the memory further stores instructions to identify a contiguous region of scene foreground in the current video frame, and comparing the contiguous region of scene foreground with an associated region of pixels in a background image.