Dynamic absorption window for foreground background detector
View Patent ↗Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.
1. A computer-implemented method for absorbing elements of scene foreground into a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the method comprising:
receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and
for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor, wherein the absorption factor is determined for each pixel in the current video frame classified as depicting scene foreground based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window and wherein the absorption factor is inversely proportional to the frequency at which the pixel is classified as depicting scene foreground over the background window.
2. The method of claim 1 , wherein the background model includes, for each pixel in video frames captured by the video camera, a distribution modeling each of the one or more color channel values for the pixel in the sequence of video frames.
3. The method of claim 2 , wherein the distribution predicts an expected value for each of the one or more color channel values for that pixel in video frames depicting an element of scene background.
4. The method of claim 2 , wherein updating the corresponding pixel data comprises updating a mean associated each distribution modeling one of the one or more color channel values.
5. The method of claim 1 , further comprising, for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixels classified as depicting background in the current video frame, wherein updating the corresponding pixel data comprises updating a mean and a variance associated with a distribution modeling each of the one or more color channel values.
6. The method of claim 1 , wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.
7. A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for absorbing elements of scene foreground into a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the operation comprising:
receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background; and
for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor, wherein the absorption factor is determined for each pixel in the current video frame classified as depicting scene foreground based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window and wherein the absorption factor is inversely proportional to the frequency at which the pixel is classified as depicting scene foreground over the background window.
8. The computer-readable storage medium of claim 7 , wherein the background model includes, for each pixel in video frames captured by the video camera, a distribution modeling each of the one or more color channel values for the pixel in the sequence of video frames.
9. The computer-readable storage medium of claim 8 , wherein the distribution predicts an expected value for each of the one or more color channel values for that pixel in video frames depicting an element of scene background.
10. The computer-readable storage medium of claim 8 , wherein updating the corresponding pixel data comprises updating a mean associated each distribution modeling one of the one or more color channel values.
11. The computer-readable storage medium of claim 7 , wherein the operation further comprises, for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixels classified as depicting background in the current video frame, wherein updating the corresponding pixel data comprises updating a mean and a variance associated with a distribution modeling each of the one or more color channel values.
12. The computer-readable storage medium of claim 7 , wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.
13. A system, comprising:
a video input source configured to provide a sequence of video frames, each depicting a scene;
a central processing unit (CPU);
a graphics processing unit (GPU); and
a memory containing a program, which, when executed on the CPU and GPU is configured to perform an operation for absorbing elements of scene foreground into a background model associated with a scene depicted in a sequence of video frames captured by a video camera, the operation comprising:
receiving image data for a current video frame, wherein the image data classifies each pixel in the current video frame as depicting either foreground or background, and
for each pixel in the current video frame classified as depicting scene foreground, updating corresponding pixel data in the background model based on one or more color channel values of the pixel in the current video frame and an absorption factor, wherein the absorption factor is determined for each pixel in the current video frame classified as depicting scene foreground based, in part, on a frequency at which the pixel is classified as depicting scene foreground over a background window and wherein the absorption factor is inversely proportional to the frequency at which the pixel is classified as depicting scene foreground over the background window.
14. The system of claim 13 , wherein the background model includes, for each pixel in video frames captured by the video camera, a distribution modeling each of the one or more color channel values for the pixel in the sequence of video frames.
15. The system of claim 14 , wherein the distribution predicts an expected value for each of the one or more color channel values for that pixel in video frames depicting an element of scene background.
16. The system of claim 14 , wherein updating the corresponding pixel data comprises updating a mean associated each distribution modeling one of the one or more color channel values.
17. The system of claim 13 , wherein the operation further comprises, for each pixel in the current video frame classified as depicting scene background, updating corresponding pixel data in the background model based on one or more color channel values of the pixels classified as depicting background in the current video frame, wherein updating the corresponding pixel data comprises updating a mean and a variance associated with a distribution modeling each of the one or more color channel values.
18. The system of claim 13 , wherein each pixel in the current video frame is initially classified as depicting either foreground or background based on an evaluation comparing color channel values of each pixel with corresponding color channel values in the background model and a context based approach which selectively reclassifies one or more of the pixels initially classified as depicting foreground or background based on the classification of other pixels in the video as depicting either foreground or background.