Foreground detector for video analytics system
Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.
1. A processor-implemented method, comprising:
generating, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including:
classifying each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;
performing at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and
analyzing, via the at least one processor, at least one new video frame based on the background model.
2. The method of claim 1 , wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.
3. The method of claim 1 , wherein the at least one contextual evaluation includes dilating each pixel classified as a foreground pixel.
4. The method of claim 1 , wherein the at least one contextual evaluation includes eroding each pixel classified as a foreground pixel.
5. The method of claim 1 , wherein the classifying of each pixel of the plurality of pixels is based on a distance between that pixel and a pixel of the background model.
6. The method of claim 1 , wherein the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model.
7. The method of claim 1 , wherein:
the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and
as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold.
8. The method of claim 1 , wherein:
the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and
as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold,
the method further comprising dynamically updating the threshold.
9. The method of claim 1 , wherein:
the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and
as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold,
the method further comprising dynamically updating the threshold based on a camera noise model.
10. A system, comprising:
a processor; and
a memory operably coupled to the processor and storing processor-executable instructions to cause the processor to:
generate, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including:
classify each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;
perform at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and
analyze, via the at least one processor, at least one new video frame based on the background model.
11. The system of claim 10 , wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.
12. The system of claim 10 , wherein the at least one contextual evaluation includes one of dilating or eroding each pixel classified as a foreground pixel.
13. The system of claim 10 , wherein the memory stores processor-executable instructions to cause the processor to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model.
14. The system of claim 10 , wherein the memory stores processor-executable instructions to cause the processor to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model, such that the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold.
15. A non-transitory, processor-readable medium storing processor-executable instructions to:
generate, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including:
classify each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;
perform at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and
analyze, via the at least one processor, at least one new video frame based on the background model.
16. The processor-readable medium of claim 15 , wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.
17. The processor-readable medium of claim 15 , wherein the at least one contextual evaluation includes one of dilating or eroding each pixel classified as a foreground pixel.
18. The processor-readable medium of claim 15 , wherein the medium stores processor-executable instructions to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model.
19. The processor-readable medium of claim 15 the medium stores processor-executable instructions to classify each pixel as a foreground pixel if the distance exceeds a pre-determined threshold.