Detecting and responding to an out-of-focus camera in a video analytics system
Techniques are disclosed for detecting an out-of-focus camera in a video analytics system. In one embodiment, a preprocessor component performs a pyramid image decomposition on a video frame captured by a camera. The preprocessor further determines sharp edge areas, candidate blurry edge areas, and actual blurry edge areas, in each level of the pyramid image decomposition. Based on the sharp edge areas, the candidate blurry edge areas, and actual blurry edge areas, the preprocessor determines a sharpness value and a blurriness value which indicate the overall sharpness and blurriness of the video frame, respectively. Based on the sharpness value and the blurriness value, the preprocessor further determines whether the video frame is out-of-focus and whether to send the video frame to components of a computer vision engine and/or a machine learning engine.
1. A computer-implemented method for detecting a camera being out-of-focus, the method comprising:
receiving a video frame;
performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform;
determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure;
determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame;
determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame; and
determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.
2. The method of claim 1 , wherein the sharpness value is determined as the ratio of a number of sharp edge areas to total edge areas, and wherein the blurriness value is determined as a ratio of a number of blurry edge areas to candidate blurry edge areas.
3. The method of claim 1 , further comprising:
upon determining the video frame is not out-of-focus, sending the video frame to a computer vision engine,
wherein the computer vision engine is configured to analyze video frames captured by the camera and send data and provide an information stream to a machine learning engine, and
wherein the machine learning engine is configured to learn patterns of behavior of objects in a scene based on at least data supplied by the computer vision engine.
4. The method of claim 3 , wherein the camera is determined to not be in focus and the video frame is not sent to the computer vision engine if:
(blurriness>BLUR_LEVEL — 3) AND (sharpness<SHARP_LEVEL — 3) OR
(blurriness>BLUR_LEVEL — 2) AND (sharpness<SHARP_LEVEL — 2) OR
(blurriness>BLUR_LEVEL — 1) AND (sharpness<SHARP_LEVEL — 1),
where BLUR_LEVEL — 3>BLUR_LEVEL — 2>BLUR_LEVEL — 1, and SHARP_LEVEL — 3>SHARP_LEVEL — 2>SHARP_LEVEL — 1.
5. The method of claim 1 , further comprising, issuing an alert to a user interface if one or more received frames are determined to be out-of-focus.
6. The method of claim 1 , wherein the candidate blurry edge areas are areas including edges with either a roof or a Gstep structure.
7. The method of claim 1 , wherein the blurry edge areas are areas which do not include sharp edges in a first level of the pyramid image decomposition and which include blurry edges at one or more higher levels of the pyramid image decomposition.
8. The method of claim 1 , wherein the sharp edge areas, the candidate blurry edge areas, and the blurry edge areas are each square or rectangular areas in the received video frames.
9. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for detecting a camera being out-of-focus, the operations comprising:
receiving a video frame;
performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform;
determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure;
determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame;
determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame; and
determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.
10. The computer-readable storage medium of claim 9 , wherein the sharpness value is determined as the ratio of a number of sharp edge areas to total edge areas, and wherein the blurriness value is determined as a ratio of a number of blurry edge areas to candidate blurry edge areas.
11. The computer-readable storage medium of claim 9 , the operations further comprising:
upon determining the video frame is not out-of-focus, sending the video frame to a computer vision engine,
wherein the computer vision engine is configured to analyze video frames captured by the camera and send data and provide an information stream to a machine learning engine, and
wherein the machine learning engine is configured to learn patterns of behavior of objects in a scene based on at least data supplied by the computer vision engine.
12. The computer-readable storage medium of claim 11 , wherein the camera is determined to not be in focus and the video frame is not sent to the computer vision engine if:
(blurriness>BLUR_LEVEL — 3) AND (sharpness<SHARP_LEVEL — 3) OR
(blurriness>BLUR_LEVEL — 2) AND (sharpness<SHARP_LEVEL — 2) OR
(blurriness>BLUR_LEVEL — 1) AND (sharpness<SHARP_LEVEL — 1),
where BLUR_LEVEL — 3>BLUR_LEVEL — 2>BLUR_LEVEL — 1, and SHARP_LEVEL — 3>SHARP_LEVEL — 2>SHARP_LEVEL — 1.
13. The computer-readable storage medium of claim 9 , the operations further comprising, issuing an alert to a user interface if one or more received frames are determined to be out-of-focus.
14. The computer-readable storage medium of claim 9 , wherein the candidate blurry edge areas are areas including edges with roof or Gstep structures.
15. The computer-readable storage medium of claim 9 , wherein the blurry edge areas are areas which do not include sharp edges in a first level of the pyramid image decomposition and which include blurry edges at one or more high levels of the pyramid image decomposition.
16. The computer-readable storage medium of claim 9 , wherein the sharp edge areas, the candidate blurry edge areas, and the blurry edge areas are each square or rectangular areas in the received video frames.
17. A system, comprising:
a processor; and
a memory, wherein the memory includes an application program configured to perform operations for detecting a camera being out-of-focus, the operations comprising:
receiving a video frame,
performing a pyramid image decomposition on the video frame, the pyramid image decomposition being performed using a multi-level wavelet transform,
determining sharp edge areas, candidate blurry edge areas, and blurry edge areas in each level of the pyramid image decomposition, wherein the sharp edge areas are areas including edges with either a Dirac or an Astep structure,
determining a sharpness value based on at least the sharp edge areas, the sharpness value indicating an overall sharpness of the video frame,
determining a blurriness value based on at least the blurry edge areas and the candidate blurry edge areas, the blurriness value indicating an overall blurriness of the video frame, and
determining, based on the sharpness value and the blurriness value, whether the video frame is out-of-focus.