Systems, processes and devices for occlusion detection for video-based object tracking
Processes, systems, and devices for occlusion detection for video-based object tracking (VBOT) are described herein. Embodiments process video frames to compute histogram data and depth level data for the object to detect a subset of video frames for occlusion events and generate output data that identifies each video frame of the subset of video frames for the occlusion events. Threshold measurement values are used to attempt to reduce or eliminate false positives to increase processing efficiency.
1. A process for occlusion detection for video-based object tracking (VBOT) comprising:
processing a first video frame to compute first histogram data and first depth level data for a region of interest (ROI) of the first video frame, the ROI of the first video frame including an object being track that is not occluded;
processing a second video frame to compute second histogram data for a ROI of the second video frame, the ROI of the second video frame including the object;
comparing the first histogram data and second histogram data by computing a histogram variation and comparing the histogram variation to a histogram threshold measurement value;
after determining that the histogram variation is greater than the histogram threshold measurement value:
processing the second video frame to compute second depth level data for the ROI of the second video frame;
comparing the first depth level data and the second depth level data to compute a depth level variation;
determining that the second video frame contains occlusion of the object by determining that the depth level variation is greater than a depth threshold measurement value;
transmitting output data that indicates the second video frame contains occlusion of the object.
2. The process of claim 1 wherein processing the first video frame to compute the first histogram data comprises detecting the object being tracked, and defining the ROI of the first video frame.
3. The process of claim 1 wherein computing the histogram variation comprises determining a similarity of two histogram vectors for each of the first histogram data and second histogram data, the histogram vector for the first histogram data representing a distribution of color values of pixels for the first video frame or the ROI of the first video frame, the histogram vector for the second histogram data representing a distribution of color values of pixels for the second video frame or the ROI of the second video frame.
4. The process of claim 1 wherein the first depth level data is computed by processing ROI or an expanded ROI of the first video frame to compute depth level values for pixels of the ROI or expanded ROI of the first video frame, and wherein the first depth level data is computed based on the depth level values for the pixels.
5. The process of claim 1 further comprising updating an average histogram value based on the first histogram data and second histogram data, the histogram threshold measurement value being based on the average histogram value.
6. The process of claim 1 wherein the first depth level data is a value that represents a closeness of the ROI for the first video frame to a camera and the second depth level data is a value that represents a closeness of the ROI for the second video frame to the camera.
7. The process of claim 1 further comprising tracking the object by defining the ROI of the first video frame and ROI of the second video frame and computing the first histogram data and the second histogram data using the ROI of the first video frame and the ROI of the second video frame.
8. The process of claim 7 comprising processing a first expanded ROI based on the ROI of the first video frame and a second expanded ROI based on the ROI of the second video frame and computing the first depth level data by estimating a depth-level value based on the expanded ROI and computing the second depth level data by estimating a depth-level value based on the second expanded ROI.
9. The process of claim 1 wherein the output data comprises an index for the second video frame or a copy of the second video frame.
10. A device for occlusion detection for video-based object tracking (VBOT) comprising:
a transceiver receiving video frames for tracking an object;
a processor configured to:
process a first video frame to compute first histogram data and first depth level data for a region of interest (ROI) of the first video frame, the ROI of the first video frame including an object being track that is not occluded;
process a second video frame to compute second histogram data for a ROI of the second video frame, the ROI second video frame including the-object;
compare the first histogram data and second histogram data by computing a histogram variation and comparing the histogram variation to a histogram threshold measurement value;
after determining that the histogram variation is greater than the histogram threshold measurement value:
process the second video frame to compute second depth level data for the ROI of the second video frame;
compare the first depth level data and the second depth level data to compute a depth level variation;
determine that the second video frame contains occlusion of the object by determining that the depth level variation is greater than a depth threshold measurement value; and
generate output data that indicates that the second video frame contains occlusion of the object; and
a data storage device for storing the output data.
11. The device of claim 10 wherein the processor is configured to process the first video frame to compute the first histogram data by detecting the object being tracked, defining the ROI of the first video frame.
12. The device of claim 10 wherein the processor is configured to compute the histogram variation by determining a similarity of two histogram vectors for each of the first histogram data and second histogram data, the histogram vector for the first histogram data representing a distribution of color values of pixels for the first video frame or the ROI of the first video frame, and the histogram vector for the second histogram data representing a distribution of color values of pixels for the second video frame or the ROI of the second video frame.
13. The device of claim 10 wherein the processor is configured to compute the first depth level data is computed by processing the ROI or an expanded ROI of the first video frame to compute depth level values for pixels of the ROI or expanded ROI of the first video frame, and wherein the first depth level data is computed based on the depth level values for the pixels.
14. The device of claim 10 wherein the processor is configured to update an average histogram value based on the first histogram data and second histogram data, the histogram threshold measurement value being based on the average histogram value.
15. The device of claim 10 wherein the first depth level data is a value that represents a closeness of the ROI for the first video frame to a camera and the second depth level data is a value that represents a closeness of the ROI of the second video frame to the camera.
16. The device of claim 10 wherein the processor is configured to track the object by defining the ROI of the first video frame and the ROI of the second video frame and computing the first histogram data and the second histogram data using the ROI of the first video frame and the ROI of the second video frame.
17. The device of claim 10 wherein the processor is configured to process a first expanded ROI based on the ROI of the first video frame and a second expanded ROI based on the ROI of the second video frame and computing the first depth level data for the object second depth level data for the object by estimating a depth-level value based on the expanded ROIs of the object.
18. A system for occlusion detection for video-based object tracking (VBOT) comprising:
one or more cameras to capture video frames for tracking an object;
a processor configured to:
process a first video frame to compute first histogram data and first depth level data for a region of interest (ROI) of the first video frame, the ROI of the first video frame including an object being track that is not occluded;
process a second video frame to compute second histogram data for a ROI of the second video frame, the ROI second video frame including the object;
compare the first histogram data and second histogram data by computing a histogram variation and comparing the histogram variation to a histogram threshold measurement value;
after determining that the histogram variation is greater than the histogram threshold measurement value:
process the second video frame to compute second depth level data for the ROI of the second video frame;
compare the first depth level data and the second depth level data to compute a depth level variation;
determine that the second video frame contains the occlusion of the object by determining that the depth level variation is greater than a depth threshold measurement value; and
generate output data that indicates that the second video frame contains the occlusion of the object; and
a display device to display a visual representation of the output data or the second video frame.
19. The process of claim 1 wherein the second depth level data is computed by processing the ROI or an expanded ROI of the second video frame to compute depth level values for pixels of the ROI or the expanded ROI of the second video frame, and wherein the second depth level data is computed based on the depth level values for the pixels.
20. The device of claim 10 wherein the second depth level data is computed by processing the ROI or an expanded ROI of the second video frame to compute depth level values for pixels of the ROI or the expanded ROI of the second video frame, and wherein the second depth level data is computed based on the depth level values for the pixels.
21. The system of claim 18 , wherein the processor is configured to compute the histogram variation by determining a similarity of two histogram vectors for each of the first histogram data and second histogram data, the histogram vector for the first histogram data representing a distribution of color values of pixels for the first video frame or the ROI of the first video frame, and the histogram vector for the second histogram data representing a distribution of color values of pixels for the second video frame or the ROI of the second video frame.
22. The system of claim 18 , wherein the processor is configured to compute the first depth level data is computed by processing the ROI or an expanded ROI of the first video frame to compute depth level values for pixels of the ROI or expanded ROI of the first video frame, and wherein the first depth level data is computed based on the depth level values for the pixels.
23. The system of claim 18 , wherein the processor is configured to update an average histogram value based on the first histogram data and second histogram data, the histogram threshold measurement value being based on the average histogram value.
24. The system of claim 18 wherein the second depth level data is computed by processing the ROI or an expanded ROI of the second video frame to compute depth level values for pixels of the ROI or the expanded ROI of the second video frame, and wherein the second depth level data f is computed based on the depth level values for the pixels.