Granted Patent B2

US 12,254,659 · App. 17/662,050 · Granted Mar 18, 2025

Inventors: Peter L. Venetianer (McLean, VA); Burak Kakillioglu (Syracuse, NY); Aleksey Lipchin (Newton, MA); Xiao Xiao (Winchester, MA)

Assignee: MOTOROLA SOLUTIONS, INC.

G06V10/255G06T7/194G06V10/82G06V20/52

Abstract

Claims (87)

1. A video surveillance system comprising:

a video camera configured to capture a video; and

an object detector in communication with the video camera and including an electronic processor configured to

receive the video from the video camera,

detect a plurality of candidate objects in the video using a convolutional neural network detection process and a background subtraction detection process;

identify a candidate object from the plurality of candidate objects, the candidate object detected by the background subtraction detection process in a location of the video with no candidate objects detected by the convolutional neural network detection process;

determine a background subtraction confidence level of the candidate object;

categorize the candidate object as a detected object in the video in response to the background subtraction confidence level satisfying a background subtraction confidence threshold; and

in response to the background subtraction confidence level not satisfying the background subtraction confidence threshold

perform an additional convolutional neural network detection process on a location of the candidate object;

determine an additional convolutional neural network confidence level of the candidate object; and

categorize the candidate object as the detected object in the video in response to the additional convolutional neural network confidence level satisfying an additional convolutional neural network confidence threshold.

2. The video surveillance system of claim 1 , wherein the electronic processor is further configured to discard the candidate object in response to the additional convolutional neural network confidence level not satisfying the additional convolutional neural network confidence threshold.

3. The video surveillance system of claim 1 , wherein the electronic processor is further configured to

determine whether a convolutional neural network confidence level of a second candidate object from the plurality of candidate objects satisfies a convolutional neural network confidence threshold, the second candidate object detected by the convolutional neural network detection process;

categorize the second candidate object as a second detected object in the video in response to the convolutional neural network confidence level satisfying the convolutional neural network confidence threshold;

in response to the convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold

determine whether the convolutional neural network confidence level satisfies a second convolutional neural network confidence threshold lower than the convolutional neural network confidence threshold,

in response to the convolutional neural network confidence level satisfying the second convolutional neural network confidence threshold

determine whether a size parameter of the second candidate object satisfies a size threshold for detection,

in response to the size parameter not satisfying the size threshold for detection

determine whether a motion of the second candidate object is isolated by the background subtraction detection process,

in response to the motion of the second candidate object being isolated by the background subtraction detection process

 determine a boosted convolutional neural network confidence level of the second candidate object based on the convolutional neural network confidence level and an overlap ratio of a second location of the second candidate object detected by the convolutional neural network detection process and a third location of the second candidate object detected by the background subtraction detection process; and

discard the second candidate object in response to the boosted convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold.

4. The video surveillance system of claim 3 , wherein the electronic processor further configured to

in response to the boosted convolutional neural network confidence level satisfying the convolutional neural network confidence threshold

categorize the second candidate object as the second detected object.

5. The video surveillance system of claim 1 , wherein the background subtraction detection process includes foreground detection for detecting candidate objects.

6. The video surveillance system of claim 1 , wherein the additional convolutional neural network detection process is different from the convolutional neural network detection process.

7. The video surveillance system of claim 1 , wherein performing the additional convolutional neural network detection process on the location of the candidate object includes performing the additional convolutional neural network detection process on a cropped portion of the location of the video including the candidate object.

8. An object detector comprising:

an electronic processor configured to

detect a plurality of candidate objects in a video using a convolutional neural network detection process and a background subtraction detection process;

identify a candidate object from the plurality of candidate objects, the candidate object detected by the background subtraction detection process in a location of the video with no candidate objects detected by the convolutional neural network detection process;

determine a background subtraction confidence level of the candidate object; and

categorize the candidate object as a detected object in the video in response to the background subtraction confidence level satisfying a background subtraction confidence threshold; and

in response to the background subtraction detection level not satisfying the background subtraction confidence threshold

perform an additional convolutional neural network detection process on a location of the candidate object;

determine an additional convolutional neural network confidence level of the candidate object; and

categorize the candidate object as the detected object in the video in response to the additional convolutional neural network confidence level satisfying an additional convolutional neural network confidence threshold.

9. The object detector of claim 8 , wherein the electronic processor is further configured to discard the candidate object in response to the additional convolutional neural network confidence level not satisfying the additional convolutional neural network confidence threshold.

10. The object detector of claim 8 , wherein the electronic processor is further configured to

determine whether a convolutional neural network confidence level of a second candidate object from the plurality of candidate objects satisfies a convolutional neural network confidence threshold, the second candidate object detected by the convolutional neural network detection process;

categorize the second candidate object as a second detected object in the video in response to the convolutional neural network confidence level satisfying the convolutional neural network confidence threshold;

in response to the convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold

determine whether the convolutional neural network confidence level satisfies a second convolutional neural network confidence threshold lower than the convolutional neural network confidence threshold;

in response to the convolutional neural network confidence level satisfying the second convolutional neural network confidence threshold

determine whether a size parameter of the second candidate object satisfies a size threshold for detection,

in response to the size parameter not satisfying the size threshold for detection

determine whether a motion of the second candidate object is isolated by the background subtraction detection process,

in response to the motion of the second candidate object being isolated by the background subtraction detection process

 determine a boosted convolutional neural network confidence level of the second candidate object based on the convolutional neural network confidence level and an overlap ratio of a second location of the second candidate object detected by the convolutional neural network detection process and a third location of the second candidate object detected by the background subtraction detection process; and

discard the second candidate object in response to the boosted convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold.

11. The object detector of claim 10 , wherein the electronic processor further configured to

in response to the boosted convolutional neural network confidence level satisfying the convolutional neural network confidence threshold

categorize the second candidate object as the second detected object in response to detecting movement of the second candidate object between multiple frames of the video.

12. The object detector of claim 8 , wherein the background subtraction detection process includes foreground detection for detecting candidate objects.

13. The object detector of claim 8 , wherein the additional convolutional neural network detection process is be different from the convolutional neural network detection process.

14. The object detector of claim 8 , wherein performing the additional convolutional neural network detection process on a location of the candidate object includes performing the additional convolutional neural network detection process on a cropped portion of the location of the video including the candidate object.

15. A method for object detection comprising:

detecting, using an electronic processor, a plurality of candidate objects in a video using a convolutional neural network detection process and a background subtraction detection process;

identifying, using the electronic processor, a candidate object from the plurality of candidate objects, the candidate object detected by the background subtraction detection process in a location of the video with no candidate objects detected by the convolutional neural network detection process;

determining, using the electronic processor, a background subtraction confidence level of the candidate object;

categorizing, using the electronic processor, the candidate object as a detected object in the video in response to the background subtraction confidence level satisfying a background subtraction confidence threshold; and

in response to the background subtraction detection level not satisfying the background subtraction confidence threshold

performing an additional convolutional neural network detection process on a location of the candidate object;

determining an additional convolutional neural network confidence level of the candidate object; and

categorizing the candidate object as the detected object in the video in response to the additional convolutional neural network confidence level satisfying an additional convolutional neural network confidence threshold.

16. The method of claim 15 , further comprising discarding the candidate object in response to the additional convolutional neural network confidence level not satisfying the additional convolutional neural network confidence threshold.

17. The method of claim 15 , further comprising:

determining whether a convolutional neural network confidence level of a second candidate object from the plurality of candidate objects satisfies a convolutional neural network confidence threshold, the second candidate object detected by the convolutional neural network detection process;

categorizing the second candidate object as a second detected object in the video in response to the convolutional neural network confidence level satisfying the convolutional neural network confidence threshold;

in response to the convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold

determining whether the convolutional neural network confidence level satisfies a second convolutional neural network confidence threshold lower than the convolutional neural network confidence threshold;

in response to the convolutional neural network confidence level satisfying the second convolutional neural network confidence threshold

determining whether a size parameter of the second candidate object satisfies a size threshold for detection;

in response to the size parameter not satisfying the size threshold for detection

determining whether a motion of the second candidate object is isolated by the background subtraction detection process;

in response to the motion of the second candidate object being isolated by the background subtraction detection process

 determining a boosted convolutional neural network confidence level of the second candidate object based on the convolutional neural network confidence level and an overlap ratio of a second location of the second candidate object detected by the convolutional neural network detection process and a third location of the second candidate object detected by the background subtraction detection process; and

discarding the second candidate object in response to the boosted convolutional neural network confidence level not satisfying the convolutional neural network confidence threshold.

18. The method of claim 17 , further comprising:

in response to the boosted convolutional neural network confidence level satisfying the convolutional neural network confidence threshold

categorizing the second candidate object as the second detected object in response to detecting movement of the second candidate object between multiple frames of the video.

19. The method of claim 15 , wherein the background subtraction detection process includes foreground detection for detecting candidate objects.

20. The method of claim 15 , wherein the additional convolutional neural network detection process different from the convolutional neural network detection process.

Assignments (1)

Continuity (1)

References Cited (19)