Granted Patent B2

US 12,494,063 · App. 17/743,619 · Granted Dec 9, 2025

Inventors: Shaomin Xiong (Newark, CA); Toshiki Hirano (San Jose, CA)

Assignee: Sandisk Technologies, Inc.

G06V20/52G06T7/11G06T7/70G06V20/41G06V20/49H04N5/77H04N7/183H04N17/002G06T2207/10016G06T2207/20021G06T2207/30232

Abstract

Claims (124)

1 . A system, comprising:

a video image sensor configured to be positioned with a field of view of a location; and

a calibration controller configured to, during a calibration period:

receive calibration video data from the video image sensor;

determine, from the calibration video data and using a first set of object classes, at least one background object corresponding to a background setting of the field of view;

determine, based on the at least one background object, scene description parameters for the location;

determine, based on the scene description parameters and from a plurality of possible rules, a post-processing rule set for detected objects at the location, wherein:

the detected objects are a different object class than the at least one background object; and

at least a portion of the plurality of possible rules are indexed to different scene description parameters;

a surveillance controller configured to:

receive, during a surveillance period that is after the calibration period, surveillance video data from the video image sensor, wherein the surveillance video data is different than the calibration video data;

determine, using a field object detector, surveillance object data for a detected object in the surveillance video data;

post-process, using the post-processing rule set, the surveillance object data to determine whether the detected object violates at least one rule from the post-processing rule set, wherein at least one rule in the post-processing rule set is configured to compare a value from the surveillance object data to a rule value indicating that a violation has occurred;

send, responsive to no violation of the post-processing rule set occurring, an event notification for the detected object to a video surveillance application; and

prevent, responsive to the detected object violating the at least one rule, the event notification.

2 . The system of claim 1 , wherein:

determining the surveillance object data for the detected object comprises detecting, by the field object detector, the detected object from the surveillance video data with a confidence score that meets an object detection threshold for the event notification; and

post-processing the surveillance object data is responsive to the detected object meeting the object detection threshold.

3 . The system of claim 2 , further comprising:

a video camera housing, wherein:

the video camera housing encloses:

the video image sensor;

the surveillance controller; and

a network interface configured to communicate with a network; and

the surveillance controller is further configured to:

send, to the calibration controller and over the network:

first calibration object data from the field object detector; and

calibration video data from the video image sensor; and

receive, from the calibration controller and over the network, the plurality of rules for the post-processing rule set.

4 . The system of claim 2 , wherein:

the first set of object classes comprises a plurality of object types corresponding to different background settings;

determining the at least one background object comprises detecting the at least one background object and determining corresponding calibration object data;

the first set of object classes and calibration object data are mapped to the scene description parameters; and

the field object detector is configured for detecting objects that do not include the first set of object classes.

5 . The system of claim 4 , wherein:

the calibration object data includes at least one of:

an object class; and

an object attribute; and

the calibration controller is further configured to, during the calibration period, evaluate, using the calibration object data, a rules data store comprising the plurality of possible rules to determine the at least one rule in the post-processing rule set.

6 . The system of claim 2 , wherein:

the calibration controller is further configured to, during the calibration period:

segment the field of view into a plurality of regions corresponding to the background setting; and

determine, based on the calibration video data, a region identifier for each region of the plurality of regions;

the scene description parameters include at least one region identifier mapped to the at least one background object; and

the at least one rule in the post-processing rule set is based on the detected object appearing in the region corresponding to the at least one region identifier.

7 . The system of claim 2 , wherein:

the calibration controller is further configured to, during the calibration period, determine a plurality of time periods for the location; and

the at least one rule in the post-processing rule set is based on at least one time period of the plurality of time periods.

8 . The system of claim 2 , further comprising:

a rules data store comprising a plurality of rule templates corresponding to the plurality of possible rules, wherein:

each rule template includes:

at least one selection index value; and

at least one rule comparator field;

the calibration controller is further configured to, during the calibration period, use at least one scene description parameter to index, in the rules data store, a selected rule template of the plurality of rule templates to determine the at least one rule in the post-processing rule set; and

the at least one scene description parameter indicates the at least one selection index value of the selected rule template.

9 . The system of claim 8 , wherein the at least one rule comparator field for each rule template of the plurality of rule templates includes at least one of:

a region identifier value;

a time value;

an object class value;

an object attribute value; and

a location condition value.

10 . The system of claim 1 , wherein the surveillance controller is further configured to:

determine at least one location condition; and

use the at least one location condition to post-process the surveillance object data to determine whether the detected object violates the at least one rule from the post-processing rule set.

11 . A computer-implemented method, comprising:

receiving, during a calibration period, calibration video data from a video image sensor positioned with a field of view of a location; and

determining, from the calibration video data and using a first set of object classes, at least one background object corresponding to a background setting of the field of view;

determining, based on the at least one background object, scene description parameters for the location;

determining, based on the scene description parameters and from a plurality of possible rules, a post-processing rule set for detected objects at the location, wherein:

the detected objects are a different object class than the at least one background object; and

at least a portion of the plurality of possible rules are indexed to different scene description parameters;

receiving, during a surveillance period following the calibration period, surveillance video data from the video image sensor, wherein the surveillance video data is different than the calibration video data;

determining, using a field object detector, surveillance object data for a detected object in the surveillance video data;

post-processing, using the post-processing rule set, the surveillance object data to determine whether the detected object violates at least one rule from the post-processing rule set, wherein at least one rule in the post-processing rule set is configured to compare a value from the surveillance object data to a rule value indicating that a violation has occurred;

sending, responsive to no violation of the post-processing rule set occurring, an event notification for the detected object to a video surveillance application; and

preventing, responsive to the detected object violating the at least one rule, the event notification.

12 . The computer-implemented method of claim 11 , wherein:

determining the surveillance object data for the detected object comprises detecting, by the field object detector, the detected object from the surveillance video data with a confidence score that meets an object detection threshold for the event notification; and

post-processing the surveillance object data is responsive to the detected object meeting the object detection threshold.

13 . The computer-implemented method of claim 12 , further comprising, during the calibration period:

determining, based on the calibration video data, calibration object data for the at least one background object in the calibration video data, wherein:

the scene description parameters include the calibration object data; and

the at least one rule in the post-processing rule set is based on the calibration object data.

14 . The computer-implemented method of claim 13 , further comprising, during the calibration period:

evaluating, using the calibration object data, a rules data store comprising the plurality of possible rules to determine the at least one rule in the post-processing rule set, wherein the calibration object data includes at least one of:

an object class; and

an object attribute.

15 . The computer-implemented method of claim 12 , further comprising, during the calibration period:

segmenting the field of view into a plurality of regions corresponding to the background setting; and

determining, based on the calibration video data, a region identifier for each region of the plurality of regions, wherein:

the scene description parameters include at least one region identifier mapped to the at least one background object; and

the at least one rule in the post-processing rule set is based on the detected object appearing in the region corresponding to the at least one region identifier.

16 . The computer-implemented method of claim 12 , further comprising, during the calibration period:

determining a plurality of time periods for the location, wherein the at least one rule in the post-processing rule set is based on at least one time period of the plurality of time periods.

17 . The computer-implemented method of claim 12 , further comprising, during the calibration period:

using at least one scene description parameter to index a selected rule template of a plurality of rule templates to determine the at least one rule in the post-processing rule set, wherein:

each rule template of the plurality of rule templates includes:

at least one selection index value; and

at least one rule comparator field; and

the at least one scene description parameter indicates the at least one selection index value of the selected rule template.

18 . The computer-implemented method of claim 17 , wherein the at least one rule comparator field for each rule template of the plurality of rule templates includes at least one of:

a region identifier value;

a time value;

an object class value;

an object attribute value; and

a location condition value.

19 . The computer-implemented method of claim 12 , further comprising:

determining at least one location condition; and

using the at least one location condition to post-process the surveillance object data to determine whether the detected object violates the at least one rule from the post-processing rule set.

20 . A video camera, comprising:

a video image sensor configured to be positioned with a field of view of a location;

a processor;

a memory;

means for receiving, during a calibration period, calibration video data from the video image sensor;

means for determining, from the calibration video data and using a first set of object classes, at least one background object corresponding to a background setting of the field of view;

means for determining, based on the at least one background object, scene description parameters for the location;

means for determining, based on the scene description parameters and from a plurality of possible rules, a post-processing rule set for detected objects at the location, wherein:

the detected objects are a different object class than the at least one background object; and

at least a portion of the plurality of possible rules are indexed to different scene description parameters;

means for receiving, during a surveillance period, surveillance video data from the video image sensor, wherein the surveillance video data is different than the calibration video data;

means for determining object data for a detected object in the surveillance video data;

means for post-processing, using the post-processing rule set, the object data to determine whether the detected object violates at least one rule from the post-processing rule set, wherein at least one rule in the post-processing rule set is configured to compare a value from the object data to a rule value indicating that a violation has occurred;

means for sending, responsive to no violation of the post-processing rule set occurring, an event notification for the detected object to a video surveillance application; and

means for preventing, responsive to the detected object violating the at least one rule, the event notification.

Assignments (8)

Continuity (1)

References Cited (52)