Object monitoring system and methods
In one aspect of the present disclosure, an object monitoring system for a secured area is provided. The object monitoring system includes an image sensor operable to capture an image of the secured area and a memory configured to store a machine learning algorithm trained to identify a vehicle in the secured area, the machine learning algorithm including feature maps of training images captured by the image sensor. The object monitoring system further includes a processor operably coupled to the image sensor and the memory, the processor configured to calculate a feature descriptor of the image and to utilize the machine learning algorithm and the image of the secured area to determine whether a vehicle is present in the secured area by determining a correlation between the feature descriptor of the image and the feature maps of the training images.
1 . An object monitoring system for a secured area, the object monitoring system comprising:
an image sensor operable to capture an image of the secured area, the secured area corresponding to an interior of a garage, wherein the image sensor is part of a movable barrier operator configured to raise and lower a movable barrier associated with the garage;
a memory configured to store a machine learning algorithm trained to identify a vehicle in the secured area, the machine learning algorithm including feature maps of training images captured by the image sensor; and
a processor operably coupled to the image sensor and the memory, the processor having a run mode in which the processor calculates a feature descriptor of the image and utilizes the machine learning algorithm and the image of the secured area to determine whether a vehicle is present in the secured area by determining a correlation between the feature descriptor of the image and the feature maps of the training images, wherein the processor determines a confidence of the image corresponding to one of a plurality of conditions, wherein the processor has a retrain mode wherein the processor retrains the machine learning algorithm, wherein the processor exits the run mode and changes to the retrain mode upon the confidence being below a prescribed threshold, and wherein the processor exits the retrain mode and changes to the run mode after completing retraining.
2 . An object monitoring system for a secured area, the object monitoring system comprising:
an image sensor operable to capture an image of the secured area, wherein the image sensor is part of a movable barrier operator configured to raise and lower a movable barrier associated with the secured area;
a memory configured to store a machine learning algorithm trained to identify a vehicle in the secured area, the machine learning algorithm including feature maps of training images captured by the image sensor; and
a processor operably coupled to the image sensor and the memory, the processor configured to calculate a feature descriptor of the image and to utilize the machine learning algorithm and the image of the secured area to determine whether a vehicle is present in the secured area by determining a correlation between the feature descriptor of the image and the feature maps of the training images, wherein the processor determines a confidence of the image corresponding to one of a plurality of conditions, wherein the processor has a retrain mode wherein the processor retrains the machine learning algorithm, wherein the processor exits the run mode and changes to the retrain mode upon the confidence being below a prescribed threshold, wherein the processor exits the retrain mode and changes to the run mode after completing retraining, wherein the retrain mode is more resource intensive than the run mode, and wherein the processor is configured to cause the image sensor to capture the image of the secured area upon a state change of a movable barrier of the secured area.
3 . The object monitoring system of claim 1 , wherein the retrain mode is more resource intensive than the run mode.
4 . The object monitoring system of claim 1 wherein the feature maps include features extracted from training images captured by the image sensor; and
wherein calculating the feature descriptor includes extracting features of the image, wherein utilizing the machine learning algorithm and the image of the secured area includes comparing the extracted features of the image to the extracted features of the training images.
5 . The object monitoring system of claim 1 wherein the feature maps of the training images are calculated using at least one of:
a histogram of oriented gradients; and
a scale-invariant feature transform.
6 . The object monitoring system of claim 1 wherein the machine learning algorithm is trained to identify a plurality of conditions, wherein at least one of the conditions corresponds to a presence of the vehicle;
wherein the feature maps of the training images include a plurality of feature maps for individual ones of the conditions, the feature maps determined using training images captured by the image sensor; and
wherein the processor is configured to determine the presence of the vehicle by comparing the feature descriptor to the plurality of feature maps of at least one of the conditions.
7 . The object monitoring system of claim 1 wherein the machine learning algorithm is trained to identify a plurality of conditions, wherein at least one of the conditions corresponds to a presence of the vehicle; and
wherein the feature maps are calculated by averaging feature descriptors of training images that represent the same condition.
8 . The object monitoring system of claim 1 wherein the image sensor has a field of view for capturing images;
wherein the machine learning algorithm includes a mask corresponding to a portion of the field of view of the image sensor; and
wherein the processor is configured to analyze the portion of the field of view of the image to determine whether the vehicle is present.
9 . The object monitoring system of claim 1 wherein the machine learning algorithm includes masks produced from training images captured by the image sensor; and
wherein the processor is configured to utilize the machine learning algorithm and the image of the secured area including comparing a mask calculated for the image with the masks produced from the training images.
10 . The object monitoring system of claim 1 wherein the machine learning algorithm includes a first machine learning algorithm trained using training images of the secured area captured by the image sensor and a second machine learning algorithm that utilizes data from the first machine learning algorithm; and
wherein the processor is configured to utilize the second machine learning algorithm and the image to determine whether the vehicle is present in the secured area.
11 . The object monitoring system of claim 1 wherein the processor is configured to request a user input regarding the image; and
wherein, in the retrain mode, the processor is configured to cause the machine learning algorithm to learn a condition of the image in response to the user input indicating the image represents new condition.
12 . The object monitoring system of claim 1 wherein the processor is configured to utilize the machine learning algorithm and the image of the secured area to determine a state of a movable barrier of the secured area.
13 . The object monitoring system of claim 1 wherein the machine learning algorithm is trained using training images of the secured area captured by the image sensor; and
wherein the machine learning algorithm is trained to identify a plurality of conditions of the secured area, the plurality of conditions including a condition associated with a presence of the vehicle; and
wherein the processor is configured to determine the presence of the vehicle by associating the image of the secured area with one of the conditions of the secured area.
14 . The object monitoring system of claim 1 wherein the image sensor is operable to capture the image of the secured area including a plurality of parking spaces of the secured area; and
wherein the processor is configured to determine whether the vehicle is present in the secured area including identifying one of the parking spaces of the secured area in which the vehicle is present.
15 . The object monitoring system of claim 1 wherein at least a portion of the machine learning algorithm is performed by a neural network.
16 . The object monitoring system of claim 1 , wherein the processor is configured to communicate the determination of whether the vehicle is present in the secured area to a user device.
17 . A method of monitoring a secured area using an object monitoring system having a non-transitory computer readable memory storing a machine learning algorithm trained to identify a vehicle in the secured area, the machine learning algorithm including feature maps of training images captured by an image sensor of the object monitoring system, the method comprising:
capturing, via the image sensor contained by a movable barrier operator configured to raise and lower a movable barrier associated with the secured area, an image of the secured area;
calculating a feature descriptor of the image;
determining, via a processor of the object monitoring system, whether a vehicle is present in the secured area using the machine learning algorithm and the image of the secured area at least by determining a correlation between the feature descriptor of the image and the feature maps of the training images;
determining a confidence of the image corresponding to one of a plurality of conditions;
communicating the determination of whether the vehicle is present in the secured area to a smart home system when the confidence is above a prescribed threshold, the smart home system configured to control an action in response to receiving the communicated determination;
upon the confidence being below the prescribed threshold, exiting a run mode and changing to a retrain mode wherein the processor retrains the machine learning algorithm; and
after completing retraining, exiting the retrain mode and changing to the run mode.
18 . The method of claim 17 wherein capturing the image of the secured area includes capturing the image upon a state change of a movable barrier of the secured area.
19 . The method of claim 17 further comprising:
receiving information regarding a movable barrier operator system associated with the secured area; and
wherein determining whether the vehicle is present includes using the machine learning algorithm, the image, and a state change of a movable barrier caused by the movable barrier operator system.
20 . The method of claim 17 wherein the feature maps include features extracted from training images captured by the image sensor; and
wherein calculating the feature descriptor of the image includes extracting features of the image, wherein determining whether the vehicle is present includes comparing the extracted features of the image to extracted features of the training images.
21 . The method of claim 17 wherein the feature maps of the training images are calculated using at least one of:
a histogram of oriented gradients; and
a scale-invariant feature transform.
22 . The method of claim 17 wherein the machine learning algorithm is trained to identify a plurality of conditions, wherein at least one of the conditions corresponds to a presence of the vehicle;
wherein the feature maps of the training images include a plurality of feature maps for individual ones of the conditions, the feature maps determined using training images captured by the image sensor; and
wherein determining the presence of the vehicle includes comparing the feature descriptor to the plurality of feature maps of at least one of the conditions.
23 . The method of claim 17 wherein the machine learning algorithm is trained to identify a plurality of conditions, wherein at least one of the conditions corresponds to a presence of the vehicle; and
wherein the feature maps are calculated by averaging feature descriptors of training images that represent the same condition.
24 . The method of claim 17 wherein the image sensor has a field of view for capturing images;
wherein the machine learning algorithm includes a mask corresponding to a portion of the field of view of the image sensor; and
wherein determining whether the vehicle is present includes analyzing the portion of the field of view of the image to determine the presence of the vehicle.
25 . The method of claim 17 wherein the machine learning algorithm includes masks produced from training images captured by the image sensor; and
wherein determining whether the vehicle is present includes comparing a mask calculated for the image with the masks produced from the training images.
26 . The method of claim 17 wherein the machine learning algorithm includes a first machine learning algorithm trained using training images of the secured area captured by the image sensor and a second machine learning algorithm that utilizes data from the first machine learning algorithm; and
wherein determining whether the vehicle is present includes using the second machine learning algorithm and the image.
27 . The method of claim 17 wherein determining whether the vehicle is present includes associating the image with one of the plurality of conditions of the machine learning algorithm.
28 . The method of claim 17 further comprising requesting a user input regarding the image; and
wherein retraining the machine learning algorithm includes learning a condition of the image in response to the user input indicating the image represents a new condition.
29 . The method of claim 17 further comprising determining a state of a movable barrier of the secured area using the machine learning algorithm and the image of the secured area.
30 . The method of claim 17 wherein the machine learning algorithm is trained using training images of the secured area captured by the image sensor; and
wherein the machine learning algorithm is trained to identify a plurality of conditions of the secured area, the plurality of conditions including a condition associated with the presence of the vehicle; and
wherein determining whether the vehicle is present includes associating the image of the secured area with one of the conditions of the secured area.
31 . The method of claim 17 wherein capturing the image of the secured area includes capturing the image of the secured area including a plurality of parking spaces of the secured area; and
wherein determining whether the vehicle is present includes determining in which one of the parking spaces of the secured area the vehicle is present.
32 . The method of claim 17 wherein at least a portion of the machine learning algorithm is performed by a neural network.
33 . The method of claim 17 further comprising communicating the determination of whether the vehicle is present in the secured area to a user device.