IP Library Granted Patent US 11,734,932
Granted Patent B2
US 11,734,932 · App. 17/085,726 · Granted Aug 22, 2023

State and event monitoring

Inventors: Aaron Lee Roberts (Centreville, VA); Bret Jutras (Sterling, VA)
Assignee: Alarm.com Incorporated
G06V20/52G06F18/214G06F18/2193G06F18/22G06V10/809
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,734,932
App. No.
17/085,726
Granted
Aug 22, 2023
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for state and event monitoring. In some implementations, images captured by a camera are obtained, the images depicting an area of a property. Two or more images of the images are provided to a machine learning model. An output of the machine learning model is obtained, the output corresponding to the two or more images. One or more potential states of the area of the property are determined using the output of the machine learning model, each state of the one or more potential states corresponding to an image in the two or more images. An action is performed based on the one or more potential states.

Claims (93)

1. A computer-implemented method comprising:

obtaining images captured by a camera, the images depicting an area of a property;

providing two or more images of the images to a machine learning model;

obtaining an output of the machine learning model corresponding to the two or more images;

determining two or more potential states of the area of the property using the output of the machine learning model, each of the two or more potential states corresponding to an image in the two or more images;

determining that the two or more potential states are dissimilar using a threshold similarity score; and

performing an action based on the two or more potential states not satisfying the threshold similarity score.

2. The method of claim 1 ,

comprising determining that the two or more potential states do not satisfy the threshold similarity score,

wherein performing the action comprises performing an action based on the two or more states not satisfying the threshold similarity score.

3. The method of claim 2 , wherein:

determining that the two or more states do not satisfy the threshold similarity score comprises:

for each of the two or more images, determining, from the output of the machine learning model, a confidence for one or more potential states corresponding to a respective image;

for each of the two or more images, (i) selecting a state for the respective image from the one or more potential states based on the confidences corresponding to one or more potential states or (ii) determining that no state can be identified for the respective image based on the confidences corresponding to one or more potential states, wherein the two or more states are the selected states;

calculating a similarity score for the two or more images based on the two or more states; and

determining that the similarity score corresponding to the two or more images does not satisfy the threshold similarity score.

4. The method of claim 3 , wherein selecting the state for the respective image from the one or more potential states comprises selecting, for the respective image, a state from the one or more potential states associated with the highest confidence.

5. The method of claim 3 , wherein selecting the state for the respective image from the one or more potential states comprises identifying, for the respective image, a state from the one or more potential states associated with a confidence that meets a threshold confidence.

6. The method of claim 3 , wherein determining that no state can be identified for the respective image comprises determining, for the respective image, that none of the confidences associated with the one or more potential states meet a threshold confidence.

7. The method of claim 3 , wherein calculating the similarity score for the two or more images comprises calculating a similarity score that indicates extent of state matches between the two or more states corresponding to the two or more images.

8. The method of claim 3 , wherein calculating the similarity score for the two or more images comprises:

determining a highest number of state matches for a particular state in the two or more states; and

calculating the similarity score using the highest number of state matches, wherein the similarity score is indicative of a comparison between the highest number of state matches and a total number of states in the two or more states.

9. The method of claim 2 , wherein performing the action based on the two or more states not satisfying the threshold similarity score comprises obtaining external data, and the method comprises:

determining a current state of the area of the property using the external data.

10. The method of claim 9 , wherein:

obtaining the external data comprises:

in response to the two or more states not satisfying the threshold similarity score,

generating a request that queries a user to select the current state from among a list of potential states of the area of the property, or that queries the user to input the current state;

transmitting the request to a user device of the user; and

receiving a response to the request from the user device, the response indicating the current state of the area of the property, and

determining the current state of the area of the property using the external data comprises determining the current state of the area of the property using the response.

11. The method of claim 10 , wherein:

generating the request that queries the user to select the current state from among the list of potential states of the area of the property comprises generating a request that queries the user to select from each unique state in the two or more states, or each unique state in the two or more states that is associated with a confidence that meets a threshold confidence,

receiving the response to the request from the user device comprises receiving a response that includes a selection of a first unique state in the two or more states, and

determining the current state of the area of the property using the response comprises determining that the current state of the area of the property is the first unique state of the two or more states.

12. The method of claim 9 , wherein:

obtaining the external data comprises obtaining sensor data from one or more electronic devices, and

determining the current state of the area of the property using the external data comprises verifying a particular state of the two or more states as the current state of the area of the property using the sensor data.

13. The method of claim 9 , wherein:

obtaining the external data comprises:

obtaining one or more new images;

providing the one or more new images to the machine learning model;

obtaining an new output of the machine learning model corresponding to the one or more new images; and

determining a new state of the area of the property using the new output of the machine learning model, and

determining the current state of the area of the property using the external data comprises verifying a particular state of the two or more states as the current state of the area of the property using the new state.

14. The method of claim 9 , comprising:

determining a prior state for the area of the property;

determining that the prior state does not match the current state; and

based on the prior state not matching the current state, notifying a user device of a change in the state of the area of the property.

15. The method of claim 14 , wherein:

determining the current state of the area of the property comprises determining a device is present in the area of the property,

determining that the prior state for the area of the property comprises determining that device was previously not present in the area of the property, and

notifying the user device of the change in the state of the area of the property comprises transmitting a notice to the user device indicating at least one of the following:

a change in state of the area of the property has occurred,

the device was previously not present in the area of the property, and

the device is currently present in the area of the property.

16. The method of claim 14 , wherein:

determining the current state of the area of the property comprises determining a device is connected to a second device,

determining that the prior state for the area of the property comprises determining that device was previously not connected to the second device, and

notifying the user device of the change in the state of the area of the property comprises transmitting a notice to the user device indicating at least one of the following:

a change in state of the area of the property has occurred;

the device was previously not connected to the second device in the area of the property; and

the device is currently connected to the second device in the area of the property.

17. The method of claim 14 , wherein:

determining the current state of the area of the property comprises determining a device is in a first state,

determining that the prior state for the area of the property comprises determining that device was previously in a second state different from the first state, and

notifying the user device of the change in the state of the area of the property comprises transmitting a notice to the user device indicating at least one of the following:

a change in state of the area of the property has occurred;

the device was previously in the second state; and

the device is currently in the first state.

18. The method of claim 9 , comprising:

determining a prior state for the area of the property;

determining that the prior state does not match the current state; and

based on the prior state not matching the current state,

generating instructions for one more external electronic devices to change a mode of the of the one or more electronic devices; and

transmitting the instructions to the one or more electronic devices.

19. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that, when executed, cause the one or more computers to perform operations comprising:

obtaining images captured by a camera, the images depicting an area of a property;

providing two or more images of the images to a machine learning model;

obtaining an output of the machine learning model corresponding to the two or more images;

determining two or more potential states of the area of the property using the output of the machine learning model, each of the two or more potential states corresponding to an image in the two or more images;

determining that the two or more potential states are dissimilar using a threshold similarity score; and

performing an action based on the two or more potential states not satisfying the threshold similarity score.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

obtaining images captured by a camera, the images depicting an area of a property;

providing two or more images of the images to a machine learning model;

obtaining an output of the machine learning model corresponding to the two or more images;

determining two or more potential states of the area of the property using the output of the machine learning model, each of the two or more potential states corresponding to an image in the two or more images;

determining that the two or more potential states are dissimilar using a threshold similarity score; and

performing an action based on the two or more potential states not satisfying the threshold similarity score.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2021
From: ROBERTS, AARON LEE; JUTRAS, BRET
To: ALARM.COM INCORPORATED
Reel/Frame 056170/0570 →
Continuity (2)
Provisional Application 62928558 · Oct 31, 2019
Related Publication 20210133462A1 · May 6, 2021