Systems and Methods for Controlling Smart Devices Based Upon Image Data From Image Sensors
The present embodiments relate to managing the operation of devices within a home or other property based upon received image data. According to certain aspects, a controller within the home may receive, with customer permission or consent, image data from image sensors disposed throughout the home. The controller may analyze the image data in combination with profile data for individuals associated with the home to determine that a certain individual is indicated in the image data. The controller may further determine an action to facilitate based upon the individual being detected in a certain location of the property, and may direct smart devices to perform that action. As a result, the present embodiments may facilitate providing occupant-location based (or occupant-presence or preference based) functionality, functions, or services, and/or directing operations of smart devices located about a property based upon occupant location, presence, preferences, and/or activities at the property.
1 . A computer-implemented method of controlling a plurality of devices associated with a property, the property populated with a hardware controller in communication with the plurality of devices, the method comprising:
receiving, by the hardware controller via a communication network, image data from a set of image sensors disposed at a location of the property;
accessing, from a memory, at least one occupant profile associated with at least one occupant of the property, the at least one occupant profile containing a set of images depicting the at least one occupant;
comparing, by a processor, the image data to the set of images contained in the at least one occupant profile to determine that the at least one occupant is depicted in the image data, the at least one occupant located at an occupant location that is exterior to the property;
determining, by the processor, a set of additional auxiliary items depicted in the image data and associated with the at least one occupant;
determining, by the processor from the image data, how the at least one occupant is interacting with the set of additional auxiliary items;
based upon the at least one occupant profile:
determining, based upon how the at least one occupant is interacting with the set of additional auxiliary items and the occupant location that is exterior to the property, an action to be performed, and
determining, based upon the location of the property at which the set of image sensors is disposed, at least one of the plurality of devices to perform the action the at least one of the plurality of devices located at a device location at an interior of the property; and
directing the at least one of the plurality of devices to perform the action.
2 . The computer-implemented method of claim 1 , wherein accessing the at least one occupant profile comprises:
accessing the set of images depicting the at least one occupant and a set of physical attributes of the at least one occupant.
3 . The computer-implemented method of claim 2 , wherein comparing the image data to the set of images contained in the at least one occupant profile comprises:
analyzing the image data to detect an individual and a set of characteristics of the individual; and
comparing the set of characteristics of the individual to the set of physical attributes of the at least one occupant to determine that the at least one occupant is depicted in the image data.
4 . The computer-implemented method of claim 1 , wherein comparing the image data to the set of images contained in the at least one occupant profile comprises:
performing a facial recognition technique on the image data and the set of images contained in the at least one occupant profile to determine that the at least one occupant is depicted in the image data.
5 . The computer-implemented method of claim 1 , wherein accessing the at least one occupant profile comprises:
accessing a set of preferences of the at least one occupant;
and wherein determining the action to be performed comprises:
determining the action to be performed according to the set of preferences of the at least one occupant.
6 . The computer-implemented method of claim 5 , wherein determining the at least one of the plurality of devices to perform the action comprises:
determining the at least one of the plurality of devices that is capable of performing the action to comply with the set of preferences of the at least one occupant.
7 . The computer-implemented method of claim 1 , wherein directing the at least one of the plurality of devices to perform the action comprises:
directing the at least one of the plurality of devices to adjust operation.
8 . The computer-implemented method of claim 1 , wherein directing the at least one of the plurality of devices to perform the action comprises:
generating a command to perform the action; and
transmitting, by the hardware controller via the communication network, the command to the at least one of the plurality of devices, wherein the at least one of the plurality of devices executes the command.
9 . The computer-implemented method of claim 1 , further comprising:
receiving, from the at least one of the plurality of devices via the communication network, a confirmation that the action was performed.
10 . (canceled)
11 . The computer-implemented method of claim 1 , further comprising:
determining, based upon the image data, an activity being performed by the at least one occupant;
wherein determining the action to be performed is further based upon the activity being performed by the at least one occupant.
12 . A hardware controller for controlling a plurality of devices associated with a property, the property populated with a hardware controller in communication with the plurality of devices, comprising:
at least one transceiver configured to receive image data from a set of image sensors disposed at a location of the property;
a memory configured to store non-transitory computer executable instructions and a set of occupant profiles; and
a processor interfacing with the at least one transceiver and the memory, wherein the processor is configured to execute the non-transitory computer executable instructions to cause the processor to:
access, from the memory, at least one occupant profile of the set of occupant profiles, the at least one occupant profile associated with at least one occupant of the property, the at least one occupant profile containing a set of images depicting the at least one occupant,
compare the image data to the set of images contained in the at least one occupant profile to determine that the at least one occupant is depicted in the image data, the at least one occupant located at an occupant location that is exterior to the property,
determine a set of additional auxiliary items depicted in the image data and associated with the at least one occupant,
determine, from the image data, how the at least one occupant is interacting with the set of additional auxiliary items,
based upon the data contained in the at least one occupant profile:
determine, based upon how the at least one occupant is interacting with the set of additional auxiliary items and the occupant location that is exterior to the property, an action to be performed, and
determine, based upon the location of the property at which the set of image sensors is disposed, at least one of the plurality of devices to perform the action, the at least one of the plurality of devices located at a device location at an interior of the property, and
direct the at least one of the plurality of devices to perform the action.
13 . The hardware controller of claim 12 , wherein to access the at least one occupant profile, the processor is configured to:
access the set of images depicting the at least one occupant and a set of physical attributes of the at least one occupant.
14 . The hardware controller of claim 13 , wherein to compare the image data to the set of images contained in the at least one occupant profile, the processor is configured to:
analyze the image data to detect an individual and a set of characteristics of the individual, and
compare the set of characteristics of the individual to the set of physical attributes of the at least one occupant to determine that the at least one occupant is depicted in the image data.
15 . The hardware controller of claim 12 , wherein to compare the image data to the set of images contained in the at least one occupant profile, the processor is configured to:
perform a facial recognition technique on the image data and the set of images contained in the at least one occupant profile to determine that the at least one occupant is depicted in the image data.
16 . The hardware controller of claim 12 , wherein to access the at least one occupant profile, the processor is configured to:
access a set of preferences of the at least one occupant,
and wherein to determine the action to be performed, the processor is configured to:
determine the action to be performed according to the set of preferences of the at least one occupant.
17 . The hardware controller of claim 16 , wherein to determine the at least one of the plurality of devices to perform the action, the processor is configured to:
determine the at least one of the plurality of devices that is capable of performing the action to comply with the set of preferences of the at least one occupant.
18 . The hardware controller of claim 12 , wherein to direct the at least one of the plurality of devices to perform the action, the processor is configured to:
direct the at least one of the plurality of devices to adjust operation.
19 . The hardware controller of claim 12 , wherein to direct the at least one of the plurality of devices to perform the action, the processor is configured to:
generate a command to perform the action, and
transmit, via the at least one transceiver, the command to the at least one of the plurality of devices, wherein the at least one of the plurality of devices executes the command.
20 . The hardware controller of claim 12 , wherein the processor is further configured to:
receive, from the at least one of the plurality of devices via the at least one transceiver, a confirmation that the action was performed.
21 .- 40 . (canceled)