Apparatus and method for controlling a device

1 . A method of training a user interface application of a user device to interact with a controllable device, the user device configured to execute a controllable device application for the controllable device, the user interface application, and an operating system, the method comprising:

receiving, at the user interface application, a request to affect a state of the controllable device, the controllable device being a movable barrier operator;

scanning, by the user interface application, a graphical user interface (GUI) of the controllable device application using human-readable text recognition or shape recognition;

obtaining, by the user interface application, a location of a user interface element in the controllable device application in response to scanning the GUI;

identifying, by the user interface application, a specific string of characters associated with the user interface element;

determining, by the user interface application, a current state of the controllable device based on the identified string of characters; and

controlling, by the user interface application, the controllable device to operate to change from the current state to a different state based on the received request and the determined current state of the controllable device.

2 . The method of claim 1 , wherein controlling the controllable device comprises the user interface application providing, via the operating system, an emulated user input to the controllable device application to cause the controllable device application to affect the state change.

3 . The method of claim 2 , wherein providing the emulated user input to the controllable device application includes providing the emulated user input to cause the controllable device application to affect the movable barrier operator moving an associated barrier.

4 . The method of claim 1 , further comprising:

detecting, by the user interface application, a subsequent notification of the controllable device application communicated to the operating system of the user device, the subsequent notification including a subsequent state of the controllable device;

determining, by the user interface application, the subsequent state of the controllable device based at least in part on the subsequent notification of the controllable device application communicated to the operating system of the user device; and

providing, via a user interface of the user device, the subsequent state of the controllable device determined by the user interface application.

5 . The method of claim 1 , wherein determining, by the use interface application, the state of the controllable device includes emulating user input at the user interface to operate the controllable device application.

6 . The method of claim 5 , wherein emulating user input to operate the controllable device application includes navigating to the location of the user interface element in the GUI; and wherein determining the state of the controllable device includes the user interface application reviewing the string of characters associated with the user interface element.

7 . The method of claim 1 , further comprising:

generating, by the user interface application, a request for authorization to operate the controllable device, wherein the request for authorization is for a second user device;

providing, via a user interface of the user device, a notification of the request for authorization;

receiving, at the user device, a user input granting authorization to control the controllable device;

causing a credential to be provided to the second user device based on the received user input, the credential granting authorization to operate the controllable device; and

controlling, by a user interface application of the second user device, a controllable device application of the second user device to affect a state of the controllable device.

8 . The method of claim 7 , wherein controlling the controllable device application of the second user device comprises the user interface application providing, via the operating system, an emulated user input to the controllable device application of the second user device to cause the controllable device application to affect the state change.

9 . The method of claim 8 , wherein providing the emulated user input to the controllable device application of the second user device includes providing the emulated user input to cause the controllable device application of the second user device to affect the movable barrier operator to move an associated barrier.

10 . The method of claim 8 , wherein causing the credential to be provided to the second user device and controlling the controllable device application of the second user device occur at separate times.

11 . The method of claim 7 , further comprising:

detecting, by the user interface application of the second user device, a subsequent notification of the controllable device application communicated to the operating system of the user device, the subsequent notification including a subsequent state of the controllable device;

determining, by the user interface application of the second user device, the subsequent state of the controllable device based at least in part on the subsequent notification of the controllable device application communicated to the operating system of the second user device; and

providing, via the user interface of the user device, the subsequent state of the controllable device determined by the user interface application of the second user device.

12 . The method of claim 7 , wherein controlling, by the user interface application of the second user device, the controllable device comprises:

executing, on a processor of the second user device, the user interface application of the second user device to control operation of the controllable device application of the second user device;

detecting, by the user interface application of the second user device, configuration data regarding interface components of the controllable device application of the second user device;

determining, by the user interface application of the second user device, configuration data regarding interface components of the controllable device application of the second user device; and

controlling, by the user interface application of the second user device, via the operating system, the controllable device application to affect the state change of the controllable device.

13 . The method of claim 7 , wherein generating the request for authorization occurs when the second user device is in a geofenced area.

14 . The method of claim 7 , wherein generating the request for authorization occurs before the second user device enters a geofenced area.

15 . The method of claim 7 , wherein generating the request for authorization occurs when an item is purchased on the user device.

16 . The method of claim 7 , wherein the user interface application is an e-commerce application.

17 . A system for training a user device to interact with a movable barrier operator, the user device configured to execute a controllable device application for the movable barrier operator, a user interface application, and an operating system, the system comprising:

a processor; and

a memory coupled to the processor, the memory storing instructions, wherein the instructions, when executed by the processor, implements a method comprising:

receiving, at the user interface application, a request to affect a state of the movable barrier operator;

scanning, by the user interface application, a graphical user interface (GUI) of the controllable device application using human-readable text recognition or shape recognition;

obtaining, by the user interface application, a location of a user interface element in the controllable device application in response to scanning the GUI;

identifying, by the user interface application, a specific string of characters associated with the user interface element;

determining, by the user interface application, a current state of the movable barrier operator based on the identified string of characters; and

controlling, by the user interface application, the movable barrier operator to operate to change from the current state to a different state based on the received request and the determined current state of the movable barrier operator.

18 . A non-transitory computer-readable medium storing computer-executable instructions for training a user device to interact with a movable barrier operator, the user device configured to execute a controllable device application for the movable barrier operator, a user interface application, and an operating system, wherein the computer-readable instructions, when executed by a processor, implement a method comprising:

receiving, at the user interface application, a request to change a state of the movable barrier operator;

scanning, by the user interface application, a graphical user interface (GUI) of the controllable device application using a user interface discovery process;

obtaining, by the user interface application, a location of a user interface element in the controllable device application in response to scanning the GUI;

identifying, by the user interface application, a specific string of characters associated with the user interface element;

determining, by the user interface application, a current state of the movable barrier operator based on the identified specific string of characters; and

controlling, by the user interface application, the movable barrier operator to operate to change from the current state to a different state based on the received request and the determined current state of the movable barrier operator.

19 . The non-transitory computer-readable medium of claim 18 , wherein the user interface application is configured to control the movable barrier operator to operate to change from the current state to the different state via the controllable device application running in the background without being displayed on the user device.

20 . The non-transitory computer-readable medium of claim 18 , wherein the determined current state is open or closed, and wherein the different state is the other of open or closed.