Augmented reality with eyewear triggered IoT
Systems, devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device. A portable eyewear device includes a processor, a memory, and a display projected onto at least one lens assembly. The memory has programming stored therein that, when executed by the processor, captures information depicting an environment surrounding the device and identifies a match between objects in that information and predetermined objects in previously obtained information for the same environment. When the position of the eyewear device reaches a preselected location with respect to the matched objects, a physical output is provided to produce the immersive experience. The physical output changes as the position of the eyewear device moves to maintain the immersive experience.
1 . A system comprising:
a device configured to be head mounted on a user, the device comprising;
a processor, a memory, a transceiver, and at least one image sensor; and
programming in said memory, wherein execution of said programming by said processor configures the processor to:
capture, with the at least one image sensor, image information of an environment surrounding the device;
monitor a position of the device based on the image information;
establish, via the transceiver, communication with a remote device, the remote device configured to produce a physical output; and
control the remote device to vary the physical output as the position of the device changes within the environment until the device exits a predefined area of the environment.
2 . The system of claim 1 , wherein to monitor the position of the device the programming configures the processor to:
determine a location of the device within the environment; and
determine an orientation of the device at the determined location.
3 . The system of claim 2 , wherein the device establishes communication with the remote device responsive to the monitored location matching a predetermined location and wherein the device controls the remote device responsive to the location and the orientation.
4 . The system of claim 3 , wherein the predefined area is a predefined radius around the predetermined location.
5 . The system of claim 3 , wherein the programming further configures the processor to:
identify a match between at least one object in the image information and at least one stored object in previously obtained information;
wherein to establish communication when the position of the device reaches the predetermined location the programming configures the processor to:
determine, using said device, a position of said device with respect to objects obtained from the match;
construct, using said device, a map of the environment surrounding said device and determine said device's location within the environment; and
determine, using said device, a head pose of said device within the environment.
6 . The system of claim 1 , wherein the programming further configures the processor to:
identify a match between at least one object in the image information and at least one stored object in previously obtained information;
wherein the previously obtained information includes position information for location points associated with the objects, the at least one matched object includes a first matched object; and
wherein to monitor the position of the device the programming configures the processor to:
identify at least two location points associated with the first matched object;
determine relative position between the at least two location points associated with the first matched object in the image information and corresponding location points for the first matched object in the previously obtaining information; and
calculate the position of the device from the determined relative position and the position information for the corresponding location points.
7 . The system of claim 1 , wherein execution of said programming includes providing, when the position of the device reaches a predetermined location within the environment, a signal to said remote device; and wherein, upon receipt of said signal, the physical output is produced.
8 . The system of claim 1 , wherein the programming further configures the processor to:
identify a match between at least one object in the image information and at least one stored object in previously obtained information;
wherein identifying the match between objects in the information depicting the environment surrounding the device and objects in previously obtained information comprises identifying a first match between a feature in the information depicting the environment surrounding the device and a predetermined feature in the previously obtained information.
9 . The system of claim 1 , the device further comprising a global positioning sensor (GPS) and wherein execution of said programming further configures the processor to:
identify a match between at least one object in the image information and at least one stored object in previously obtained information
receive GPS location information;
request the previously obtained information from a server system using the GPS location information, wherein the previously obtained information corresponds to the GPS location information; and
receive the requested previously obtained information corresponding to the GPS location information.
10 . A method for use with a device configured to be head mounted on a user, the device comprising a processor, a memory, a transceiver, and at least one image sensor, the method comprising:
capturing, with the at least one image sensor, image information of an environment surrounding the device;
monitoring a position of the device within the environment based on the image information;
establishing, via the transceiver, communication with a remote device, the remote device configured to produce a physical output; and
controlling the remote device to vary the physical output as the position of the device changes within the environment until the device exits a predefined area of the environment.
11 . The method of claim 10 , wherein the monitoring the position of the device comprises:
determining a location of the device within the environment; and
determining an orientation of the device at the determined location.
12 . The method of claim 11 , wherein establishing communication with the remote device comprises matching a predetermined location responsive to the monitored location and controlling the remote device comprises controlling the remote device responsive to the location and the orientation.
13 . The method of claim 12 , wherein the predefined area is a predefined radius around the predetermined location.
14 . The method of claim 12 , further comprising providing, when the position of the device reaches the predetermined location within the environment, a signal to said remote device; and wherein, upon receipt of said signal, the physical output is produced.
15 . The method of claim 10 , further comprising:
identifying a match between at least one object in the image information and at least one stored object in previously obtained information;
wherein the previously obtained information includes position information for location points associated with the objects, the at least one matched object includes a first matched object, and wherein the monitoring the position of the device includes:
identifying at least two location points associated with the first matched object;
determining relative position between the at least two location points associated with the first matched object in the image information and corresponding location points for the first matched object in the previously obtaining information; and
calculating the position of the device from the determined relative position and the position information for the corresponding location points.
16 . The method of claim 10 , the device further comprising a global positioning sensor (GPS) and wherein the method further comprises:
identifying a match between at least one object in the image information and at least one stored object in previously obtained information
receiving GPS location information;
requesting the previously obtained information from a server system using the GPS location information, wherein the previously obtained information corresponds to the GPS location information; and
receiving the requested previously obtained information corresponding to the GPS location information.
17 . A non-transitory computer-readable medium storing program code for use with a device configured to be head mounted on a user, the device comprising a processor, a memory, a transceiver, and at least one image sensor, the program code, when executed, is operative to cause a processor to:
capture, with the at least one image sensor, image information of an environment surrounding the device;
monitor a position of the device within the environment based on the image information;
establish, via the transceiver, communication with a remote device, the remote device configured to produce a physical output; and
control the remote device to vary the physical output as the position of the device changes within the environment until the device exits a predefined area of the environment.
18 . The non-transitory computer-readable medium of claim 17 , wherein to monitor the position of the device the program code, when executed, is operative to further cause the processor to:
determine a location of the device within the environment; and
determine an orientation of the device at the determined location.
19 . The non-transitory computer-readable medium of claim 18 , wherein the device establishes communication with the remote device responsive to the monitored location matching a predetermined location and wherein the device controls the remote device responsive to the location and the orientation.
20 . The non-transitory computer-readable medium of claim 19 , wherein the predefined area is a predefined radius around the predetermined location.