Augmented reality based management of a representation of a smart environment
View Patent ↗A capability for managing a representation of a smart environment is presented herein. The capability for managing a representation of a smart environment is configured to support augmented reality (AR)-based management of a representation of a smart environment, which may include AR-based generation of a representation of the smart environment, AR-based alignment of the representation of the smart environment with the physical reality of the smart environment, and the like.
1. An apparatus, comprising:
a processor and a memory communicatively connected to the processor, the processor configured to:
determine a localization of a device within an environment comprising a physical area having a set of objects located therein, the device having a field of view associated therewith;
determine, based on the localization of the device, detected beacon localization information indicative of a detected localization of a beacon detected as being within the field of view of the device;
determine, based on the field of view of the device and object registration information associated with the environment, a set of registered objects registered as being within the field of view of the device and associated object localization information for the set of registered objects registered as being within the field of view of the device;
determine, based on the detected beacon localization information and the object localization information for the set of registered objects registered as being within the field of view of the device, whether an object is registered as having a registered localization matching the detected localization of the beacon, wherein the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon comprises a determination as to whether the object localization information for the set of registered objects registered as being within the field of view of the device is indicative that an object of the set of objects is registered as having a registered localization matching the detected localization of the beacon; and
perform a management action based on the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon.
2. The apparatus of claim 1 , wherein the processor is configured to determine the localization of the device within the environment based on at least one of:
a model of the physical area and imagery captured by a camera of the device; or
an indoor location determination technique.
3. The apparatus of claim 1 , wherein, to perform the management action, the processor is configured to:
based on a determination that no object is registered as having a registered localization matching the detected localization of the beacon:
identify an object associated with the beacon; and
determine whether the object associated with the beacon is registered as being associated with the environment.
4. The apparatus of claim 3 , wherein, to identify the object associated with the beacon, the processor is configured to:
identify the object associated with the beacon based on information received from the beacon; or
determine an identifier of the beacon, and identify the object associated with the beacon based on a mapping of the identifier of the beacon to the object associated with the beacon.
5. The apparatus of claim 3 , wherein the processor is configured to:
based on a determination that the object associated with the beacon is registered as being associated with the environment:
generate a message indicative that the object associated with the beacon is registered as being associated with the environment and that the object localization information for the object associated with the beacon does not match the detected beacon localization information of the beacon;
generate a message related to updating of the object localization information for the object associated with the beacon; or
generate a message related to movement of the object associated with the beacon to a new location within the environment.
6. The apparatus of claim 3 , wherein the processor is configured to:
based on a determination that the object associated with the beacon is not registered as being associated with the environment:
determine object registration information for the object associated with the beacon; and
initiate storage of the object registration information for the object associated with the beacon.
7. The apparatus of claim 6 , wherein the object registration information for the object associated with the beacon is determined from at least one of information received from the beacon or information received via at least one user input interface.
8. The apparatus of claim 6 , wherein the object registration information for the object associated with the beacon comprises at least one of an identifier of the object associated with the beacon, object localization information for the object associated with the beacon, or object characteristic or capability information for the object associated with the beacon.
9. The apparatus of claim 1 , wherein, to perform the management action, the processor is configured to:
based on a determination that no object is registered as having a registered localization matching the detected localization of the beacon:
determine object registration information for an object associated with the beacon; and
initiate storage of the object registration information for the object associated with the beacon.
10. The apparatus of claim 9 , wherein the object registration information is determined from at least one of information received from the beacon or information received via at least one user input interface.
11. The apparatus of claim 9 , wherein the object registration information comprises at least one of an identifier of the object associated with the beacon, object localization information for the object associated with the beacon, or object characteristic or capability information for the object associated with the beacon.
12. The apparatus of claim 1 , wherein, to perform the management action, the processor is configured to:
based on a determination that no object is registered as having a registered localization matching the detected localization of the beacon:
generate a message related to updating of object localization information for an object associated with the beacon; or
generate a message related to movement of an object associated with the beacon to a new location within the environment.
13. The apparatus of claim 1 , wherein, to perform the management action, the processor is configured to:
based on a determination that an object is registered as having a registered localization matching the detected localization of the beacon:
obtain, from the beacon, detected object configuration information associated with the object registered as having a registered localization matching the detected localization of the beacon;
obtain registered object configuration information associated with the object registered as having a registered localization matching the detected localization of the beacon; and
compare the detected object configuration information and the registered object configuration information.
14. The apparatus of claim 13 , wherein the processor is configured to:
based on a determination that the detected object configuration information and the registered object configuration information do not match:
update the registered object configuration information based on the detected object configuration information.
15. The apparatus of claim 13 , wherein the processor is configured to:
based on a determination that the detected object configuration information and the registered object configuration information do not match:
generate a message indicative that the detected object configuration information and the registered object configuration information do not match;
generate a message related to updating of the registered object configuration information based on the detected object configuration information; or
generate a message related to reconfiguration of the object based on the registered object configuration information.
16. The apparatus of claim 1 , wherein the apparatus is the device or a network device configured for communication with the device.
17. The apparatus of claim 1 , wherein the device comprises an augmented reality device.
18. The apparatus of claim 1 , wherein the apparatus further comprises:
a time-of-flight camera configured to capture imagery of the environment.
19. A non-transitory computer-readable storage medium storing instructions which, when executed by a computer, cause the computer to perform a method, the method comprising:
determining a localization of a device within an environment comprising a physical area having a set of objects located therein, the device having a field of view associated therewith;
determining, based on the localization of the device, detected beacon localization information indicative of a detected localization of a beacon detected as being within the field of view of the device;
determining, based on the field of view of the device and object registration information associated with the environment, a set of registered objects registered as being within the field of view of the device and associated object localization information for the set of registered objects registered as being within the field of view of the device;
determining, based on the detected beacon localization information and the object localization information for the set of registered objects registered as being within the field of view of the device, whether an object is registered as having a registered localization matching the detected localization of the beacon, wherein the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon comprises a determination as to whether the object localization information for the set of registered objects registered as being within the field of view of the device is indicative that an object of the set of objects is registered as having a registered localization matching the detected localization of the beacon; and
performing a management action based on the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon.
20. A method, comprising:
determining, by a processor, a localization of a device within an environment comprising a physical area having a set of objects located therein, the device having a field of view associated therewith;
determining, by the processor based on the localization of the device, detected beacon localization information indicative of a detected localization of a beacon detected as being within the field of view of the device;
determining, by the processor based on the field of view of the device and object registration information associated with the environment, a set of registered objects registered as being within the field of view of the device and associated object localization information for the set of registered objects registered as being within the field of view of the device;
determining, by the processor based on the detected beacon localization information and the object localization information for the set of registered objects registered as being within the field of view of the device, whether an object is registered as having a registered localization matching the detected localization of the beacon, wherein the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon comprises a determination as to whether the object localization information for the set of registered objects registered as being within the field of view of the device is indicative that an object of the set of objects is registered as having a registered localization matching the detected localization of the beacon; and
performing, by the processor, a management action based on the determination as to whether an object is registered as having a registered localization matching the detected localization of the beacon.