Computer vision based activation
An augmented-reality (AR) device detects a physical device located within a first predefined distance of the augmented-reality device. The AR device detects, using an optical sensor, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device. The AR device determines that the physical object is associated with the physical device and identifies a command based on an identification of both the physical object and the physical device. The physical device is configured to operate the command and display, in the display, virtual content as an overlay to the physical object.
1. An augmented-reality device comprising:
an optical sensor;
a display; and
one or more hardware processors comprising a physical device control application, the physical device control application configured to perform operations comprising:
detecting a physical device located within a first predefined distance of the augmented-reality device;
detecting, using the optical sensor, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device;
determining that the physical object is associated with the physical device;
identifying a command based on an identification of both the physical object and the physical device, the physical device configured to operate the command; and
displaying, in the display, virtual content as an overlay to the physical object, the virtual content identifying the command.
2. The augmented-reality device of claim 1 , wherein the operations further comprise:
determining, using the optical sensor, a position of the physical object relative to the physical device, a first orientation of the physical object relative to the augmented-reality device, and a second orientation of the physical object relative to the physical device,
wherein the command is identified based on a combination of the identification of the physical object, the position of the physical object relative to the physical device, the first orientation of the physical object relative to the augmented-reality device, and the second orientation of the physical object relative to the physical device.
3. The augmented-reality device of claim 1 , wherein the operations further comprise:
identifying, using the optical sensor, the physical device and the physical object; and
determining, using the optical sensor, that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
4. The augmented-reality device of claim 1 , wherein the operations further comprise:
wirelessly communicating with the physical device;
identifying the physical device based on the wireless communication between the augmented reality device and the physical device; and
determining, using the optical sensor, that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
5. The augmented-reality device of claim 4 , wherein the operations further comprise:
communicating the command to the physical device using the wireless communication.
6. The augmented-reality device of claim 1 , wherein the operations further comprise:
communicating the command to a server, the server associated with the physical device and coupled to the physical device.
7. The augmented-reality device of claim 1 , wherein the operations further comprise:
detecting that the physical device has performed the command on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
8. The augmented-reality device of claim 1 , wherein the operations further comprise:
receiving a confirmation from the physical device that the command has been performed on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
9. The augmented-reality device of claim 1 , wherein the operations further comprise:
detecting, using the optical sensor, a first and a second physical object located within the predefined distance of the physical device, the first and second physical objects being electronically unconnected to the physical device;
determining that the first and second physical objects are associated with the physical device;
identifying a first request based on a combination of an identification of the first and second physical objects, the position of the first physical object relative to the physical device, the position of the second physical object relative to the physical device, the orientation of the first physical object relative to the augmented-reality device, and the orientation of the of the second physical object relative to the augmented-reality device; and
communicating the first request to the physical device.
10. The augmented-reality device of claim 1 , wherein the operations further comprise:
generating a first virtual content to be displayed over at least one of the first and second physical objects, the first virtual content identifying the first request;
generating a second virtual content identifying a second request, the second virtual content including visual instructions illustrating how to move the first and second physical objects to trigger the second request; and
displaying, in the display, both the first and second virtual content over the first and second physical objects.
11. A method comprising:
detecting a physical device located within a first predefined distance of an augmented-reality device;
detecting, using an optical sensor of the augmented-reality device, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device;
determining, using a hardware processor of the augmented-reality device, that the physical object is associated with the physical device;
identifying a command based on an identification of both the physical object and the physical device, the physical device configured to operate the command; and
displaying, in a display of the augmented-reality office, virtual content as an overlay to the physical object, the virtual content identifying the command.
12. The method of claim 11 , further comprising:
determining, using the optical sensor, a position of the physical object relative to the physical device, a first orientation of the physical object relative to the augmented-reality device, and a second orientation of the physical object relative to the physical device,
wherein the command is identified based on a combination of the identification of the physical object, the position of the physical object relative to the physical device, the first orientation of the physical object relative to the augmented-reality device, and the second orientation of the physical object relative to the physical device.
13. The method of claim 11 , further comprising:
identifying, using the optical sensor, the physical device and the physical object; and
determining, using the optical sensor, that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
14. The method of claim 11 , further comprising:
wirelessly communicating with the physical device;
identifying the physical device based on the wireless communication between the augmented reality device and the physical device; and
determining, using the optical sensor, that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
15. The method of claim 14 , further comprising:
communicating the command to the physical device using the wireless communication or to a server, the server associated with the physical device and coupled to the physical device.
16. The method of claim 11 , further comprising:
detecting that the physical device has performed the command on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
17. The method of claim 11 , further comprising:
receiving a confirmation from the physical device that the command has been performed on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
18. The method of claim 11 , further comprising:
detecting, using the optical sensor, a first and a second physical object located within the predefined distance of the physical device, the first and second physical objects being electronically unconnected to the physical device;
determining that the first and second physical objects are associated with the physical device;
identifying a first request based on a combination of an identification of the first and second physical objects, the position of the first physical object relative to the physical device, the position of the second physical object relative to the physical device, the orientation of the first physical object relative to the augmented-reality device, and the orientation of the of the second physical object relative to the augmented-reality device; and
communicating the first request to the physical device.
19. The augmented-reality device of claim 1 , wherein the operations further comprise:
generating a first virtual content to be displayed over at least one of the first and second physical objects, the first virtual content identifying the first request;
generating a second virtual content identifying a second request, the second virtual content including visual instructions illustrating how to move the first and second physical objects to trigger the second request; and
displaying, in the display, both the first and second virtual content as an overlay over the first and second physical objects.
20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
detecting a physical device located within a first predefined distance of an augmented-reality device;
detecting, using an optical sensor of the augmented-reality device, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device;
determining, using a hardware processor of the augmented-reality device, that the physical object is associated with the physical device;
identifying a command based on an identification of both the physical object and the physical device, the physical device configured to operate the command; and
displaying, in a display of the augmented-reality office, virtual content as an overlay to the physical object, the virtual content identifying the command.