Computer vision based activation
An augmented-reality (AR) device detects a physical device located within a first predefined distance of the augmented-reality device. The AR device detects, using an optical sensor, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device. The AR device determines that the physical object is associated with the physical device and identifies a command based on an identification of both the physical object and the physical device. The physical device is configured to operate the command and display, in the display, virtual content as an overlay to the physical object.
1. An augmented-reality device comprising:
a display; and
one or more hardware processors configured to perform operations comprising:
detecting that a physical device is located within a first threshold distance of the augmented-reality device;
detecting that a physical object is located within a second threshold distance of the physical device, the physical object being electronically unconnected to the physical device; and
displaying, in the display, an identification of a command to be performed by the physical device based on a presence of the physical device and the physical object, in response to detecting that the physical device is located within the first threshold distance of the augmented-reality device, and that the physical object is located within the second threshold distance of the physical device.
2. The augmented-reality device of claim 1 , further comprising:
an optical sensor,
wherein displaying the identification of the command further comprises:
displaying virtual content as an overlay to the physical object, the virtual content identifying the command.
3. The augmented-reality device of claim 1 , wherein the operations further comprise:
determining that the physical object is associated with the physical device; and
identifying the command based on an identification of both the physical object and the physical device, the physical device configured to operate the command.
4. The augmented-reality device of claim 1 , wherein the operations further comprise:
determining a position of the physical object relative to the physical device, a first orientation of the physical object relative to the augmented-reality device, and a second orientation of the physical object relative to the physical device,
wherein the command is identified based on a combination of the identification of the physical object, the position of the physical object relative to the physical device, the first orientation of the physical object relative to the augmented-reality device, and the second orientation of the physical object relative to the physical device.
5. The augmented-reality device of claim 1 , wherein the operations further comprise:
identifying the physical device and the physical object; and
determining that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
6. The augmented-reality device of claim 1 , wherein the operations further comprise:
wirelessly communicating with the physical device;
identifying the physical device based on the wireless communication between the augmented reality device and the physical device; and
determining that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
7. The augmented-reality device of claim 1 , wherein the operations further comprise:
communicating the command to the physical device using wireless communication.
8. The augmented-reality device of claim 2 , wherein the operations further comprise:
detecting that the physical device has performed the command on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
9. The augmented-reality device of claim 2 , wherein the operations further comprise:
receiving a confirmation from the physical device that the command has been performed on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
10. The augmented-reality device of claim 1 , wherein the operations further comprise:
detecting that a first and a second physical object are located within the predefined distance of the physical device, the first and second physical objects being electronically unconnected to the physical device;
determining that the first and second physical objects are associated with the physical device;
identifying a first request based on a combination of an identification of the first and second physical objects, the position of the first physical object relative to the physical device, the position of the second physical object relative to the physical device, the orientation of the first physical object relative to the augmented-reality device, and the orientation of the of the second physical object relative to the augmented-reality device; and
communicating the first request to the physical device.
11. A method comprising:
detecting that a physical device is located within a first threshold distance of the augmented-reality device;
detecting that a physical object is located within a second threshold distance of the physical device, the physical object being electronically unconnected to the physical device; and
displaying, in the display, an identification of a command to be performed by the physical device based on a presence of the physical device and the physical object, in response to detecting that the physical device is located within the first threshold distance of the augmented-reality device, and that the physical object is located within the second threshold distance of the physical device.
12. The method of claim 11 , wherein displaying the identification of the command further comprises: displaying virtual content as an overlay to the physical object, the virtual content identifying the command.
13. The method of claim 11 , further comprising:
determining that the physical object is associated with the physical device; and
identifying the command based on an identification of both the physical object and the physical device, the physical device configured to operate the command.
14. The method of claim 11 , further comprising:
determining a position of the physical object relative to the physical device, a first orientation of the physical object relative to the augmented-reality device, and a second orientation of the physical object relative to the physical device,
wherein the command is identified based on a combination of the identification of the physical object, the position of the physical object relative to the physical device, the first orientation of the physical object relative to the augmented-reality device, and the second orientation of the physical object relative to the physical device.
15. The method of claim 11 , further comprising:
identifying the physical device and the physical object; and
determining that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
16. The method of claim 11 , further comprising:
wirelessly communicating with the physical device;
identifying the physical device based on the wireless communication between the augmented reality device and the physical device; and
determining that the physical object is adjacent to the physical device,
wherein the command is based on the identification of both the physical device and the physical object, and the physical object being adjacent to the physical device.
17. The method of claim 12 , further comprising:
detecting that the physical device has performed the command on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
18. The method of claim 12 , further comprising:
receiving a confirmation from the physical device that the command has been performed on the physical device; and
modifying the virtual content in response to the detecting that the physical device has performed the command, the modified virtual content indicating that the physical device has performed the command.
19. The method of claim 11 , further comprising:
detecting that a first and a second physical object are located within the predefined distance of the physical device, the first and second physical objects being electronically unconnected to the physical device;
determining that the first and second physical objects are associated with the physical device;
identifying a first request based on a combination of an identification of the first and second physical objects, the position of the first physical object relative to the physical device, the position of the second physical object relative to the physical device, the orientation of the first physical object relative to the augmented-reality device, and the orientation of the of the second physical object relative to the augmented-reality device; and
communicating the first request to the physical device.
20. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
detecting that a physical device is located within a first threshold distance of the augmented-reality device;
detecting that a physical object is located within a second threshold distance of the physical device, the physical object being electronically unconnected to the physical device; and
displaying, in the display, an identification of a command to be performed by the physical device based on a presence of the physical device and the physical object, in response to detecting that the physical device is located within the first threshold distance of the augmented-reality device, and that the physical object is located within the second threshold distance of the physical device.