Manipulation of virtual object in augmented reality via intent
A system and method for manipulating a virtual object based on intent is described. A reference identifier from a physical object is captured. The reference identifier is communicated via a network to a remote server. The remote server includes virtual object data associated with the reference identifier. The virtual object data is received at the computing device. The virtual image is displayed in a virtual landscape using the virtual object data. In response to relative movement between the computing device and the physical object caused by a user, the virtual image is modified. Brain activity data of the user is received. A state of the virtual object in the virtual landscape is changed based on the brain activity data.
1. A device comprising:
a first camera configured to generate an image of a physical object;
a second camera configured to track eye movements of a user of the device;
a display; and
an augmented reality application implemented with a hardware processor of the device, the augmented reality application configured to:
identify a three-dimensional virtual object based on the image of the physical object,
render the three-dimensional virtual object in the display,
determine a gaze of the user of the device based on the eye movements of the user of the device,
identify a location on the three-dimensional virtual object based on the gaze of the user of the device,
identify an interactive feature of the three-dimensional virtual object, the interactive feature corresponding to the location on the three-dimensional virtual object;
determine that the interactive feature is located within a predefined portion of the display,
receive brain activity data of the user in response to the gaze of the user on the identified location on the three-dimensional virtual object, and
change a state of the interactive feature of the three-dimensional virtual object at the identified location on the three-dimensional virtual object by replacing a first component of the interactive feature of the three-dimensional virtual object in the predefined portion of the display with a second component of interactive feature of the three-dimensional virtual object in the predefined portion of the display based on the brain activity data and in response to determining that the interactive feature is located within the predefined portion of the display.
2. The device of claim 1 , further comprising:
a storage device including a database configured to store visual references, virtual objects that correspond to the visual references, and features of the virtual objects, the features of the virtual objects being configured to change components in response to the brain activity data of the user.
3. The device of claim 1 , wherein the predefined portion of the display includes at least one of a central area of the display, a perimeter area adjacent to an edge of the display, a corner area adjacent to a corner of the display, and a user-defined portion of the display.
4. The device of claim 1 , wherein the first component of the interactive feature of the three-dimensional virtual object is replaced with the second component of the interactive feature of the three-dimensional virtual object in response to the interactive feature being located within the predefined portion of the display for a time duration exceeding a time threshold.
5. The device of claim 1 , wherein the first component of the interactive feature of the three-dimensional virtual object is replaced with the second component of the interactive feature of the three-dimensional virtual object in response to the brain activity data exceeding a threshold for a time duration exceeding a time threshold.
6. A method comprising:
generating an image of a physical object with a first camera of a device;
tracking eye movements of a user of the device with a second camera of the device;
identifying a three-dimensional virtual object based on the image of the physical object;
rendering, using a hardware processor of the device, the three-dimensional virtual object in a display of the device;
determining a gaze of the user of the device based on the eye movements of the user of the device,
identifying a location on the three-dimensional virtual object based on the gaze of the user of the device;
identifying an interactive feature of the three-dimensional virtual object corresponding to the location on the three-dimensional virtual object;
determining that the interactive feature is located within a predefined portion of the display;
receiving brain activity data of the user in response to the gaze of the user on the identified location on the three-dimensional virtual object; and
changing a state of the interactive feature of the three-dimensional virtual object at the identified location on the three-dimensional virtual object by replacing a first component of the interactive feature of the three-dimensional virtual object in the predefined portion of the display with a second component of interactive feature of the three-dimensional virtual object in the predefined portion of the display based on the brain activity data and in response to determining that the interactive feature is located within the predefined portion of the display.
7. The method of claim 6 , further comprising:
storing visual references, virtual objects that correspond to the visual references, and features of the virtual objects in a database, the features of the virtual objects being configured to change components in response to the brain activity data of the user.
8. The method of claim 6 , wherein the predefined portion of the display includes at least one of a central area of the display, a perimeter area adjacent to an edge of the display, a corner area adjacent to a corner of the display, and a user-defined portion of the display.
9. The method of claim 6 , wherein replacing the first component of the interactive feature of the three-dimensional virtual object in the predefined portion of the display with the second component of the three-dimensional virtual object is in response to the interactive feature being located within the predefined portion of the display for a time duration exceeding a time threshold.
10. The method of claim 6 , replacing the first component of the interactive feature of the three-dimensional virtual object in the predefined portion of the display with the second component of the three-dimensional virtual object is in response to the brain activity data exceeding a threshold for a time duration exceeding a time threshold.
11. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
generating an image of a physical object with a first camera of a device;
tracking eye movements of a user of the device with a second camera of the device;
identifying a three-dimensional virtual object based on the image of the physical object;
rendering, using a hardware processor of the device, the three-dimensional virtual object in a display of the device;
determining a gaze of the user of the device based on the eye movements of the user of the device,
identifying a location on the three-dimensional virtual object based on the gaze of the user of the device;
identifying an interactive feature of the three-dimensional virtual object corresponding to the location on the three-dimensional virtual object;
determining that the interactive feature is located within a predefined portion of the display;
receiving brain activity data of the user in response to the gaze of the user on the identified location on the three-dimensional virtual object; and
changing a state of the interactive feature of the three-dimensional virtual object at the identified location on the three-dimensional virtual object by replacing a first component of the interactive feature of the three-dimensional virtual object in the predefined portion of the display with a second component of interactive feature of the three-dimensional virtual object in the predefined portion of the display based on the brain activity data and in response to determining that the interactive feature is located within the predefined portion of the display.