Touch interaction in augmented and virtual reality applications
Disclosed herein are system, method, and computer program product embodiments for providing accessible menus in 3D visualization contexts, such as augmented reality and virtual reality environments. In one aspect, a menu in the shape of a circle segment may be presented within a corner of a 3D visualization, allowing a user to easily access icons present within that menu using a touch interface. The user may rotate the menu in order to access additional icons.
1. A computer implemented method, comprising:
selecting, by one or more computing devices, an object of a 3D scene responsive to the object being centered within an area of interest of a 3D visualization, wherein the 3D visualization comprises a perspective of a virtual camera within the 3D scene;
rendering, by the one or more computing devices, a 2D overlay menu over a portion of the 3D visualization responsive to the selecting of the object, wherein the 2D overlay menu is depicted as a section of a circle comprising a plurality of icons configured to perform respective tasks, and wherein at least a first of the plurality of icons is configured to perform its context-specific respective task on the object;
receiving, by the one or more computing devices, a translation interaction within the portion of the 3D visualization overlaid by the 2D overlay menu; and
modifying, by the one or more computing devices, the position of the plurality of icons within the 2D overlay menu responsive to the translation interaction.
2. The computer implemented method of claim 1 , wherein the 3D visualization is rendered as an augmented reality (AR) visualization on a handheld display, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch screen region of the handheld display.
3. The computer implemented method of claim 2 , wherein rendering the 2D overlay menu over the portion of the 3D visualization responsive to the selecting of the object comprises:
rendering, by the one or more computing devices, the 2D overlay menu such that the portion of the 3D visualization includes a position of a touch interaction within the touch screen region of the handheld display.
4. The computer implemented method of claim 1 , wherein the 3D visualization is presented as a virtual reality (VR) visualization on a VR display device, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch pad region of a 3D VR input controller.
5. The computer implemented method of claim 1 , wherein modifying the position of the plurality of icons comprises:
rotating, by the one or more computing devices, the plurality of icons along an arc of the circle of the overlay menu, such that a first of the plurality of icons exits a visible region of the 2D overlay menu at a first end of the section of the circle, and a second of the plurality of icons enters the visible region of the 2D overlay menu at a second end of the section of the circle.
6. The computer implemented method of claim 1 , wherein at least a second of the plurality of icons is configured to perform its respective task on the 3D visualization.
7. The computer implemented method of claim 1 , wherein the context-specific respective task is configured to change an appearance of the object within the 3D scene, the method further comprising:
rendering, by the one or more computing devices, the 3D visualization with the object having its appearance changed responsive to performance of the context-specific respective task on the object.
8. A system, comprising:
a memory configured to store operations; and
one or more processors configured to perform the operations, the operations comprising:
selecting an object of a 3D scene responsive to the object being centered within an area of interest of a 3D visualization, wherein the 3D visualization comprises a perspective of a virtual camera within the 3D scene;
rendering a 2D overlay menu over a portion of the 3D visualization responsive to the selecting of the object, wherein the 2D overlay menu is depicted as a section of a circle comprising a plurality of icons configured to perform respective tasks, and wherein at least a first of the plurality of icons is configured to perform its context-specific respective task on the object,
receiving a translation interaction within the portion of the 3D visualization overlaid by the 2D overlay menu, and
modifying the position of the plurality of icons within the 2D overlay menu responsive to the translation interaction.
9. The system of claim 8 , wherein the 3D visualization is rendered as an augmented reality (AR) visualization on a handheld display, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch screen region of the handheld display.
10. The system of claim 9 , wherein rendering the 2D overlay menu over the portion of the 3D visualization responsive to the selecting of the object comprises:
rendering the 2D overlay menu such that the portion of the 3D visualization includes a position of a touch interaction within the touch screen region of the handheld display.
11. The system of claim 8 , wherein the 3D visualization is presented as a virtual reality (VR) visualization on a VR display device, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch pad region of a 3D VR input controller.
12. The system of claim 8 , wherein modifying the position of the plurality of icons comprises:
rotating the plurality of icons along an arc of the circle of the overlay menu, such that a first of the plurality of icons exits a visible region of the 2D overlay menu at a first end of the section of the circle, and a second of the plurality of icons enters the visible region of the 2D overlay menu at a second end of the section of the circle.
13. The system of claim 8 , wherein at least a second of the plurality of icons is configured to perform its respective task on the 3D visualization.
14. The system of claim 8 , wherein the context-specific respective task is configured to change an appearance of the object within the 3D scene, the operations further comprising:
rendering the 3D visualization with the object having its appearance changed responsive to performance of the context-specific respective task on the object.
15. A non-transitory computer readable storage device having instructions stored thereon, execution of which, by one or more processing devices, causes the one or more processing devices to perform operations comprising:
selecting an object of a 3D scene responsive to the object being centered within an area of interest of a 3D visualization, wherein the 3D visualization comprises a perspective of a virtual camera within the 3D scene;
rendering a 2D overlay menu over a portion of the 3D visualization responsive to the selecting of the object, wherein the 2D overlay menu is depicted as a section of a circle comprising a plurality of icons configured to perform respective tasks, and wherein at least a first of the plurality of icons is configured to perform its context-specific task on the object;
receiving a translation interaction within the portion of the 3D visualization overlaid by the 2D overlay menu; and
modifying the position of the plurality of icons within the 2D overlay menu responsive to the translation interaction.
16. The non-transitory computer readable storage device of claim 15 , wherein the 3D visualization is rendered as an augmented reality (AR) visualization on a handheld display, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch screen region of the handheld display.
17. The non-transitory computer readable storage device of claim 16 , wherein rendering the 2D overlay menu over the portion of the 3D visualization responsive to the selecting of the object comprises:
rendering the 2D overlay menu such that the portion of the 3D visualization includes a position of a touch interaction within the touch screen region of the handheld display.
18. The non-transitory computer readable storage device of claim 15 , wherein the 3D visualization is presented as a virtual reality (VR) visualization on a VR display device, and wherein the portion of the 3D visualization overlaid by the 2D overlay menu comprises a touch pad region of a 3D VR input controller.
19. The non-transitory computer readable storage device of claim 15 , wherein modifying the position of the plurality of icons comprises:
rotating the plurality of icons along an arc of the circle of the overlay menu, such that a first of the plurality of icons exits a visible region of the 2D overlay menu at a first end of the section of the circle, and a second of the plurality of icons enters the visible region of the 2D overlay menu at a second end of the section of the circle.
20. The non-transitory computer-readable storage device of claim 15 , wherein the context-specific respective task is configured to change an appearance of the object within the 3D scene, the operations further comprising:
rendering the 3D visualization with the object having its appearance changed responsive to performance of the context-specific respective task on the object.