Snapping, virtual inking, and accessibility in augmented reality
Embodiments of the present disclosure relate to providing an augmented reality experience. Embodiments include receiving, from an augmented reality or virtual reality (AR/VR) device, first input that identifies application content and a location of an artifact in an AR/VR environment. Embodiments include identifying a region in the environment based on the location, the region corresponding to the artifact. Embodiments include receiving the application content from an application. Embodiments include composing a first scene, the application content being overlaid onto the region. Embodiments include transporting the first scene to the AR/VR device for display. Embodiments include receiving, from the client device, second input in the region in the environment. Embodiments include composing a second scene wherein the application content is modified based on the second input. Embodiments include transporting the second scene to the AR/VR device for display.
1. A method for providing an augmented reality experience, comprising:
receiving, by a computing device from an augmented reality or virtual reality (AR/VR) device, first input that identifies application content and a location of an artifact in an AR/VR environment;
identifying, by the computing device, a primary display region in the environment based on the location, wherein the primary display region corresponds to the artifact;
receiving, by the computing device, the application content from an application, wherein the application content comprises a first display screen of a virtual desktop;
determining that a policy indicates to augment, based on the first input, an additional region that is adjacent to the primary display region with second application content comprising a second display screen of the virtual desktop such that the application content and the second application content simulate a multiple monitor setup for the virtual desktop within the environment;
composing, by the computing device, a first scene wherein:
the application content is overlaid onto the primary display region; and
the second application content is overlaid onto the additional region;
transporting, by the computing device, the first scene to the AR/VR device, wherein the AR/VR device displays the first scene;
receiving, by the computing device from the AR/VR device, second input in the primary display region in the environment;
composing, by the computing device, a second scene wherein the application content is modified based on the second input; and
transporting, by the computing device, the second scene to the AR/VR device, wherein the AR/VR device displays the second scene.
2. The method of claim 1 , wherein composing, by the computing device, the first scene comprises augmenting the primary display region in the first scene with the application content based on default augmentation configuration values.
3. The method of claim 1 , wherein the application content comprises at least a portion of a remote desktop screen, and wherein the first input comprises a drag-and-drop action moving the portion of the remote desktop screen onto the location in the environment.
4. The method of claim 1 , the second input comprises touch input, and wherein the second scene comprises a visual representation of the touch input.
5. The method of claim 4 , wherein composing, by the computing device, the second scene comprises:
providing the touch input to the application; and
receiving modified application content from the application, wherein the modified application content comprises the visual representation of the touch input.
6. The method of claim 1 , wherein the second input comprises a selection of an accessibility feature, wherein the second scene comprises the accessibility feature, and wherein the accessibility feature comprises one of:
magnification of a portion of the application content;
narration of a portion of the application content; or
a color change to the application content.
7. A non-transitory computer readable medium comprising instructions to be executed in a computer system, wherein the instructions when executed in the computer system perform a method for providing an augmented reality experience, the method comprising:
receiving, by a computing device from an augmented reality or virtual reality (AR/VR) device, first input that identifies application content and a location of an artifact in an AR/VR environment;
identifying, by the computing device, a primary display region in the environment based on the location, wherein the primary display region corresponds to the artifact;
receiving, by the computing device, the application content from an application, wherein the application content comprises a first display screen of a virtual desktop;
determining that a policy indicates to augment, based on the first input, an additional region that is adjacent to the primary display region with second application content comprising a second display screen of virtual desktop such that the application content and the second application content simulate a multiple monitor setup for the virtual desktop within the environment;
composing, by the computing device, a first scene wherein:
the application content is overlaid onto the primary display region; and
the second application content is overlaid onto the additional region;
transporting, by the computing device, the first scene to the AR/VR device, wherein the AR/VR device displays the first scene;
receiving, by the computing device from the AR/VR device, second input in the primary display region in the environment;
composing, by the computing device, a second scene wherein the application content is modified based on the second input; and
transporting, by the computing device, the second scene to the AR/VR device, wherein the AR/VR device displays the second scene.
8. The non-transitory computer readable medium of claim 7 , wherein composing, by the computing device, the first scene comprises augmenting the primary display region in the first scene with the application content based on default augmentation configuration values.
9. The non-transitory computer readable medium of claim 7 , wherein the application content comprises at least a portion of a remote desktop screen, and wherein the first input comprises a drag-and-drop action moving the portion of the remote desktop screen onto the location in the environment.
10. The non-transitory computer readable medium of claim 7 , wherein the second input comprises touch input, and wherein the second scene comprises a visual representation of the touch input.
11. The non-transitory computer readable medium of claim 10 , wherein composing, by the computing device, the second scene comprises:
providing the touch input to the application; and
receiving modified application content from the application, wherein the modified application content comprises the visual representation of the touch input.
12. The non-transitory computer readable medium of claim 7 , wherein the second input comprises a selection of an accessibility feature, wherein the second scene comprises the accessibility feature, and wherein the accessibility feature comprises one of:
magnification of a portion of the application content;
narration of a portion of the application content; or
a color change to the application content.
13. A computer system, wherein system software for the computer system is programmed to execute a method for providing an augmented reality experience, the method comprising:
receiving, by a computing device from an augmented reality or virtual reality (AR/VR) device, first input that identifies application content and a location of an artifact in an AR/VR environment;
identifying, by the computing device, a primary display region in the environment based on the location, wherein the primary display region corresponds to the artifact;
receiving, by the computing device, the application content from an application, wherein the application content comprises a first display screen of a virtual desktop;
determining that a policy indicates to augment, based on the first input, an additional region that is adjacent to the primary display region with second application content comprising a second display screen of the virtual desktop such that the application content and the second application content simulate a multiple monitor setup for the virtual desktop within the environment;
composing, by the computing device, a first scene wherein:
the application content is overlaid onto the primary display region; and
the second application content is overlaid onto the additional region;
transporting, by the computing device, the first scene to the AR/VR device, wherein the AR/VR device displays the first scene;
receiving, by the computing device from the AR/VR device, second input in the primary display region in the environment;
composing, by the computing device, a second scene wherein the application content is modified based on the second input; and
transporting, by the computing device, the second scene to the AR/VR device, wherein the AR/VR device displays the second scene.
14. The computer system of claim 13 , wherein composing, by the computing device, the first scene comprises augmenting the primary display region in the first scene with the application content based on default augmentation configuration values.
15. The computer system of claim 13 , wherein the application content comprises at least a portion of a remote desktop screen, and wherein the first input comprises a drag-and-drop action moving the portion of the remote desktop screen onto the location in the environment.
16. The computer system of claim 13 , wherein the second input comprises touch input, and wherein the second scene comprises a visual representation of the touch input.
17. The computer system of claim 16 , wherein composing, by the computing device, the second scene comprises:
providing the touch input to the application; and
receiving modified application content from the application, wherein the modified application content comprises the visual representation of the touch input.