Converting a 2D positional input into a 3D point in space
A user may interact and select positions in three-dimensional space arbitrarily through conversion of a two-dimensional positional input into a three-dimensional point in space. The system may allow a user to use one or more user input devices for pointing, annotating, or drawing on virtual objects, real objects or empty space in reference to the location of the three-dimensional point in space within an augmented reality or mixed reality session.
1. A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
determining a head pose of a user associated with a wearable display device;
determining a cursor screen position at a first distance from the wearable display device;
determining a cursor plane position at a second distance from the wearable display device;
determining a two-dimensional position on the cursor screen;
determining a one-dimensional position defined by at least one of the cursor plane or a mesh, wherein the one-dimensional position is further defined by a first intersection of a ray cast orthogonal from a head pose origin associated with a position and a rotation; and
displaying a virtual object to the user at a three-dimensional position comprised of the two-dimensional position and the one-dimensional position.
2. The computerized method of claim 1 , wherein the head pose of the user determines a view frustum.
3. The computerized method of claim 2 , wherein the cursor screen is positioned closer to the wearable display device than to the cursor plane.
4. The computerized method of claim 3 , wherein the cursor screen bisects the view frustum.
5. The computerized method of claim 1 , wherein the cursor plane is configured to have a maximum and minimum distance from the user.
6. The computerized method of claim 1 , wherein the two-dimensional position on the cursor screen is a default two-dimensional position.
7. The computerized method of claim 6 , wherein the default two-dimensional position is a center of the cursor screen.
8. The computerized method of claim 1 , wherein the first intersection is with at least one of the cursor plane or the mesh.
9. The computerized method of claim 8 , wherein the mesh comprises one or more real world, virtual, or mixed objects.
10. The computerized method of claim 1 , wherein the virtual object is a cursor.
11. The computerized method of claim 1 , wherein the virtual object is a point or a line.
12. A system for converting a two-dimensional positional input into a three-dimensional position for visualizing augmented reality, the system comprising:
a wearable headset;
one or more non-transitory computer readable storage mediums having program instructions embodied therewith; and
one or more processors configured to execute the program instructions to cause the system to:
determine a three-dimensional position within an augmented reality environment that includes one or more virtual objects and one or more real-world objects;
determine a two-dimensional position on a cursor screen based on the first two-dimensional offset and the second two-dimensional offset;
determine a one-dimensional position on a cursor plane or a mesh associated with one of the virtual objects or real-world objects;
identify an intersection of the determined one-dimensional position within the augmented reality environment where an axis orthogonal from a head pose origin associated with a position and a rotation, through the determined two-dimensional position on the cursor screen, and intersecting the cursor plane or the mesh associated with one of the virtual objects or real-world objects;
determining a head movement compensation based on a change in at least one of an orientation and a position from a previous head pose and the head pose, wherein the determined head movement compensation is a two-dimensional offset;
determine an updated three-dimensional position within the augmented reality environment; and
display a virtual object at the updated three-dimensional position within the augmented reality environment to a user via the wearable headset, wherein the three-dimensional position is back projected on to the two-dimensional cursor screen by using the determined head movement compensation offset.
13. The system of claim 12 , wherein at least one of the cursor plane or the mesh is fixed at the updated three-dimensional position.
14. The system of claim 13 , further comprising displaying annotations at the updated three-dimensional position based on user interactions with the input device.
15. The system of claim 13 , wherein the user draws on at least one of the cursor plane or the mesh.
16. A system for converting a two-dimensional positional input into a three-dimensional position for visualizing augmented reality, the system comprising:
a wearable headset:
one or more non-transitory computer readable storage mediums having program instructions embodied therewith; and
one or more processors configured to execute the program instructions to cause the system to:
determine a three-dimensional position within an augmented reality environment that includes one or more virtual objects and one or more real-world objects;
determine a two-dimensional position on a cursor screen based on the first two-dimensional offset and the second two-dimensional offset;
determine a one-dimensional position on a cursor plane or a mesh associated with one of the virtual objects or real-world objects;
identify an intersection of the determined one-dimensional position within the augmented reality environment where an axis orthogonal from a head pose origin associated with a position and a rotation, through the determined two-dimensional position on the cursor screen, and intersecting the cursor plane or the mesh associated with one of the virtual objects or real-world objects;
determine an input device movement based on a change in a previous two-dimensional position and the two-dimensional position, based at least on a detected interaction between a contactor operated by the user and a touch sensitive surface of an input device;
determine an updated three-dimensional position within the augmented reality environment; and
display a virtual object at the updated three-dimensional position within the augmented reality environment to a user via the wearable headset.
17. The system of claim 16 , wherein the one or more processors is configured to execute the program instructions to further cause the system to determine a head movement compensation based on a change in at least one of an orientation and a position from a previous head pose and the head pose.
18. The system of claim 17 , wherein the determined head movement compensation is a two-dimensional offset.
19. The system of claim 18 , wherein the three-dimensional position is back projected on to the two-dimensional cursor screen by using the determined head movement compensation offset.
20. A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
determining a head pose of a user associated with a wearable display device, wherein the head pose of the user determines a view frustum;
determining a cursor screen position at a first distance from the wearable display device;
determining a cursor plane position at a second distance from the wearable display device;
determining a two-dimensional position on the cursor screen;
determining a one-dimensional position defined by at least one of the cursor plane or a mesh, wherein the cursor screen is positioned closer to the wearable display device than to the cursor plane and the cursor screen bisects the view frustum; and
displaying a virtual object to the user at a three-dimensional position comprised of the two-dimensional position and the one-dimensional position.
21. The computerized method of claim 20 , wherein the one-dimensional position is further defined by a first intersection of a ray cast orthogonal from a head pose origin associated with a position and a rotation.
22. The computerized method of claim 20 , wherein the cursor plane is configured to have a maximum and minimum distance from the user.
23. The computerized method of claim 20 , wherein the two-dimensional position on the cursor screen is a default two-dimensional position, and wherein the default two-dimensional position is a center of the cursor screen.
24. The computerized method of claim 20 , further comprising:
determining a head movement compensation based on a change in at least one of an orientation and a position from a previous head pose and the head pose, wherein the determined head movement compensation is a two-dimensional offset.
25. The computerized method of claim 24 , wherein the three-dimensional position is back projected on to the two-dimensional cursor screen by using the determined head movement compensation offset.
26. The computerized method of claim 20 , further comprising:
determining an input device movement based on a change in a previous two-dimensional position and the two-dimensional position, based at least on a detected interaction between a contactor operated by the user and a touch sensitive surface of an input device.