Converting a 2D positional input into a 3D point in space
A user may interact and select positions in three-dimensional space arbitrarily through conversion of a two-dimensional positional input into a three-dimensional point in space. The system may allow a user to use one or more user input devices for pointing, annotating, or drawing on virtual objects, real objects or empty space in reference to the location of the three-dimensional point in space within an augmented reality or mixed reality session.
1. A system for converting a two-dimensional positional input into a three-dimensional position for visualizing augmented reality, the system comprising:
a wearable headset;
one or more non-transitory computer readable storage mediums having program instructions embodied therewith; and
one or more processors configured to execute the program instructions to cause the system to:
determine a three-dimensional position within an augmented reality environment that includes one or more virtual objects and one or more real-world objects;
determine a head movement compensation based on a change in at least one of an orientation and a position from a previous head pose and a current head pose as a first two-dimensional offset;
determine an input device compensation based on a further input to the cursor screen of the input device as a second two-dimensional offset;
determine a two-dimensional position on a cursor screen based on the first two-dimensional offset and the second two-dimensional offset;
determine a one-dimensional position on a cursor plane or a mesh associated with one of the virtual objects or real-world objects;
identify an intersection of the determined one-dimensional position within the augmented reality environment where an axis orthogonal from a head pose origin associated with a position and a rotation, through the determined two-dimensional position on the cursor screen, and intersecting the cursor plane or the mesh associated with one of the virtual objects or real-world objects;
determine an updated three-dimensional position within the augmented reality environment; and
display a virtual object at the updated three-dimensional position within the augmented reality environment to a user via the wearable headset.
2. The system of claim 1 , wherein the system further comprises the input device, and wherein the input device is a two-dimensional writable panel.
3. The system of claim 2 , wherein the two-dimensional writable panel is at least one of a tablet, smartphone, or pen display.
4. The system of claim 3 , wherein the two-dimensional writable panel comprises a Bluetooth radio module, Electromagnetic (EM) antenna and EM transmitter, wherein the two-dimensional writable panel communicates to the wearable display device by Bluetooth and wherein the two-dimensional writable panel communicates with a contactor by at least one of Bluetooth or Electromagnetic wave.
5. The system of claim 4 , wherein the contactor is at least one of a stylus, pen or electronic pen.
6. The system of claim 1 , wherein at least one of the cursor plane or the mesh is fixed at the updated three-dimensional position.
7. The system of claim 1 , wherein the instructions further cause the system to display annotations at the updated three-dimensional position based on user interactions with the input device.
8. The system of claim 1 , wherein the user draws on at least one of the cursor plane or the mesh.
9. The system of claim 1 , wherein the program instructions further cause the system to receive a range status associated with contact with the input device.
10. The system of claim 9 , wherein the range state indicates whether a contactor is within a predetermined distance of the input device.
11. The system of claim 10 , wherein the contactor is operated by the user.
12. The system of claim 10 , wherein the predetermined distance indicates a distance at which input is received.
13. The system of claim 1 , wherein based on the contactor being within the predetermined distance and not applying pressure to the input device, the system is configured to receive input.
14. The system of claim 1 , wherein the program instructions further cause the system to receive a pressure status indicating a pressure associated with contact with the input device.
15. The system of claim 1 , wherein the cursor plane is configured to be moved by the user.
16. The system of claim 1 , wherein the cursor plane or the mesh includes at least one of a visible grid or pattern to indicate location in space.