Spatially-aware projection pen interface
One embodiment of the present invention sets forth a technique for providing an end user with a digital pen embedded with a spatially-aware miniature projector for use in a design environment. Paper documents are augmented to allow a user to access additional information and computational tools through projected interfaces. Virtual ink may be managed in single and multi-user environments to enhance collaboration and data management. The spatially-aware projector pen provides end-users with dynamic visual feedback and improved interaction capabilities.
1. A computer-implemented method for configuring a spatially-aware projection pen to provide a user interface, the method comprising:
receiving a position of the spatially-aware projection pen in three-dimensional space from a position tracking mechanism, wherein the three-dimensional space is partitioned into a surface input layer that includes a display surface, a hover input layer that is directly above the display surface, and a spatial input layer that is directly above the hover input layer;
receiving a signal indicating that a user interface menu has been activated through an input mechanism of the spatially-aware projection pen;
outputting the user interface menu as a projected image that is displayed on the display surface by a projector within the spatially-aware projection pen, wherein the user interface menu comprises a radial menu that includes a plurality of regions separated by boundaries, wherein each of the regions represents a different menu item included in the radial menu;
locking the user interface menu to a specific position on the display surface;
determining that a menu item specified by the user interface menu has been selected, wherein determining comprises the spatially-aware projection pen determining that a boundary delineating a first region of the user interface menu has been crossed by a virtual cursor to select a first menu item associated with the first region, wherein the virtual cursor has a position that is controlled by movements of the spatially-aware projection pen;
deactivating the function for displaying the user interface menu when the spatially-aware projection pen is lifted above the surface input layer; and
configuring the spatially-aware projection pen for an operation based on the selected menu item.
2. The method of claim 1 , further comprising, prior to the step of determining, the step of locking the user interface menu to the position of the spatially-aware projection pen in the three-dimensional space.
3. The method of claim 2 , wherein the step of determining comprises locking a virtual cursor to the position of the spatially-aware projection pen in the three-dimensional space and activating the input mechanism to lock the user interface menu to a position on the display surface.
4. The method of claim 1 , wherein the boundary delineating the first region of the user interface menu is crossed in the hover input layer.
5. The method of claim 1 , wherein the boundary delineating the first region of the user interface menu is crossed in the surface input layer.
6. The method of claim 1 , further comprising repeating the steps of outputting and determining for each level of hierarchy of the selected menu item.
7. A non-transitory computer-readable medium including instructions that, when executed by a processing unit within a spatially-aware projection pen, cause the spatially-aware projection pen to provide a user interface, by performing the steps of:
receiving a position of the spatially-aware projection pen in three-dimensional space from a position tracking mechanism, wherein the three-dimensional space is partitioned into a surface input layer that includes a display surface, a hover input layer that is directly above the display surface, and a spatial input layer that is directly above the hover input layer;
receiving a signal indicating that a user interface menu has been activated through an input mechanism of the spatially-aware projection pen;
outputting the user interface menu as a projected image that is displayed on the display surface by a projector within the spatially-aware projection pen, wherein the user interface menu comprises a radial menu that includes a plurality of regions separated by boundaries, wherein each of the regions represents a different menu item included in the radial menu;
locking the user interface menu to a specific position on the display surface;
determining that a menu item specified by the user interface menu has been selected, wherein determining comprises the spatially-aware projection pen determining that a boundary delineating a first region of the user interface menu has been crossed by a virtual cursor to select a first menu item associated with the first region, wherein the virtual cursor has a position that is controlled by movements of the spatially-aware projection pen;
deactivating the function for displaying the user interface menu when the spatially-aware projection pen is lifted above the surface input layer; and
configuring the spatially-aware projection pen for an operation based on the selected menu item.
8. The non-transitory computer-readable medium of claim 7 , further comprising, prior to the step of determining, the step of locking the user interface menu to the position of the spatially-aware projection pen in the three-dimensional space.
9. The non-transitory computer-readable medium of claim 8 , wherein the step of determining comprises locking a virtual cursor to the position of the spatially-aware projection pen in the three-dimensional space and activating an input mechanism to lock the user interface menu to a position on the display surface.
10. The non-transitory computer-readable medium of claim 7 , wherein the boundary delineating the first region of the user interface menu is crossed in the hover input layer.
11. The non-transitory computer-readable medium of claim 7 , wherein the boundary delineating the first region of the user interface menu is crossed in the surface input layer.
12. The non-transitory computer-readable medium of claim 7 , further comprising repeating the steps of outputting and determining for each level of hierarchy of the selected menu item.
13. A spatially-aware projection pen configured to provide a user interface, comprising:
an enclosure that is configured to be grasped by a human hand and housing an input mechanism, a position tracking mechanism, a projector, and a processor,
wherein the processor is configured to:
receive an input signal indicating that display of a user interface menu is activated through the input mechanism of the spatially-aware projection pen,
receive a position of the spatially-aware projection pen in three-dimensional space from the position tracking mechanism, wherein the three-dimensional space is partitioned into a surface input layer that includes a display surface, a hover input layer that is directly above the display surface, and a spatial input layer that is directly above the hover input layer,
output the user interface menu to a projected image that is displayed on the display surface by the projector within the spatially-aware projection pen, wherein the user interface menu comprises a radial menu that includes a plurality of regions separated by boundaries, wherein each of the regions represents a different menu item included in the radial menu,
lock the user interface menu to a specific position on the display surface,
determine a menu item specified by the user interface menu is selected, wherein determining comprises the spatially-aware projection pen determining that a boundary delineating a first region of the user interface menu has been crossed by a virtual cursor to select a first menu item associated with the first region, wherein the virtual cursor has a position that is controlled by movements of the spatially-aware projection pen,
deactivate the function for displaying the user interface menu when the spatially-aware projection pen is listed above the surface input layer, and
configure the spatially-aware projection pen for an operation based on the selected menu item.
14. The method of claim 1 , wherein the boundary delineating the first region of the user interface menu is crossed in the hover spatial input layer.
15. The non-transitory computer-readable medium of claim 7 , wherein the boundary delineating the first region of the user interface menu is crossed in the hover input layer.
16. The method of claim 1 , wherein outputting the user interface menu as a projected image on the display surface further comprises searching for an element on the display surface.
17. The non-transitory computer-readable medium of claim 7 , wherein outputting the user interface menu as a projected image on the display surface further comprises searching for an element on the display surface.