Pointer projection for natural user input

1. In a compute system operatively coupled to a machine vision system and to a display, a method comprising:

receiving, from the machine vision system, prior and subsequent video imaging a head and pointer of a user;

computing, based on the prior video, a prior geometric line of sight of the user partly occluded by the pointer;

assigning spatial coordinates to one or more non-display objects situated along the prior geometric line of sight, the spatial coordinates defining a position of each of the non-display objects within a field of view of the machine vision system;

computing, based on the subsequent video, a subsequent geometric line of sight of the user partly occluded by the pointer;

identifying, by reference to the spatial coordinates of the one or more non-display objects, a targeted display object situated on the display and along the subsequent geometric line of sight;

identifying in the subsequent video movement of the pointer along the subsequent geometric line of sight;

and executing code for a process associated with the targeted display object in response to the movement.

2. The method of claim 1 , further comprising presenting the targeted display object on the display.

3. The method of claim 1 , wherein computing the subsequent geometric line of sight includes computing spatial coordinates of the head from the subsequent video.

4. The method of claim 1 , wherein computing the subsequent geometric line of sight includes computing spatial coordinates of an eye or pupil of the user from the subsequent video.

5. The method of claim 1 , wherein computing the subsequent geometric line of sight includes

computing spatial coordinates of the pointer from the subsequent video,

and wherein the subsequent geometric line of sight is a straight line passing through the pointer and through the eye of the user.

6. The method of claim 1 , wherein identifying the targeted display object includes determining whether the targeted display object is situated along the subsequent geometric line of sight.

7. The method of claim 1 , wherein the pointer includes a finger of the user.

8. The method of claim 1 , wherein the machine vision system includes a depth camera, and wherein each of the prior and subsequent video includes a series of time-resolved depth images from the depth camera.

9. The method of claim 1 , wherein the machine vision system includes a flat-image camera, and wherein each of the prior and subsequent video includes a series of time-resolved images from the flat-image camera.

10. The method of claim 1 , wherein depth and flat-image cameras of the machine vision system have parallel optical axes oriented in the same direction.

11. The method of claim 1 , further comprising representing the targeted display object visually as a selected object in response to a selection command spoken by the user.

12. The method of claim 1 , further comprising receiving user input defining the spatial coordinates.

13. The method of claim 1 , wherein the one or more non-display objects define a viewing area of the display.

14. The method of claim 1 , further comprising:

receiving, from the machine vision system, final video imaging the head and pointer of the user;

computing, based on the final video, a final line of sight of the user partly occluded by the pointer; and

forming on the display, along the final line of sight, an image to represent the targeted display object.

15. The method of claim 1 , wherein the code is further executable to select the targeted display object in response to a selection command from the user.

16. The method of claim 1 , wherein the machine vision system and the display are integrated in a head-mounted device worn by the user.

17. In a compute system operatively coupled to a machine vision system and to a display, a method comprising:

receiving, from the machine vision system, video imaging a head and hand of a user;

computing, based on the video, a geometric line of sight of the user partly occluded by the hand;

identifying, with reference to position data for one or more objects, a targeted display object situated on the display and along the geometric line of sight;

identifying, in the video, movement of the hand away from the user, along the geometric line of sight;

and shifting a depth coordinate of the display object in response to movement of the hand away from the user, along the geometric line of sight.

18. The method of claim 17 further comprising:

processing the video to identify closure of the hand, with the hand remaining in the geometric line of sight; and

reducing the size of the display object in response to closure of the hand, with the hand maintained in the geometric line of sight.

19. In a compute system operatively coupled to a machine vision system, a method comprising:

associating a process of the compute system to a non-display object arranged in a field of view of the machine-vision system, the process being one of a plurality of processes associated with a corresponding plurality of non-display objects in the field of view of the machine-vision system;

receiving, from the machine vision system, video imaging a head and pointer of a user;

computing, based on the video, a geometric line of sight of the user partly occluded by the pointer;

identifying the non-display object by reference to position data defining spatial coordinates of the plurality of non-display objects, the non-display object being situated along the geometric line of sight;

identifying in the video movement of the pointer along the geometric line of sight;

and executing the process associated with the targeted non-display object in response to the movement.

20. The method of claim 19 , wherein the machine vision system is integrated into a head-mounted display device worn by the user.