User interaction in head-mounted display with eye tracking
A virtual reality (VR) system includes a console an imaging device, a head mounted display (HMD) and a user input device. The console includes a virtual reality (VR) engine, a tracking module and an application store. The HMD includes a display element configured to display content to a user wearing the HMD. The virtual reality engine is configured to determine a vector for a virtual object thrown at a target in response to physical input from the user, a gaze vector for the eyes of the user and a virtual object position vector for the target and modify the vector for the virtual object based on at least one of the gaze vector and the virtual object position vector. The virtual reality engine may also be configured to determine a virtual object position vector for a virtual object to be thrown at a target, a gaze vector for the eyes of the user and a virtual object position vector for a virtual rendering of the user's hand and enabling user input to pick up the virtual object based on at least one of the gaze vector and the virtual object position vector of the virtual rendering of the user's hand.
1. An AR/VR system comprising:
a head-mounted display (HMD) including an eye-tracking system and a display module for displaying content to a user wearing the HMD; and
a console having an AR/VR engine configured to:
determine a position vector of a virtual object to be picked up, determine a first gaze vector for eyes of the user using the eye-tracking system, and enable a user control input to effect a pick up of the virtual object when the first gaze vector is less than a predetermined distance away from the position vector of the virtual object;
determine a displacement vector for the virtual object thrown at a target in response to physical input from the user,
determine a second gaze vector for the eyes of the user using the eye-tracking system; and
modify the displacement vector so as to lessen a difference between the displacement vector and the second gaze vector.
2. The AR/VR system of claim 1 , wherein the AR/VR engine is configured to modify the displacement vector based on a direction component of the second gaze vector.
3. The AR/VR system of claim 1 , wherein the difference between the displacement vector and the second gaze vector is lessened by an amount depending upon a skill level of the user.
4. The AR/VR system of claim 1 , wherein the target comprises a moving target, and wherein the second gaze vector is determined when the user is looking ahead of the moving target.
5. The AR/VR system of claim 1 , wherein the AR/VR engine is further configured to determine a position vector of a hand of the user, and enable a user control input to effect a pick up of the virtual object based on the first gaze vector and the position vector of the hand of the user being within a predetermined distance.
6. In an AR/VR system comprising a head-mounted display (HMD) including an eye-tracking system and a display module for displaying content to a user wearing the HMD, and a console having an AR/VR engine, a method comprising:
determining a position vector of a virtual object to be picked up, determining a first gaze vector for eyes of the user using the eye-tracking system, and enabling a user control input to effect a pick up of the virtual object when the first gaze vector is less than a predetermined distance away from the position vector of the virtual object;
determining a displacement vector for the virtual object thrown at a target in response to physical input from the user,
determining a second gaze vector for the eyes of the user using the eye-tracking system; and
modifying the displacement vector so as to lessen a difference between the displacement vector and the second gaze vector.
7. The method of claim 6 , wherein the step of modifying the displacement vector is based on a direction component of the second gaze vector.
8. The method of claim 6 , wherein the difference between the displacement vector and the second gaze vector is lessened by an amount depending upon a skill level of the user.
9. The method of claim 6 , wherein the target comprises a moving target, and wherein the second gaze vector is determined when the user is looking ahead of the moving target.
10. The method of claim 6 , wherein the AR/VR engine is further configured to determine a position vector of a hand of the user, and enable a user control input to effect a pick up of the virtual object based on the first gaze vector and the position vector of the hand of the user being within a predetermined distance.
11. A computer product comprising a non-transitory computer usable medium including a computer-readable program which when executed on a computer configures the computer to:
determine a position vector of a virtual object to be picked up, determine a first gaze vector for eyes of a user, and enable a user control input to effect a pick up of the virtual object when the first gaze vector is less than a predetermined distance away from the position vector of the virtual object;
determine a displacement vector for a virtual object thrown at a target in response to physical input from the user;
determine a second gaze vector for eyes of the user using the eye-tracking system; and
modify the displacement vector so as to lessen a difference between the displacement vector and the second gaze vector.
12. The computer product of claim 11 , wherein the displacement vector is modified based on a direction component of the second gaze vector.
13. The computer product of claim 11 , wherein the difference between the displacement vector and the second gaze vector is lessened by an amount depending upon a skill level of the user.
14. The computer product of claim 11 , wherein the target comprises a moving target, and wherein the second gaze vector is determined when the user is looking ahead of the moving target.
15. The computer product of claim 11 , wherein the computer is further configured to determine a position vector of a hand of the user, and enable a user control input to effect a pick up of the virtual object based on the first gaze vector and the position vector of the hand of the user being within a predetermined distance.