Method and system for providing roughness haptic sense of virtual object by using space-time encoding

1. A method of providing a roughness haptic sense of a virtual object, the method comprising:

displaying virtual objects to a user;

tracking a position of a hand of the user;

predicting a virtual object most likely to collide with the hand among the virtual objects; and

in response to the hand contacting a haptic sense providing device, controlling the haptic sense providing device according to a surface roughness of the predicted virtual object to feed back a roughness haptic sense to the user,

wherein the predicting of the virtual object comprises:

predicting a region of interest of the user based on a gaze direction of the user; and

predicting the virtual object comprising the region of interest to be a virtual object most likely to collide with the hand.

2. The method of claim 1 , wherein the feeding back of the roughness haptic sense comprises:

rotating or moving an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

3. The method of claim 2 , wherein, for the end effector,

polygonal or elliptical shaped protrusions in which an X-axis radius is different from a radius of another axis are formed, or a textured surface on which the protrusions are formed is attached, so that different roughness haptic senses are fed back to the user according to an angle in which the end effector is in contact with the hand.

4. The method of claim 1 , wherein the predicting of the virtual object comprises:

predicting a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand; and

moving a position of the end effector that makes contact with the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user according to the predicted position and time.

5. A system for providing a roughness of a virtual object, the system comprising:

an HMD for displaying virtual objects to a user;

a virtual object roughness providing device for tracking a position of a hand of the user and predicting a virtual object most likely to collide with the hand among the virtual objects; and

a haptic sense providing device for feeding back a roughness haptic sense to the user according to a surface roughness of the predicted virtual object, in response to the hand contacting the haptic sense providing device,

wherein the virtual object roughness providing device predicts a region of interest of the user based on a gaze direction of the user, and predicts a virtual object including the region of interest to be a virtual object most likely to collide with the hand.

6. The system of claim 5 , wherein the virtual object roughness providing device rotates or moves an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

7. The system of claim 6 , wherein, for the end effector,

polygonal or elliptical shaped protrusions in which an X-axis radius is different from a radius of another axis are formed, or a textured surface on which the protrusions are formed is attached, so that different roughness haptic senses are fed back to the user according to an angle in which the end effector is in contact with the hand.

8. The system of claim 5 , wherein the virtual object roughness providing device predicts a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand, and according to the predicted position and time, moves a position of the end effector that contacts the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user.

9. A virtual object roughness providing device, the device comprising:

a visual rendering unit for displaying virtual objects to a user using an HMD;

a user tracker for tracking a position of a hand of the user;

an object prediction unit for predicting a virtual object most likely to collide with the hand among the virtual objects; and

a haptic sense rendering unit for, in response to the hand contacting a haptic sense providing device, controlling the haptic sense providing device according to a surface roughness of the predicted virtual object to feed back a roughness haptic sense to the user,

wherein the object prediction unit predicts a region of interest of the user based on a gaze direction of the user, and predicts a virtual object including the region of interest to be a virtual object most likely to collide with the hand.

10. The device of claim 9 , wherein the haptic sense rendering unit rotates or moves an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

11. The device of claim 9 , wherein

the object prediction unit predicts a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand, and

the haptic sense rendering unit moves a position of the end effector that contacts the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user according to the predicted position and time.