HEAD MOUNT DISPLAY DEVICE PROVIDING EYE GAZE CALIBRATION AND CONTROL METHOD THEREOF
An eye gaze calibration method includes sensing an image of surroundings located in front of the HMD device using a camera unit, detecting a reference object from the sensed image of the surroundings using an image processing unit, detecting an eye gaze direction in which a user gazes at the detected reference object using an eye gaze detection unit, and performing eye gaze calibration based on the detected reference object and the detected eye gaze direction using a controller, wherein detecting the reference object includes detecting at least one object contained in the sensed image as a first object, detecting a hand of the user contained in the sensed image as a second object, detecting an event of the second object with respect to the first object, and deciding the first object, on which the event has been detected, as the reference object.
1 . A head mounted display (HMD) device comprising:
a camera unit configured to sense an image of surroundings located in front of the HMD device;
an image processing unit configured to detect a reference object from the sensed image of the surroundings;
an eye gaze detection unit configured to detect an eye gaze direction in which a user gazes at the detected reference object; and
a controller configured to perform eye gaze calibration based on the detected reference object and the detected eye gaze direction,
wherein the image processing unit is configured to
detect at least one object contained in the sensed image of the surroundings as a first object,
detect a hand of the user contained in the sensed image of the surroundings as a second object,
detect an event of the second object with respect to the first object, and
decide the first object, on which the event has been detected, as the reference object.
2 . The HMD device according to claim 1 , wherein
the first object is an external device,
the second object is a tip of a finger of the user, and
the event of the second object with respect to the first object comprises an operation of controlling the external device.
3 . The HMD device according to claim 2 , wherein
the first object is a keyboard, and
the event of the second object with respect to the first object comprises an operation of pressing a key of the keyboard.
4 . The HMD device according to claim 1 , wherein
the controller performs the eye gaze calibration based on a reference point located on the detected reference object, and
the reference point is a portion of the first object contacted by the second object to control the first object.
5 . The HMD device according to claim 4 , wherein
in a case in which the reference object is an electronic instrument,
the reference point comprises a control button of the electronic instrument, and
the event comprises an operation of controlling the control button of the electronic instrument.
6 . The HMD device according to claim 5 , wherein the control button of the electronic instrument comprises at least one selected from among a power button, volume control buttons, and channel control buttons.
7 . The HMD device according to claim 4 , wherein
in a case in which the reference object is a device comprising a touch sensitive display,
the reference point comprises a soft button displayed on the touch sensitive display.
8 . The HMD device according to claim 1 , wherein
in a case in which the first object is a pen,
the event of the second object with respect to the first object comprises a writing operation using the pen, and
the reference object is a tip of the pen.
9 . The HMD device according to claim 1 , further comprising:
a display unit configured to display a virtual object, wherein
the display unit controls a position of the displayed virtual object according to a result of the eye gaze calibration, and
the virtual object is a graphical user interface provided to the user by the display unit.
10 . The HMD device according to claim 9 , wherein the display unit displays the virtual object based on a position of the reference object using an augmented reality method.
11 . The HMD device according to claim 9 , wherein the display unit is a transparent optical display unit.
12 . The HMD device according to claim 9 , further comprising:
a distance measurement unit configured to measure a first distance from the HMD device to the detected reference object, wherein
the HMD device decides depth in which the virtual object is displayed based on the measured first distance.
13 . The HMD device according to claim 12 , wherein the distance measurement unit measures the first distance using at least one selected from among an ultrasonic sensor, a laser sensor, and an infrared sensor.
14 . The HMD device according to claim 12 , wherein the display unit provides a left eye and a right eye of the user with a left image and a right image, respectively, and expresses the depth of the virtual object using binocular disparity between the left image and the right image.
15 . The HMD device according to claim 1 , wherein the controller performs the eye gaze calibration only in a case in which a difference between a direction of the reference object and the eye gaze direction in which the user gazes at the reference object is within an allowable error range.
16 . The HMD device according to claim 1 , wherein the eye gaze detection unit detects the eye gaze direction based on movement of a pupil of the user.
17 . The HMD device according to claim 1 , wherein the second object is detected by at least one of a shape and a gesture of the hand of the user contained in the sensed image of the surroundings.
18 . The HMD device according to claim 1 , wherein the camera unit senses an image having a range corresponding to a visual field of the user with respect to the surroundings.
19 . An eye gaze calibration method of an HMD device comprising:
sensing an image of surroundings located in front of the HMD device using a camera unit;
detecting a reference object from the sensed image of the surroundings using an image processing unit;
detecting an eye gaze direction in which a user gazes at the detected reference object using an eye gaze detection unit; and
performing eye gaze calibration based on the detected reference object and the detected eye gaze direction using a controller,
wherein the step of detecting the reference object comprises:
detecting at least one object contained in the sensed image of the surroundings as a first object;
detecting a hand of the user contained in the sensed image of the surroundings as a second object;
detecting an event of the second object with respect to the first object; and
deciding the first object, on which the event has been detected, as the reference object.
20 . An eye gaze calibration method of an HMD device comprising:
sensing an image of surroundings located in front of the HMD device using a camera unit;
detecting a reference object from the sensed image of the surroundings using an image processing unit;
detecting an eye gaze direction in which a user gazes at the detected reference object using an eye gaze detection unit; and
performing eye gaze calibration based on the detected reference object and the detected eye gaze direction using a controller,
wherein the step of detecting the reference object comprises:
detecting a hand of the user contained in the sensed image of the surroundings as a first object;
detecting at least one object contained in the sensed image of the surroundings as a second object;
detecting an event of the first object with respect to the second object; and
deciding the second object, on which the event has been detected, as the reference object.
21 . An HMD device comprising:
a camera unit configured to sense an image of surroundings located in front of the HMD device;
an image processing unit configured to detect a reference object from the sensed image of the surroundings;
an eye gaze detection unit configured to detect an eye gaze direction in which a user gazes at the detected reference object; and
a controller configured to perform eye gaze calibration based on the detected reference object and the detected eye gaze direction,
wherein the image processing unit is configured to
detect a hand of the user contained in the sensed image of the surroundings,
detect a gesture performed by the hand of the user, and
decide the detected hand of the user together with the gesture as the reference object.