Head mounted display device and method for providing visual aid using same
An external scene image captured by an external scene imaging electronic camera attached to a head mounted display (HMD) is projected and displayed onto an image display screen arranged in front of the eyes of the user as a virtual image with a suitable viewing distance corresponding to the visual acuity of the user. At this time, for each object image presented in the virtual image of the external scene image, the virtual image is processed and formatted to add a predetermined degree of binocular disparity and image blur to the virtual image projected and displayed on the right and the left image display screen on the basis of a predetermined converted distance calculated from the real distance of each object. Thus, the user is given a sense of a realistic perspective for the virtual image of the external scene, free of the discomfort or unease.
1. A head mounted display device that is mounted in a head portion of a user and displays information videos in front of eyes of the user, the head mounted display device comprising:
an external scene imaging unit that captures an external scene in a visual field area including a direct visually-recognized visual field of the user;
a virtual image projector that projects and displays the external scene video captured by the external scene imaging unit on left-eye and right-eye video display screens separately arranged in front of the eyes of the user as left-eye and right-eye virtual image videos having a predetermined visible distance;
a within-visual-field target object distance detector that detects an actual distance up to a target object shown inside the external scene video captured by the external scene imaging unit from the user; and
a perspective video generator that generates virtual image videos to which depth perception is added by performing a predetermined treatment of the left-eye and right-eye virtual image videos on the basis of actual distance information of the target object detected by the within-visual-field target object distance detector,
wherein the perspective video generator has a function of calculating a scaled distance of the visible distance that matches the actual distance in a case where the actual distance of the target object is longer than a predetermined set distance and the scaled distance has a value larger than the actual distance in a case where the actual distance is shorter than the predetermined set distance.
2. The head mounted display device according to claim 1 , wherein the scaled distance of the visible distance, that matches the actual distance in the case where the actual distance of the target object is longer than the predetermined set distance, has a characteristic of gradually increasing a coefficient of proportion for a change amount of the actual distance from 1.0 in accordance with a decrease in the actual distance in the case where the actual distance is shorter than the predetermined set distance.
3. The head mounted display device according to claim 1 , wherein the perspective video generator has a function of applying an in-plane direction deviation corresponding to a binocular parallax amount that is in inverse proportion to the scaled distance on the basis of the scaled distance of the visible distance for each video of the target object shown in the left-eye and right-eye virtual image videos to the virtual image videos.
4. The head mounted display device according to claim 1 , wherein the perspective video generator has a function of applying a video shading amount that is in proportion to the scaled distance on the basis of the scaled distance of the visible distance for each video of the target object shown in the left-eye and right-eye virtual image videos to the virtual image videos.
5. The head mounted display device according to claim 4 , further comprising:
a visual line direction detector that detects a visual line direction of the user; and
a watching target object detector that detects a target object watched by the user from the visual line direction detected by the visual line direction detector and the external scene video captured by the external scene imaging unit,
wherein a function of adjusting the video shading amount applied to the virtual image videos such that the watching target object detected by the watching target object detector is visually recognized with good visibility is included.
6. The head mounted display device according to claim 1 , wherein the left-eye and right-eye video display screens are liquid crystal-type light control glasses of which transparency changes according to an applied voltage, function as non-transparent screens when the head mounted display device is in an operated state, and function as transparent or semi-transparent glasses when the head mounted display device is in a non-operated state.
7. The head mounted display device according to claim 1 , wherein sight correction glass lenses are arranged between the left-eye and right-eye video display screens and the eyes of the user.
8. The head mounted display device according to claim 1 , further comprising:
a sound collecting microphone that is used for detecting speech given by the user; and
a speech recognizer that recognizes a predetermined keyword given by the user from speech of the user collected by the sound collecting microphone,
wherein a function of increasing an external scene capturing magnification ratio of the external scene imaging unit to be a predetermined magnification ratio on the basis of a result of the recognition of the keyword is included.
9. The head mounted display device according to claim 1 , further comprising:
a sound collecting microphone that is used for detecting speech given by the user;
a speech recognizer that recognizes a predetermined keyword given by the user from speech of the user collected by the sound collecting microphone;
a character recognizer that performs predetermined character recognition from a character video included inside an external scene video captured by the external scene imaging unit; and
an automatic translator that automatically translates a predetermined language,
wherein a function of displaying a translated sentence of a foreign word, a foreign sentence, or a difficult Chinese character included in the external scene video to overlap the virtual image videos projected and displayed on the video display screens on the basis of a result of the recognition of the keyword is included.
10. The head mounted display device according to claim 1 , further comprising:
a sight measuring unit that has a function of measuring sight of the user; and
an optimal virtual image visible distance calculator that calculates a virtual image visible distance at which good visibility can be secured by the user from sight of the user detected by the sight measuring unit,
wherein a function of automatically adjusting a visible distance of the virtual image videos projected and displayed on the video display screens on the basis of optimal virtual image visible distance information calculated by the optimal virtual image visible distance calculator is included.
11. A video display method of a head mounted display device that is mounted in a head portion of a user and displays information videos in front of eyes of the user, the video display method comprising the steps of:
capturing an external scene video in a visual field area of the user;
detecting an actual distance up to a target object shown inside the external scene video from the user;
projecting and displaying the captured external scene video on left-eye and right-eye video display screens separately arranged in front of the eyes of the user as left-eye and right-eye virtual image videos having a predetermined visible distance;
adding depth perception to the left-eye and right-eye virtual image videos on the basis of actual distance information of the detected target object; and
calculating a scaled distance of the visible distance that matches the actual distance in a case where the actual distance of the target object is longer than a predetermined set distance and the scaled distance has a value larger than the actual distance in a case where the actual distance is shorter than the predetermined set distance.
12. A visual aid method using the video display method of the head mounted display device according to claim 11 , the visual aid method further comprising the steps of:
measuring sight of the user;
calculating the visible distance at which good visibility can be secured by the user from sight of the user; and
automatically adjusting a visible distance of virtual image videos projected and displayed on the video display screens on the basis of the calculated visible distance information.