Wearable terminal device, program, and notification method in mixed reality
A wearable terminal device is configured to be used by being worn by a user. The wearable terminal device includes at least one processor. The at least one processor detects a visible area for the user inside a space and causes a display to display an instructional image located inside the visible area. When there is an instructional image located outside the visible area, the at least one processor issues a first notification making the user aware of the existence of the instructional image. As a result of this first notification being issued, the user of the wearable terminal device is able to notice the existence of the instructional image generated by an instruction given by a remote instructor.
1 . A wearable terminal device configured to be used by being worn by a user, the wearable terminal device comprising:
a transparent display through which the user views a real space;
a camera configured to capture a real-time moving image of the real space viewed by the user; and
at least one processor, wherein the at least one processor is configured to:
transmit the real-time moving image to an external device configured to display the real-time moving image,
receive, from the external device based on a selected position within a still image of the real-time moving image, a command to display an instructional image on the transparent display at a position in the real space corresponding to the selected position in the still image;
detect a visible area of the real space viewed by the user through the transparent display,
determine that the position in the real space is outside the visible area, and
display a notification image on a periphery of the transparent display, a position of the periphery at which the notification image is disposed indicating a direction in which the instructional image is located in the real space, the notification image indicating existence of the instructional image outside of the visible area,
wherein in response to determining that a plurality of instructional images are located outside the visible area, the at least one processor is configured to control the transparent display to display a corresponding plurality of notification images on the periphery of the transparent display, wherein each of the plurality of notification images corresponds to a respective one of the plurality of instructional images, and a position of each notification image on the periphery indicates a direction of the corresponding instructional image.
2 . The wearable terminal device according to claim 1 ,
wherein
the at least one processor is configured to display the instructional image on a display surface of the transparent display with the instructional image visible in the real space that is visible through the transparent display.
3 . The wearable terminal device according to claim 1 ,
wherein the at least one processor causes the transparent display to display an image of the real space captured by the camera and the instructional image superimposed on the image of the real space.
4 . The wearable terminal device according to claim 1 , further comprising:
a communication unit configured to communicate data with an external device used by a remote instructor,
wherein the at least one processor
is configured to generate the instructional image based on instructional data received by the communication unit from the external device, and
control the transparent display to display the instructional image.
5 . The wearable terminal device according to claim 4 ,
wherein the communication unit is configured to perform speech data communication with the external device, and
the at least one processor is configured to control the transparent display to display the instructional image during the speech data communication via the communication unit.
6 . The wearable terminal device according to claim 4 ,
wherein during a condition in which the instructional image cannot be displayed based on the instructional data, the at least one processor is configured to control the transparent display to display a second notification making the user aware that the instructional image is not displayed on the transparent display.
7 . The wearable terminal device according to claim 1 ,
wherein the at least one processor is configured to control the transparent display to display the notification image being visually recognizable by the user.
8 . The wearable terminal device according to claim 7 ,
wherein the notification image is a prescribed notification display performed by the transparent display.
9 . The wearable terminal device according to claim 1 ,
wherein the notification image includes an output of prescribed sound.
10 . The wearable terminal device according to claim 1 ,
wherein the at least one processor is configured to identify a work object to be worked on by the user, and
determine a display position of the instructional image within a range excluding a range where the instructional image would visually obstruct the work object.
11 . The wearable terminal device according to claim 1 ,
wherein the instructional image is a document image of a prescribed format.
12 . The wearable terminal device according to claim 1 ,
wherein the instructional image is an image of a virtual object.
13 . The wearable terminal device according to claim 12 ,
wherein the virtual object is an object representing a path traced by pen input.
14 . The wearable terminal device according to claim 1 , wherein the at least one processor is configured to control the transparent display to display the notification image in a first manner in response to determining that the instructional image not been displayed on the transparent display before and in a second manner in response to determining that the instructional image has been displayed on the transparent display before.
15 . The wearable terminal device according to claim 1 , wherein the at least one processor is configured to control the transparent display to display the notification image in a first manner in response to determining that a time elapsed since the command was received is less than or equal to a prescribed reference time and in a second manner in response to determining that the time elapsed since the command was received is greater than the prescribed reference time.
16 . A notification method for use in a wearable terminal device configured to be used by being worn by a user, the notification method comprising:
capturing a real-time moving image of a real space viewed by the user through a transparent display of the wearable terminal device;
transmitting the real-time moving image to an external device configured to display the real-time moving image;
receiving, from the external device based on a selected position within a still image of the real-time moving image, a command to display an instructional image on the transparent display at a position in the real space corresponding to the selected position in the still image;
detecting a visible area of the real space viewed by the user through the transparent display;
determining that the position in the real space is outside the visible area;
displaying a notification image on a periphery of the transparent display, a position of the periphery at which the notification image is disposed indicating a direction in which the instructional image is located in the real space, the notification image indicating existence of the instructional image outside of the visible area; and
in response to determining that a plurality of instructional images are located outside the visible area, displaying a corresponding plurality of notification images on the periphery of the transparent display, wherein each of the plurality of notification images corresponds to a respective one of the plurality of instructional images, and a position of each notification image on the periphery indicates a direction of the corresponding instructional image.