Wearable device, method, and computer-readable storage medium, for adjusting gaze information of user
In accordance with an aspect of the disclosure, a wearable device is provided. The wearable device includes a first camera, a second camera, a display, memory storing one or more computer programs; and one or more processors communicatively coupled to the first camera, the second camera, the display, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the at one or more processors individually or collectively, cause the wearable device to display, using the display, a visual object within a field of view (FoV) of a user in a state where the wearable device is worn by the user, while displaying the visual object, based on the first camera disposed toward the FoV, identify an external object, interacting with the visual object, seen through the FoV, obtain a position of the external object, based on identifying an interaction between the external object and the visual object using the first camera, and change information indicating a direction of eyes, identified by frames outputted from the second camera disposed toward the eyes of the user, using at least one of a position of the visual object or the position of the external object, based on obtaining the position of the external object.
1 . A wearable device comprising:
a first camera disposed toward a field of view (FoV) of a user;
a second camera configured to obtain an image of eyes of the user for identifying a gaze of the user;
a display;
at least one processor comprising processing circuitry; and
memory comprising one or more storage mediums storing instructions,
wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
display, using the display, a visual object within a FoV of the first camera, in a state where the wearable device is worn by the user,
while displaying the visual object, identify, using the first camera, a point in the visual object indicated by a hand of the user in the FoV, and
based on the gaze of the user being identified as positioned within a designated distance from the point using the second camera, calibrate a position of the gaze of the user in accordance with the point.
2 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to: identify a probability that the gaze of the user corresponds to the point in accordance with a distance between the position of the gaze of the user and the point,
based on the distance being identified shorter than the designated distance, identify that the probability is higher than a threshold, and
based on identifying that the probability is higher than the threshold, calibrate the position of the gaze of the user in accordance with the point.
3 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
based on the gaze of the user being identified as positioned outside of the designated distance from the point using the second camera, display, using the display, another visual object for guiding to calibrate a wearing state of the wearable device.
4 . The wearable device of claim 3 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
identify a probability that the gaze of the user corresponds to the point in accordance with a distance between the position of the gaze of the user and the point,
based on the distance being identified as longer than the designated distance, identify that the probability is lower than a threshold, and
based on identifying that the probability is lower than the threshold, display, using the display, the another visual object.
5 . The wearable device of claim 1 , wherein the point corresponds to an ending point of a designated gesture of the hand.
6 . The wearable device of claim 1 , further comprising communication circuitry,
wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
receive, using the communication circuitry, from an external object, position information indicating a position of the external object, and
calibrate the position of the gaze of the user further based on the position of the external object indicated by the position information.
7 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
calibrate the position of the gaze of the user by changing the position of the gaze of the user to the point.
8 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
identify another user using the first camera in the FoV,
while displaying the visual object, using the first camera, identify another point in the visual object indicated by a hand of the another user in the FoV, and
calibrate the position of the gaze of the user further based on the another point.
9 . The wearable device of claim 1 , further comprising communication circuitry,
wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:
while another user is in the FoV, receive, via the communication circuitry, from an external electronic device of the another user, information indicating a position of a gaze of the another user, and
calibrate the position of the gaze of the user further based on the position of the gaze of the another user indicated by the position information.
10 . The wearable device of claim 1 , wherein the hand of the user indicates the point by pointing the point in the visual object or grabbing the point in the visual object.
11 . A method performed by a wearable device, the method comprising:
displaying, using a display, a visual object within a field of view (FoV) of a first camera of the wearable device disposed toward a FoV of a user, in a state where the wearable device is worn by the user;
while displaying the visual object, identifying, using the first camera, a point in an external object indicated by a hand of the user; and
based on a gaze of the user being identified as positioned within a designated distance from the point using a second camera of the wearable device configured to obtain an image of eyes of the user for identifying the gaze of the user, calibrating a position of the gaze of the user in accordance with the point.
12 . The method of claim 11 , further comprising:
identifying a probability that the gaze of the user corresponds to the point in accordance with a distance between the position of the gaze of the user and the point,
based on the distance being identified shorter than the designated distance, identifying that the probability is higher than a threshold, and
based on identifying that the probability is higher than the threshold, calibrating the position of the gaze of the user in accordance with the point.
13 . The method of claim 11 , further comprising:
based on the gaze of the user being identified as positioned outside of the designated distance from the point using the second camera, displaying, using the display, another visual object for guiding to calibrate a wearing state of the wearable device.
14 . The method of claim 13 , further comprising:
identifying a probability that the gaze of the user corresponds to the point in accordance with a distance between the position of the gaze of the user and the point,
based on the distance being identified as longer than the designated distance, identifying that the probability is higher than a threshold, and
based on identifying that the probability is higher than the threshold, calibrating the position of the gaze of the user in accordance with the point.
15 . The method of claim 11 , wherein the point corresponds to an ending point of a designated gesture of the hand.
16 . The method of claim 11 , further comprising:
receiving, using communication circuitry, from the external object, position information indicating a position of the external object, and
calibrating the position of the gaze of the user further based on the position of the external object indicated by the position information.
17 . The method of claim 11 , further comprising:
calibrating the position of the gaze of the user by changing the position of the gaze of the user to the point.
18 . The method of claim 11 , further comprising:
identifying another user using the first camera in the FoV,
while displaying the visual object, using the first camera, identifying another point in the visual object indicated by a hand of the another user in the FoV, and
calibrating the position of the gaze of the user further based on the another point.
19 . The method of claim 11 , further comprising:
while another user is in the FoV, receiving, via communication circuitry, from an external electronic device of the another user, information indicating a position of a gaze of the another user, and
calibrating the position of the gaze of the user further based on the position of the gaze of the another user indicated by the position information.
20 . The method of claim 11 , wherein the hand of the user indicates the point by pointing the point in the visual object or grabbing the point in the visual object.