Wearable device for controlling displaying of visual object corresponding to external object and method thereof

1 . A wearable device, comprising:

at least one first camera;

at least one second camera;

at least one display;

memory including one or more storage media storing instructions;

at least one processor including processing circuitry, wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

obtain, via the at least one display, a first image frame by using the at least one first camera;

identify a gaze information by using the at least one second camera;

identify, in the first image frame, a first visual object among a plurality of visual objects to which a gaze position indicated by the gaze information is directed;

based on identifying the first visual object, display, via the at least one display, a virtual image and a portion of the first image frame in which the first visual object is maintained and other visual objects are hidden;

while displaying the virtual image and the portion of the first image frame, obtain a second image frame by using the at least one first camera;

identify at least one motion of the plurality of visual objects between the first image frame and the second image frame; and

based on identifying a motion of a second visual object among the plurality of visual objects, display the virtual image and portions of the second image frame, wherein each of the portions of the second image frame correspond to the first visual object and the second visual object.

2 . The wearable device of claim 1 , wherein the at least one display comprises:

a first display having a position corresponding to a left eye of a user while worn by the user;

a second display having a position corresponding to right eye of the user while worn by the user.

3 . The wearable device of claim 2 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify the first visual object positioned on a portion of the first display or the second display, wherein the portion is determined as focused by the user based on the gaze position indicated by the gaze information.

4 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify the second visual object based on whether each of the plurality of visual objects is overlapped to the first visual object in the first image frame.

5 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify the second visual object based on categories to which each of the plurality of visual objects is classified.

6 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

based on at least one of types of the plurality of visual objects or a user input, classify the plurality of visual objects into one of preset categories according to displaying conditions.

7 . The wearable device of claim 1 , further comprises a microphone,

wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

in a state displaying the virtual image and the portion of the image frame in which the first visual object is maintained and other visual objects are hidden, identify, by using the microphone, an external object generating sound of which an intensity is greater than a preset threshold intensity;

based on identifying the external object, display a third visual object corresponding to the external object in the display while maintaining to display the portion of the image frame in which the first visual object is maintained.

8 . The wearable device of claim 7 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify the preset threshold intensity based on a type of the first visual object.

9 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify, among the other visual objects different from the first visual object corresponding to a first external object and the second visual object, a third visual object having a type corresponding to an external object ceasing interaction between a user wearing the wearable device and the first external object based on modification of a state;

based on an event associated with the third visual object, display a fourth visual object linked to the third visual object in the frame.

10 . The wearable device of claim 9 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

identify, by using the frame, whether a distance is decreased lower than a preset distance between the external object corresponding to the third visual object and a body part;

based on identifying the distance decreased lower than the preset distance, display a message for guiding another modification of the state of the third visual object in the display.

11 . The wearable device of elain claim 9 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

in a state displaying the portion of the image frame in which the first visual object is maintained, display, based on identifying the third visual object distinguished from the plurality of visual objects, the third visual object in the display.

12 . The wearable device of claim 11 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

based on a duration while the third visual object is displayed, display a fourth visual object for selecting whether to maintain displaying the third visual object in the display.

13 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

display a texture image having a shape of the second visual object superimposed on the second visual object in the display.

14 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually or collectively, cause the wearable device to:

modify the at least the portion of the frame into the modified portion by combining an image larger than the at least the portion to the frame.

15 . A method of a wearable device including at least one first camera, at least one second camera, and at least one display, comprising:

obtaining, via the at least one display, a first image frame by using the at least one first camera;

identifying a gaze information by using the at least one second camera;

identifying, in the first image frame, a first visual object among a plurality of visual objects to which a gaze position indicated by the gaze information is directed;

based on identifying the first visual object, displaying, via the at least one display, a virtual image and a portion of the first image frame in which the first visual object is maintained and other visual objects are hidden;

displaying, while displaying the virtual image and the portion of the first image frame, obtaining a second image frame by using the at least one first camera;

identifying at least one motion of the plurality of visual objects between the first image frame and the second image frame; and

based on identifying a motion of a second visual object among the plurality of visual objects, displaying the virtual image and portions of the second image frame, wherein each of the portions of the second image frame correspond to the first visual object and the second visual object.

16 . The method of claim 15 , wherein the display comprises:

a first display having a position corresponding to a left eye of a user while worn by the user;

a second display having a position corresponding to right eye of the user while worn by the user.

17 . The method of claim 16 , wherein the identifying the first visual object comprises:

identifying the first visual object positioned on a portion of the first display or the second display, wherein the portion is determined as focused by the user based on the gaze position indicated by the gaze information.

18 . The method of claim 15 , wherein the identifying the second visual object comprises:

identifying the second visual object based on whether each of the plurality of visual objects is overlapped to the first visual object in the first image frame.

19 . The method of claim 15 , wherein the identifying the second visual object comprises:

identifying the second visual object based on categories to which each of the plurality of visual objects is classified.

20 . The method of claim 15 , wherein the displaying comprises:

displaying, with respect to the second visual object indicated as being contacted to the first visual object based on the positional relationship, a third visual object having a shape of the second visual object viewed through a field-of-view (FoV).