Wearable device for processing audio signal based on external object recognized from image and method thereof
According to an embodiment, a processor of a wearable device obtains, while displaying a first screen with respect to a virtual space on an entire displaying region of a display, an audio signal including sound of an external space by using one or more microphones. The processor identifies, in response to detection of an external object generating the sound by using the audio signal, data corresponding to the external object in information associated with the external space obtained based on the camera. The processor, based on identifying the data in the information, displays, together with the first screen in the displaying region, a second screen including a portion segmented from an image obtained from the camera based on the identified data.
1 . A wearable device, comprising:
a camera;
one or more microphones;
a display;
memory, comprising one or more storage mediums, storing instructions; and
at least one processor comprising processing circuitry, wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
obtain, while a virtual space is displayed via the display and displaying an image obtained via the camera is disallowed, an audio signal using the one or more microphones;
based on the obtained audio signal being determined as output from an external object registered in the wearable device, search for a visual object corresponding to the external object from an image obtained via the camera; and
based on the visual object being searched from an image via the camera, display, via the display, a portion of the image including the visual object, together with the virtual space for allowing the external object to be viewed through a portion of the virtual space.
2 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
determine a size of the portion of the image in accordance with a type of the external object registered in the wearable device.
3 . The wearable device of claim 1 , further comprising another camera arranged to face eyes of a user of the wearable device when worn;
wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
identify, via the another camera, a gaze of the user; and
based on the gaze being identified as facing the visual object in the portion of image, increase a size of the portion of the image displayed together with the virtual space via the display.
4 . The wearable device of claim 3 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
based on the gaze being identified as positioned out of the visual object, decrease the size of the portion of the image displayed together with the virtual space via the display.
5 . The wearable device of claim 4 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
based on the gaze positioned out of the visual object being identified as maintained while a reference time, cease to display the portion of the image.
6 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
identify, via the camera, a motion of a user of the wearable device with respect to the external object; and
based on identifying the motion, increase a size of the portion of the image displayed together with the virtual space via the display.
7 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
based on another audio signal obtained using the one or more microphones being determined as output from the external object while the portion of the image is displayed together with the virtual space, increase a size of the portion of the image displayed together with the virtual space via the display.
8 . The wearable device of claim 1 , wherein the instructions, when executed by the at least one processor individually and/or collectively, cause the wearable device to:
based on the visual object being not searched from the image, display a visual object for guiding a position at which the audio signal is output in the virtual space.
9 . A method of a wearable device, comprising:
obtaining, while a virtual space is displayed via a display of the wearable device and displaying an image obtained via a camera of the wearable device is disallowed, an audio signal using one or more microphones of the wearable device;
based on the obtained audio signal being determined as output from an external object registered in the wearable device, searching for a visual object corresponding to the external object from an image obtained via the camera; and
based on the visual object being searched from an image via the camera, displaying, via the display, a portion of the image including the visual object, together with the virtual space for allowing the external object to be viewed through a portion of the virtual space.
10 . The method of claim 9 , comprising:
determining a size of the portion of the image in accordance with a type of the external object registered in the wearable device.
11 . The method of claim 9 , comprising:
identifying, via another camera of the wearable device, a gaze of the user; and
based on the gaze being identified as facing the visual object in the portion of image, increasing a size of the portion of the image displayed together with the virtual space via the display.
12 . The method of claim 11 , comprising:
based on the gaze being identified as positioned out of the visual object, decreasing the size of the portion of the image displayed together with the virtual space via the display.
13 . The method of claim 12 , comprising:
based on the gaze positioned out of the visual object being identified as maintained while a reference time, ceasing to display the portion of the image.
14 . The method of claim 11 , comprising:
identifying, via the camera, a motion of a user of the wearable device with respect to the external object; and
based on identifying the motion, increasing a size of the portion of the image displayed together with the virtual space via the display.
15 . The method of claim 11 , comprising:
based on another audio signal obtained using the one or more microphones being determined as output from the external object while the portion of the image is displayed together with the virtual space, increasing a size of the portion of the image displayed together with the virtual space via the display.
16 . The method of claim 9 , comprising:
based on the visual object being not searched from the image, displaying a visual object for guiding a position at which the audio signal is output in the virtual space.
17 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, when executed by a wearable device comprising a camera, one or more microphones, and a display, cause the head-wearable electronic device to:
obtain, while a virtual space is displayed via the display and displaying an image obtained via the camera is disallowed, an audio signal using the one or more microphones;
based on the obtained audio signal being determined as output from an external object registered in the wearable device, search for a visual object corresponding to the external object from an image obtained via the camera; and
based on the visual object being searched from an image via the camera, display, via the display, a portion of the image including the visual object, together with the virtual space for allowing the external object to be viewed through a portion of the virtual space.
18 . The non-transitory computer readable storage medium of claim 17 , wherein the one or more programs comprising instructions, when executed by a wearable device to:
determine a size of the portion of the image in accordance with a type of the external object registered in the wearable device.
19 . The non-transitory computer readable storage medium of claim 17 , wherein the wearable device further comprises another camera arranged to face eyes of a user of the wearable device when worn,
wherein the one or more programs comprising instructions, when executed by a wearable device to:
identify, via the another camera, a gaze of the user; and
based on the gaze being identified as facing the visual object in the portion of image, increase a size of the portion of the image displayed together with the virtual space via the display.
20 . The non-transitory computer readable storage medium of claim 19 , wherein the one or more programs comprising instructions, when executed by a wearable device to:
based on the gaze being identified as positioned out of the visual object, decrease the size of the portion of the image displayed together with the virtual space via the display.