Device and method for obtaining images in augmented reality
A device and a method for recommending a custom function in an electronic device to a user is provided. The electronic device obtains a user gesture that repeatedly occur through interaction with the user and determines one or more customized functions to be suggested to the user among a plurality of accessibility functions considering the user gesture. The electronic device may execute a target customized function selected by the user from among the one or more customized functions.
1 . An augmented reality device, comprising:
a lens unit comprising a reflective surface configured to transmit light of a first wavelength and to reflect light of a second wavelength;
a support configured to mount the augmented reality device on a head of a user;
a light emitter disposed at the support, the light emitter configured to generate first light of the first wavelength and second light of the second wavelength;
a light receiver disposed at the support, the light receiver configured to:
obtain third light corresponding to the first light and output an object image signal based on the third light, and
obtain fourth light corresponding to the second light and output an eye image signal based on the fourth light,
wherein the reflective surface or a left eye lens of the lens unit is printed with a marker which transmits the second wavelength,
wherein the third light corresponds to the first light reflected from an object positioned in front of the augmented reality device, transmitted through the reflective surface, and received at the light receiver, and
wherein the fourth light corresponds to the second light reflected from an eye of the user and then reflected by a portion of the reflective surface excluding the marker before being received at the light receiver.
2 . The augmented reality device of claim 1 , wherein the light emitter comprises:
a first light emitting element configured to generate the first light, the first light emitting element disposed at a front portion of the lens unit to allow the first light to reach the object without transmitting through the reflective surface; and
a second light emitting element configured to generate the second light and provided to allow the second light to reach the eye of the user.
3 . The augmented reality device of claim 1 , wherein the light emitter comprises:
a first light emitting element configured to generate the first light and disposed at the inner surface of the support to allow the first light to reach the object through the reflective surface; and
a second light emitting element configured to generate the second light and disposed to allow the second light to reach the eye of the user.
4 . The augmented reality device of claim 2 , wherein the second light emitting element is disposed at the inner surface of the support to face the lens unit.
5 . The augmented reality device of claim 1 , wherein the first wavelength is different from the second wavelength.
6 . The augmented reality device of claim 1 , wherein the first incident light is a first infrared ray having a wavelength of 850 nm, and the second incident light is a second infrared ray having a wavelength of 940 nm.
7 . The augmented reality device of claim 1 , wherein the reflective surface comprises a first coating material disposed at an inner surface of a lens included in the lens unit.
8 . The augmented reality device of claim 1 , wherein the reflective surface comprises a first coating material disposed at an outer surface of a lens included in the lens unit.
9 . The augmented reality device of claim 1 , wherein the light receiver comprises:
a lens configured to condense an incoming light;
a splitter configured to separate light transmitted through the lens into the first reflected light and the second reflected light;
a first photosensor configured to convert the third light into the object image signal; and
a second photosensor configured to convert the second reflected light into the eye image signal.
10 . The augmented reality device of claim 1 , further comprising a processor configured to recognize the object based on the object image signal or to recognize an eye based on the eye image signal.
11 . The augmented reality device of claim 10 , wherein the light receiver comprises:
a lens configured to condense an incoming light;
a filter configured to extract only an optical signal of a specific wavelength from an optical signal condensed through the lens; and
a light detector converting the optical signal of the specific wavelength extracted by the filter into the object image signal or the eye image signal.
12 . The augmented reality device of claim 1 , wherein the lens unit comprises a right eye lens and a left eye lens, and
wherein the light emitter comprises:
a first light emitting element disposed near the right eye lens and configured to generate first right eye light to be transferred to the object;
a second light emitting element disposed near the left eye lens and configured to generate first left eye light to be transferred to the object;
a third light emitting element disposed near the right eye lens and configured to generate second right eye light to be transferred to the user's eye; and
a fourth light emitting element disposed near the left eye lens and configured to generate second left eye light to be transferred to the eye of the user, and
wherein the light receiver comprises:
a first light receiver disposed near the right eye lens and configured to convert first right eye reflected light, reflected by the object and transmitted through a first reflective surface provided in the right eye lens, into a first object image signal and output the first object image signal and to convert second right eye reflected light, reflected by the eye of the user and reflected by the first reflective surface, into a first eye image signal and output the first eye image signal; and
a second light receiver provided near the left eye lens and configured to convert first left eye reflected light, reflected by the object and transmitted through a second reflective surface provided in the left eye lens, into a second object image signal and output the second object image signal and to convert second left eye reflected light, reflected by the eye of the user and reflected by the second reflective surface, into a second eye image signal and output the second eye image signal.
13 . The augmented reality device of claim 12 , further comprising a processor is configured to obtain depth information of the object based on the first object image signal and the second object image signal.
14 . The augmented reality device of claim 1 , wherein a pattern of the marker comprises an area configured to reflect one of the first light or the second light.
15 . The augmented reality device of claim 1 , further comprising a processor configured to:
obtain an inclination angle corresponding to a degree of spread of the support by analyzing a marker image signal, and
correct a position of an object image obtained by the object image signal or an eye image obtained by the eye image signal based on the inclination angle.
16 . The augmented reality device of claim 15 , wherein the light receiver is configured to convert reflected light of the second light, reflected by the marker, into the marker image signal and output the marker image signal.
17 . A method for performing a multi-modal interaction in an augmented reality device, the method comprising:
emitting first light of a first wavelength in front of the augmented reality device through a lens;
emitting second light of a second wavelength different from the first wavelength toward an eye of a user wearing the augmented reality device;
receiving reflected light comprising third light of the first wavelength reflected from an object in front of the augmented reality device, and fourth light of the second wavelength reflected from the eye;
obtaining an object image by separating the third light from the reflected light;
obtaining an eye image by separating the fourth light from the reflected light; and
performing an interaction with a user based on the obtained object image or the obtained eye image;
wherein the first light and the second light are emitted towards a marker and disposed at a reflective surface of the lens, the marker being configured to allow transmission of the second reflected light,
wherein the third light corresponds to the first light reflected from the object, transmitted through the reflective surface, and received at a light receiver, and
wherein the fourth light corresponds to the second light reflected from the eye of the user and then reflected by a portion of the reflective surface excluding the marker before being received at the light receiver.
18 . The method of claim 17 , further comprising:
obtaining a marker image reflected from the marker based on the second light;
estimating a temple inclination angle based on a degree of distortion of the obtained marker image; and
performing position correction on the object image or the eye image based on the estimated temple inclination angle.
19 . The method of claim 17 , wherein the first light is a first infrared ray having a wavelength of 850 nm, and the second light is a second infrared ray having a wavelength of 940 nm.
20 . The method of claim 17 , wherein the first light reaches the object transmitted through the lens, and the second light reaches the eye of the user reflected from the lens.