Electronic device and method for 3-dimensional (3D) spatial mapping using the same
An electronic device and method are disclosed. The electronic device includes a first and second camera, display, memory and processor. The processor implements the method, including acquiring image data of an external environment via the first camera, detecting a plurality of objects included in the image data, identifying a first object corresponding to the detected gaze among the detected plurality of objects, configuring a first precision for spatial mapping of the identified first object and a second precision of at least one other object from among the detected plurality of objects, wherein the first precision is higher than the second precision, executing 3D spatial mapping on the image data using the first precision for the identified first object and the second precision for the at least one other object, and displaying a 3D space generated based on the image data and the spatial mapping.
1 . An electronic device, comprising:
a first camera;
a second camera;
a display;
memory storing instructions; and
a processor,
wherein the instructions, when executed by the processor, cause the electronic device to:
acquire, via the first camera, image data representing surrounding environments of the electronic device,
detect, via the second camera, a gaze of a user,
identify a first region corresponding to the detected gaze in the image data,
perform 3-dimensional (3D) spatial mapping on the acquired image data by applying a first spatial mapping precision to the identified first region and a second spatial mapping precision to a second region other than the first region, wherein the second spatial mapping precision is lower than the first spatial mapping precision, and
display, on the display, a 3D image generated based on the performed 3D spatial mapping.
2 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
generate a precision map for the acquired image data by applying the first spatial mapping precision to the identified first region and the second precision level to the second region, and
wherein the 3D spatial mapping is further performed on the image data based on the generated precision map.
3 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
obtain depth information for the acquired image data,
based on the obtained depth information, set a first value for pixels disposed in the first region, and set a second value for pixels disposed in the second region, and
generate a precision map for the image data based at least on the set first value and the set second value.
4 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
obtain, from the image data, depth information associated with the first region,
execute a multiplication of coordinates associated with the first region by the obtained depth information,
when a value resultant from the multiplication exceeds a predesignated threshold value, set a first value for pixels disposed in the first region, and set a second value for pixels disposed in the second region, and
generate a precision map for the image data based on at least one of the set first value and the set second value.
5 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
detect, via the second camera, a second gaze of the user,
if the second gaze is different from the gaze, apply the first spatial mapping precision to a third region corresponding to the detected second gaze in the image data, and apply the second spatial mapping precision to a fourth region other than the third region in the image data, and
if the second gaze is identical to the gaze, maintain applying the first spatial mapping precision to the first region.
6 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
detect a period of time for which the gaze is maintained on the first region, and
when the period of time exceeds a predesignated time threshold, set the first region as a region of interest.
7 . The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
monitor, via the second camera, the gaze of the user for a designated period of time, and
aggregate and store user gaze information based on the monitored gaze over the designated period of time in the memory.
8 . The electronic device of claim 7 , wherein the instructions, when executed by the processor, cause the electronic device to:
based on the aggregated user gaze information, set a size defining a specific region that is centered on one point on the display, and
set the specific region as a region of interest.
9 . A spatial mapping method for an electronic device, the method comprising:
acquiring, by a processor of the electronic device, via a first camera, image data representing surrounding environments of the electronic device;
detecting, by the processor, via a second camera, a gaze of a user;
identifying, by the processor, a first region corresponding to the detected gaze in the image data;
performing, by the processor, 3-dimensional (3D) spatial mapping on the acquired image data by applying a first spatial mapping precision to the identified first region and a second spatial mapping precision to a second region other than the first region, wherein the second spatial mapping precision is lower than the first spatial mapping precision; and
displaying, by the processor, a 3D image generated based on the performed 3D spatial mapping on a display.
10 . The method of claim 9 , further comprising:
generating a precision map for the acquired image data by applying the first spatial mapping precision to the identified first region and the second spatial mapping precision to the second region,
wherein the 3D spatial mapping is further performed on the image data based on the generated precision map.
11 . The method of claim 9 , further comprising:
obtaining depth information for the acquired image data;
based on the obtained depth information, setting a first value for pixels disposed in the first region, and setting a second value for pixels disposed in the second region; and
generating a precision map for the image data based at least on the set first value and the set second value.
12 . The method of claim 9 , further comprising:
obtaining, from the image data, depth information associated with the first region;
executing a multiplication of coordinates associated with the first region by the obtained depth information;
when a value resultant from the multiplication exceeds a predesignated threshold value, setting a first value for pixels disposed in the first region, and setting a second value for pixels disposed in the second region; and
generating a precision map for the image data based on at least one of the set first value and the set second value.
13 . The method of claim 9 , further comprising:
detecting, via the second camera, a second gaze of the user;
if the second gaze is different from the gaze, applying the first spatial mapping precision to a third region corresponding to the detected second gaze in the image data, and applying the second spatial mapping precision to a fourth region other than the third region in the image data; and
if the second gaze is identical to the gaze, maintaining applying the first spatial mapping precision to the first region.
14 . The method of claim 9 , further comprising:
detecting a period of time for which the gaze is maintained on the first region; and
when the period of time exceeds a predesignated time threshold, setting the first region as a region of interest.
15 . The method of claim 9 , further comprising:
monitoring, via the second camera, the gaze of the user for a designated period of time;
aggregating and storing user gaze information based on the monitored gaze over the designated period of time in a memory;
based on the aggregated user gaze information, setting a size defining a specific region that is centered on one point on the display; and
setting the specific region as a region of interest.
16 . One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions that, when executed by a processor of an electronic device, cause the electronic device to perform operations, the operations comprising:
acquiring, via a first camera, image data representing surrounding environments of the electronic device;
detecting, via a second camera, a gaze of a user;
identifying a first region corresponding to the detected gaze in the image data;
performing 3-dimensional (3D) spatial mapping on the acquired image data by applying a first spatial mapping precision to the identified first region and a second spatial mapping precision to a second region other than the first region, wherein the second spatial mapping precision is lower than the first spatial mapping precision; and
displaying, on a display, a 3D image generated based on the performed 3D spatial mapping.