IP Library › Granted Patent US 11,688,146
Granted Patent B2
US 11,688,146 · App. 17/474,599 · Granted Jun 27, 2023

Electronic device and method for displaying sharing information on basis of augmented reality

Inventors: Sunyoung Yi (Gyeonggi-do, KR); Jungeun Kim (Gyeonggi-do, KR); Harin Moon (Gyeonggi-do, KR); Hangkyu Park (Gyeonggi-do, KR); Dongjun Shin (Gyeonggi-do, KR); Younghak Oh (Gyeonggi-do, KR); Yongho Choi (Gyeonggi-do, KR)
G06T19/006G01C21/343G06T7/70H04W4/029H04W4/38
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,688,146
App. No.
17/474,599
Granted
Jun 27, 2023
Kind
B2
Abstract

An electronic device includes a communication device, a camera device, a display device, a processor, and a memory connected to the processor. The memory stores instructions that, when executed, cause the processor to identify, when information on a shared object based on augmented reality is received from an external electronic device through the communication device, a first location of the shared object included in the information, identify whether the first location is included within a camera view distance and a camera direction of the camera device, generate reception information of the shared object in a specified type for each condition in which the first location is included in a range of at least one of the camera view distance or the camera direction using a second location of the electronic device as a reference point, and output an augmented reality image obtained by merging the generated reception information with a real image obtained from the camera device to the display device.

Claims (51)

1. An electronic device, comprising:

a communication device;

a camera device;

a display device;

a processor; and

a memory connected to the processor,

wherein the memory stores instructions that, when executed, cause the processor to:

identify, when information on a shared object based on augmented reality is received from an external electronic device through the communication device, a first location of the shared object included in the information,

identify whether the first location is included within a camera view distance and a camera direction of the camera device,

generate reception information of the shared object in a specified type for each condition in which the first location is included in a range of at least one of the camera view distance or the camera direction using a second location of the electronic device as a reference point, and

output an augmented reality image obtained by merging the generated reception information with a real image obtained from the camera device to the display device.

2. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to:

generate a virtual object representing the shared object as first type reception information in a direction corresponding to the first location on a real image output from the camera device when the first location is located in the camera direction of the camera device and is included within the camera view distance,

generate direction guide information that guides a movement of the electronic device in a direction in which the shared object is located as second type reception information when the first location is included within the camera view distance, but is not located in the camera direction, and

generate an image included in shared information or movement path information corresponding to the first location as third type reception information when the first location is not included within the camera view distance.

3. The electronic device of claim 2 , wherein the shared information comprises at least one of object information selected from a camera image by a user of another electronic device, location information of the selected object, a captured image of an augmented reality image to which the selected object is output, location information where an augmented reality image is captured in the other electronic device, audio or video data related to the location information, and point of interest (POI) information related to the other electronic device.

4. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to output a real image obtained by executing an augmented reality application or driving the camera device through the display device when the electronic device receives shared information based on augmented reality.

5. The electronic device of claim 2 , wherein the memory stores instructions that, when executed, further cause the processor to:

receive information on a point of interest located inside a radius set around a location of the electronic device from a server when the shared information is received, and

generate first type reception information by differently expressing at least one of a shape, color, or form of the virtual object in order to distinguish the virtual object representing the shared object from other virtual objects included in the point of interest information when the received point of interest information includes a first location of the shared object.

6. The electronic device of claim 2 , wherein the memory stores instructions that, when executed, further cause the processor to:

receive information on a point of interest located inside a radius set around a second location of the electronic device from a server when the shared information is received, and

generate first type reception information by expressing only the virtual object representing the shared object, except for an expression of other virtual objects included in the POI information when the received point of interest information includes the first location of the shared object.

7. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to:

identify the first location based on global positioning system (GPS) information included in the shared information, and

identify the second location based on GPS information obtained through a sensor module.

8. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to:

measure a distance between the first location and the second location,

convert the measured distance to a walking time of an electronic map, and

identify that the shared object is included within the camera view distance of the camera device when the walking time is within a preset value.

9. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to identify whether the shared object is out of a range of a camera direction of the camera device based on a viewing angle or an optical angle of the camera device.

10. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to:

estimate a camera view distance based on at least one of a focal length indicating a distance between an image sensor and a lens of the camera device, a viewing angle, or a lens type, or

configure a specific threshold distance to the camera view distance.

11. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to compare, when the shared information includes a captured image, the captured image with a real image obtained from the camera device to identify whether the same object exists to identify whether the shared object is out of a range of the camera direction of the camera device.

12. The electronic device of claim 1 , wherein the memory stores instructions that, when executed, further cause the processor to:

include a control item that controls reproduction of audio data or video data on the augmented reality image, and

output the augmented reality image when the shared information includes audio data or video data related to the shared object.

13. The electronic device of claim 2 , wherein the memory stores instructions that, when executed, further cause the processor to:

receive virtual object information related to the shared object from a server through the communication device, and

output the virtual object to the display device by changing a disposition direction of the shared object according to a camera direction of the camera device based on the received virtual object information.

14. A method of displaying received shared information based on augmented reality in an electronic device, the method comprising:

receiving information on a shared object based on augmented reality from an external electronic device;

identifying a first location of the shared object included in the information and a second location of the electronic device;

identifying a camera view distance and a camera direction of a camera included in the electronic device;

generating reception information of the shared object in a specified type for each condition in which the first location of the shared object is included in at least one of a camera view distance or a camera direction of the camera device using the second location as a reference point; and

outputting an augmented reality image obtained by merging the generated reception information with a real image obtained from the camera to the display device.

15. The method of claim 14 , wherein generating reception information of the shared object comprises:

generating a virtual object representing the shared object as first type reception information in a direction corresponding to the first location on a real image output from the camera device for a condition in which a first location of the shared object is located in the camera direction of the camera and is included within the camera view distance;

generating direction guide information that guides a movement of the electronic device as second type reception information in a direction in which the shared object is located for a condition in which the first location of the shared object is included within the camera view distance of the camera, but is not located in the camera direction of the camera; and

generating an image included in the shared information or movement path information corresponding to the first location as third type reception information for a condition in which the first location of the shared object is not included within the camera view distance of the camera.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2021
From: YI, SUNYOUNG; KIM, JUNGEUN; MOON, HARIN; PARK, HANGKYU; SHIN, DONGJUN; OH, YOUNGHAK; CHOI, YONGHO
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 057606/0231 →
Priority Claims (1)
KR 10-2019-0029495 · Mar 14, 2019 · national
Continuity (2)
Continuation PCTKR2020003511 · Mar 13, 2020
Related Publication 20210407211A1 · Dec 30, 2021