IP Library › Granted Patent US 11,592,311
Granted Patent B2
US 11,592,311 · App. 16/795,400 · Granted Feb 28, 2023

Method and apparatus for displaying surrounding information using augmented reality

Inventors: Jinwoo Yoo (Suwon-si, KR); Sanghyuk Koh (Suwon-si, KR); Seunghyun Lee (Suwon-si, KR); Donghoon Kim (Suwon-si, KR); Jooyoung Son (Suwon-si, KR); Dasom Lee (Suwon-si, KR); Yonggu Lee (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
G01C21/3679G01C21/3647G06Q30/0261G06T7/74G06T19/006G06V20/20G06T2207/20101
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,592,311
App. No.
16/795,400
Filed
Feb 19, 2020
Granted
Feb 28, 2023
Kind
B2
Examiner
GRAY, RYAN M
Art Unit
2611
USPC
701/426
Abstract

A method of displaying a nearby point of interest (POI) using augmented reality and an electronic device thereof. An operation method of an electronic device includes: acquiring a preview video; acquiring location information of the electronic device; acquiring information on at least one POI located in a vicinity; selecting a focus POI among at least one POI; determining first information to be provided to a user for general POIs; determining second information including more information than the first information to be provided to the user for the focus POI; determining a location on a screen on which the preview video in which the information on the at least one POI is to be displayed; and displaying the first information or the second information on the location on the screen on which the determined preview video of all or some of the at least one POI is provided.

Claims (91)

1. An electronic device comprising:

a communication module configured to perform communication with an external device;

a camera module configured to capture a still image and a video;

at least one processor configured to be operatively connected to the communication module and the camera module; and

at least one memory configured to be operatively connected to the at least one processor,

wherein the at least one memory stores instructions that, when executed, cause the at least one processor to:

acquire a preview video captured by the camera module by controlling the camera module;

acquire location information of the electronic device;

acquire information on at least one point of interest (POI) located in a vicinity based on the location information;

select a focus POI among at least one POI based on information on the at least one POI;

determine first information to be provided to a user for general POIs among the at least one POI, the general POIs including remaining POIs except the selected focus POI;

determine second information to be provided to the user for the focus POI, wherein the second information includes more information than the first information;

determine a location on a screen on which a preview video in which the information on the at least one POI to be provided is to be displayed; and

display the first information or the second information on the determined location on the screen on which a determined preview video of all or some of the at least one POI is provided,

wherein the instructions further cause the at least one processor to:

configure POIs whose distances between each other are within a predetermined distance as a POI group based on location information of each POI included in the information on the at least one POI or a location on a screen on which the information on the at least one POI is to be displayed;

select a representative POI among the POIs within the POI group based on the information on the at least one POI or a user's preference; and

display information on the representative POI to the location on the screen of the representative POI on behalf of the POI group.

2. The electronic device of claim 1 , wherein the instructions further cause the at least one processor to:

select a POI including advertisement and/or event information in the information on the at least one POI among the at least one POI as an advertisement POI;

determine third information to be provided to the user for the advertisement POI, wherein the third information includes the advertisement and/or event information; and

display the third information on the location on the screen on which a determined preview video is provided for the advertisement POI.

3. The electronic device of claim 1 , wherein the instructions further cause the at least one processor to:

select some POIs to be displayed on the screen on which the preview video is provided among the at least one POI based on a direction in which the camera module captures the preview video and an angle of view of the camera module; and

display the first information or the second information on the location on the screen on which a determined preview video of each POI of the selected some POIs is provided.

4. The electronic device of claim 1 , wherein the instructions further cause the at least one processor to:

configure a condition corresponding to the focus POI; and

select a POI in which the information on the at least one POI satisfies the condition corresponding to the focus POI as the focus POI.

5. The electronic device of claim 4 , wherein the instructions further cause the at least one processor to:

configure a partial region in the preview video as a focus region; and

configure a POI of which actual location is included in the focus region as the condition corresponding to the focus POI.

6. The electronic device of claim 4 , wherein the instructions further cause the at least one processor to:

determine a distance between the electronic device and each POI based on the location information of the electronic device and location information of each POI included in the information on the at least one POI as the condition corresponding to the focus POI; and

configure a POI in which the distance between the electronic device and the POI is within a predetermined value based on the determined distance between the electronic device and each POI.

7. The electronic device of claim 4 , wherein the instructions further cause the at least one processor to configure a POI having a high preference based on the user's preference configured based on a user's behavior including a POI where the user has previously visited or a POI on which the user has previously searched for information as the condition corresponding to the focus POI.

8. The electronic device of claim 1 , wherein the instructions further cause the at least one processor to:

designate a portion on the screen on which the preview video is provided as a focus region; and

display to provide the second information for the focus POI within the focus region.

9. The electronic device of claim 1 , wherein the instructions further cause the at least one processor to:

determine the first information to be a category of the POI;

determine the second information to include at least one of a POI-related image, a name, or a distance from a current location of the electronic device;

display an icon corresponding to the category to provide the first information; and

display a user interface (UI) including the second information to provide the second information.

10. An operation method of an electronic device comprising:

acquiring a preview video captured by a camera module;

acquiring location information of the electronic device;

acquiring information on at least one point of interest (POI) located in a vicinity based on the location information;

selecting a focus POI among at least one POI based on information on the at least one POI;

determining first information to be provided to a user for general POIs among the at least one POI, the general POIs including remaining POIs except the selected focus POI;

determining second information including more information than the first information to be provided to the user for the focus POI;

determining a location on a screen on which a preview video in which the information on the at least one POI to be provided is to be displayed; and

displaying the first information or the second information on the determined location on the screen on which a determined preview video of all or some of the at least one POI is provided,

wherein the operation method further comprising:

configuring POIs whose distances between each other are within a predetermined distance as a POI group based on location information of each POI included in the information on the at least one POI or a location on a screen on which the information on the at least one POI is to be displayed; and

selecting a representative POI among the POIs within the POI group based on the information on the at least one POI or a user's preference, and

wherein the displaying to provide the first information or the second information includes displaying to provide information on the representative POI to the location on the screen of the representative POI on behalf of the POI group.

11. The operation method of claim 10 , further comprising:

selecting a POI including advertisement and/or event information in the information on the at least one POI among the at least one POI as an advertisement POI;

determining third information to be provided to the user for the advertisement POI, wherein the third information includes the advertisement and/or event information; and

displaying the third information on the location on the screen on which a determined preview video is provided for the advertisement POI.

12. The operation method of claim 10 , wherein the displaying to provide the first information or the second information comprises:

selecting some POIs to be displayed on the screen on which the preview video is provided among the at least one POI based on a direction in which the camera module captures the preview video and an angle of view of the camera module; and

displaying the first information or the second information on the location on the screen on which a determined preview video of each POI of the selected some POIs is provided.

13. The operation method of claim 10 , wherein the selecting of the focus POI among at least one POI comprises:

configuring a partial region in the preview video as a focus region; and

configuring a POI of which actual location is included in the focus region as the focus POI.

14. The operation method of claim 10 , wherein the selecting of the focus POI among at least one POI comprises:

determining a distance between the electronic device and each POI based on the location information of the electronic device and location information of each POI included in the information on the at least one POI; and

selecting a POI in which the distance between the electronic device and the POI is within a predetermined value as the focus POI based on the determined distance between the electronic device and each POI.

15. The operation method of claim 10 , wherein the determining of the location on the screen comprises:

designating a portion on the screen on which the preview video is provided as a focus region; and

determining the location within the designated focus region using the location on the screen on which the preview video in which the information on the focus POI is to be displayed is provided.

16. The operation method of claim 10 , wherein:

the first information is a category of the POI;

the second information may include at least one of a POI-related image, a name, or a distance from a current location of the electronic device, and

wherein the displaying to provide the first information or the second information comprises:

displaying an icon corresponding to the category as a display for providing the first information; and

displaying a user interface (UI) including the second information as a display for providing the second information.

17. An operation method of an electronic device including a screen comprising:

transmitting location information of the electronic device to a content provider (CP) server;

acquiring information on at least one point of interest (POI) located in a vicinity from the CP server;

arranging the at least one POI according to a distance from the electronic device based on the location information of the electronic device;

configuring a display position of each POI of the at least one POI based on a location of the electronic device and a preview video capturing direction of the electronic device;

configuring, when the display position is a focus POI within a configured focus region, information to be displayed in relation to the focus POI and displaying a user interface (UI) including the configured information at the configured display position on the screen on which the preview video is displayed;

acquiring, when the display position is a general POI which is out of the configured focus region, category information of the general POI and displaying an icon corresponding to the category information at the configured display position on the screen on which the preview video is displayed;

determining whether a POI group including at least one POI displayed as one display on the preview video can be generated;

generating, when it is determined that the POI group can be generated, the POI group; and

selecting a representative POI for each POI group based on the information on the at least one POI or a user's preference.

18. The operation method of claim 17 ,

wherein the displaying of the UI including the configured information comprises displaying the UI including the configured information of the representative POI at the configured display position of the representative POI on the screen on which the preview video is displayed, and

wherein the displaying of the icon corresponding to the category information comprises displaying the icon corresponding to the category information of the representative POI at the configured display position of the representative POI on the screen on which the preview video is displayed.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2020
From: YOO, JINWOO; KOH, SANGHYUK; LEE, SEUNGHYUN; KIM, DONGHOON; SON, JOOYOUNG; LEE, DASOM; LEE, YONGGU
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 051864/0217 →
Priority Claims (1)
KR 10-2019-0019172 · Feb 19, 2019 · national
Continuity (1)
Related Publication 20200264007A1 · Aug 20, 2020
Cited By (4)
US 12,223,608 US 12,265,586 US 12,322,046 US 12,561,651