IP Library Granted Patent US 9,092,897
Granted Patent B2
US 9,092,897 · App. 13/572,131 · Granted Jul 28, 2015

Method and apparatus for displaying interface elements

Inventors: Mark Fulks (Danville, CA); Gabriel Nava-Vazquez (Campbell, CA); Pankaj Jain (San Jose, CA); Sreejumon Kundilepurayil (Cupertino, CA); Andre Tadeu Santos Fialho (Berlin, DE); Christophe Marle (San Francisco, CA); Kenneth Walker (Concord, CA); Adetokunbo Bamidele (Dublin, CA); Franklin Ashley Lyons (San Antonio, TX)
Assignee: Here Global B.V.
G06T19/006G06F3/147G09G3/003G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,092,897
App. No.
13/572,131
Granted
Jul 28, 2015
Kind
B2
Abstract

A method, apparatus and computer program product are provided to provide an efficient, aesthetically pleasing display of points of interest in an AR interface that maximizes usability and display efficiency. In this regard, the method, apparatus and computer program product may utilize a mobile terminal to perform pre-processing of interface elements to reduce display clutter and increase efficiency of display processing. Interface elements may be projected onto a cylindrical surface to locate the interface elements relative to the mobile terminal. Three dimensional coordinates for the interface elements may be derived from the cylindrical projection. A perspective grid may provide a user of the device with a frame of reference to interact with the display and interface elements displayed thereon. Display elements may further conform to input operations performed by a user to interact with the AR interface.

Claims (66)

1. A method comprising:

determining a first location, the first location corresponding to the location of a mobile terminal;

receiving a second location, the second location corresponding to the location of a point of interest;

determining, using a processor, a bearing angle and a distance from the mobile terminal to the point of interest using the first location and the second location;

projecting the point of interest onto a cylinder using the bearing and distance;

determining a set of three dimensional coordinates in a Cartesian space for the point of interest relative to the mobile terminal using the projected point of interest;

causing the point of interest to be displayed as an interface element of an augmented reality interface using the set of three dimensional coordinates in a Cartesian space;

segmenting the augmented reality interface into at least a foreground region and a background region;

determining whether the point of interest is displayed in the foreground region or the background region at least in part based on the distance from the mobile terminal to the point of interest, such that the point of interest is displayed in the background region if the distance is greater than a threshold;

applying a first interface effect to the point of interest in response to the point of interest being displayed in the foreground region; and

applying a second interface effect different from the first interface effect to the point of interest in response to the point of interest being displayed in the background region.

2. The method of claim 1 , further comprising:

determining a heading of the mobile terminal; and

using the heading to determine whether the point of interest is visible in a display of the mobile terminal prior to displaying the point of interest.

3. The method of claim 1 , further comprising:

determining one or more view port settings for displaying a portion of the cylinder on a display of the mobile terminal; and

using the one or more view port settings to display a portion of the cylinder including the point of interest on the display.

4. The method of claim 3 , wherein the view port settings comprise at least one of a view port width in pixels, a view port height in pixels, a horizontal view port camera angle of view, and a vertical view port camera angle of view.

5. The method of claim 3 , wherein the view port settings define the portion of the cylinder that corresponds to the viewing angle of a camera coupled to the mobile terminal.

6. The method of claim 1 , further comprising scaling the interface element based on the distance from the mobile terminal to the point of interest associated with the interface element.

7. The method of claim 1 , further comprising:

determining a horizon based on input from one or more attitude sensors coupled to the mobile terminal; and

adjusting display of the point of interest based on the location of the point of interest relative to the horizon.

8. The method of claim 1 , wherein the height of the point of interest on the cylinder is based on the distance between the point of interest and the mobile terminal.

9. An apparatus comprising at least one processor and at least one memory including computer program instructions, the at least one memory and the computer program instructions configured to, with the at least one processor, cause the apparatus at least to:

determine a first location;

receive a second location, the second location corresponding to the location of a point of interest;

determine a bearing angle and a distance from the apparatus to the point of interest using the first location and the second location;

project the point of interest onto a cylinder using the bearing and distance;

determine a set of three dimensional coordinates in a Cartesian space for the point of interest relative to the apparatus using the projected point of interest;

cause the point of interest to be displayed as an interface element of an augmented reality interface using the set of three dimensional coordinates in a Cartesian space

segment the augmented reality interface into at least a foreground region and a background region;

determine whether the point of interest is displayed in the foreground region or the background region at least in part based on the distance from the mobile terminal to the point of interest, such that the point of interest is displayed in the background region if the distance is greater than a threshold;

apply a first interface effect to the point of interest in response to the point of interest being displayed in the foreground region; and

apply a second interface effect different from the first interface effect to the point of interest in response to the point of interest being displayed in the background region.

10. The apparatus of claim 9 , further configured to:

determine a heading of the apparatus; and

determine, using the heading, whether the point of interest is visible in a display of the apparatus prior to displaying the point of interest.

11. The apparatus of claim 9 , wherein the apparatus is further configured to:

determine one or more view port settings for displaying a portion of the cylinder on a display of the apparatus; and

use the one or more view port settings to display a portion of the cylinder including the point of interest on the display.

12. The apparatus of claim 11 , wherein the view port settings comprise at least one of a view port width in pixels, a view port height in pixels, a horizontal view port camera angle of view, and a vertical view port camera angle of view.

13. The apparatus of claim 11 , wherein the view port settings define the portion of the cylinder that corresponds to the viewing angle of a camera coupled to the apparatus.

14. The apparatus of claim 9 , wherein the apparatus is further configured to scale the interface element based on the distance from the apparatus to the point of interest associated with the interface element.

15. The apparatus of claim 9 , wherein the apparatus further comprises one or more attitude sensors and wherein the apparatus is further configured to:

determine a horizon based on input from one or more attitude sensors coupled to the mobile terminal; and

adjust display of the point of interest based on the location of the point of interest relative to the horizon.

16. The apparatus of claim 9 , wherein the height of the point of interest on the cylinder is based on the distance between the point of interest and the apparatus.

17. A computer program product comprising at least one non-transitory computer-readable storage medium bearing computer program instructions embodied therein for use with a computer, the computer program instructions comprising program instructions configured to:

determine a first location, the first location corresponding to the location of a mobile terminal;

receive a second location, the second location corresponding to the location of a point of interest;

determine a bearing angle and a distance from the mobile terminal to the point of interest using the first location and the second location;

project the point of interest onto a cylinder using the bearing and distance;

determine a set of three dimensional coordinates in a Cartesian space for the point of interest relative to the mobile terminal using the projected point of interest;

cause the point of interest to be displayed as an interface element of an augmented reality interface using the set of three dimensional coordinates in a Cartesian space;

segment the augmented reality interface into at least a foreground region and a background region;

determine whether the point of interest is displayed in the foreground region or the background region at least in part based on the distance from the mobile terminal to the point of interest, such that the point of interest is displayed in the background region if the distance is greater than a threshold;

apply a first interface effect to the point of interest in response to the point of interest being displayed in the foreground region; and

apply a second interface effect different from the first interface effect to the point of interest in response to the point of interest being displayed in the background region.

18. The computer program product of claim 17 , wherein the program instructions are further configured to:

determine a heading of the mobile terminal; and

use the heading to determine whether the point of interest is visible in a display of the mobile terminal prior to displaying the point of interest.

19. The computer program product of claim 17 , wherein the program instructions are further configured to:

determine one or more view port settings for displaying a portion of the cylinder on a display of the mobile terminal; and

use the one or more view port settings to display a portion of the cylinder including the point of interest on the display.

20. The computer program product of claim 19 , wherein the view port settings comprise at least one of a view port width in pixels, a view port height in pixels, a horizontal view port camera angle of view, and a vertical view port camera angle of view.

Assignments (3)
CHANGE OF NAME Recorded Sep 26, 2014
From: NAVTEQ B.V.
To: HERE GLOBAL B.V.
Reel/Frame 033830/0681 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2012
From: FULKS, MARK; NAVA-VAZQUEZ, GABRIEL; JAIN, PANKAJ; KUNDILEPURAYIL, SREEJUMON; FIALHO, ANDRE TADEU SANTOS; MARLE, CHRISTOPHE; WALKER, KENNETH; BAMIDELE, ADETOKUNBO; LYONS, FRANKLIN ASHLEY
To: NOKIA CORPORATION
Reel/Frame 029175/0202 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2012
From: NOKIA CORPORATION
To: NAVTEQ B.V.
Reel/Frame 029101/0516 →
Continuity (1)
Related Publication 20140043322A1 · Feb 13, 2014