IP Library Granted Patent US 10,916,056
Granted Patent B2
US 10,916,056 · App. 14/923,293 · Granted Feb 9, 2021

Method of displaying virtual information in a view of a real environment

Inventor: Peter Meier (Munich, DE)
Assignee: Apple Inc.
G06T19/006G02B27/017G06F3/011G06F3/14G02B2027/014G02B2027/0138G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,916,056
App. No.
14/923,293
Granted
Feb 9, 2021
Kind
B2
Abstract

A method of displaying virtual information in a view of a real environment comprising the following steps: providing the system relative to at least one part of the real environment and providing accuracy information of the current pose, providing multiple pieces of virtual information, and assigning a respective one of the pieces of virtual information to one of different parameters indicative of different pose accuracy information, and displaying at least one of the pieces of virtual information in the view of the real environment according to the accuracy information of the current pose in relation to the assigned parameter.

Claims (50)

1. A method of displaying virtual information in a view of a real environment, comprising:

determining a current pose of a system that displays virtual information in a real environment, wherein the current pose is determined relative to at least one part of the real environment;

determining an uncertainty value for the current pose indicative of an accuracy of a location of the pose;

obtaining, based on the current pose, a plurality of points of interest and an uncertainty parameter for each of the plurality of points of interest, wherein the uncertainty parameter indicates an accuracy of a location of a corresponding point of interest; and

determining a display method for each of the plurality of points of interest based on the uncertainty value and each point of interest's corresponding uncertainty parameter.

2. The method of claim 1 , wherein at least one of the determined display methods results in a point of interest not being displayed.

3. The method of claim 1 , wherein the display method for each of the plurality of points of interest is selected from a group including a 3D model and an image overlay.

4. The method of claim 1 , wherein determining a display method comprises:

determining, for a particular point of interest, whether the uncertainty value satisfies an uncertainty threshold associated with the particular point of interest, and

displaying the particular point of interest in a view of the real environment when the uncertainty value is determined to satisfy the uncertainty threshold for the particular point of interest.

5. The method of claim 1 , wherein determining a display method comprises:

determining a current distance from the system to a particular point of interest, and

displaying the particular point of interest when the current distance satisfies a distance parameter associated with the particular point of interest.

6. The method of claim 1 , wherein the uncertainty value is based on whether the current pose is determined using an optical pose determination method.

7. The method of claim 1 , wherein the uncertainty parameter for each of the plurality of points of interest is one of a plurality of uncertainty parameters for each of the plurality of points of interest, and

wherein determining the display method comprises determining the display method for each of the plurality of points of interest based on the uncertainty value and each point of interest's corresponding plurality of uncertainty parameters.

8. A computer readable medium for displaying virtual information in a view of a real environment, comprising computer readable code executable by a processor to:

determine a current pose of a system that displays virtual information in a real environment, wherein the current pose is determined relative to at least one part of the real environment;

determine an uncertainty value for the current pose indicative of an accuracy of a location of the pose;

obtain, based on the current pose, a plurality of points of interest and an uncertainty parameter for each of the plurality of points of interest; and

determine a display method for each of the plurality of points of interest based on the uncertainty value and each point of interest's corresponding uncertainty parameter, wherein the uncertainty parameter indicates an accuracy of a location of a corresponding point of interest.

9. The computer readable medium of claim 8 , wherein at least one of the determined display methods results in a point of interest not being displayed.

10. The computer readable medium of claim 8 , wherein the display method for each of the plurality of points of interest is selected from a group including a 3D model and an image overlay.

11. The computer readable medium of claim 8 , wherein the computer readable code to determine a display method comprises computer readable code to:

determine, for a particular point of interest, whether the uncertainty value satisfies an uncertainty threshold associated with the particular point of interest, and

display the particular point of interest in a view of the real environment when the uncertainty value is determined to satisfy the uncertainty threshold for the particular point of interest.

12. The computer readable medium of claim 8 , wherein the computer readable code to determine a display method comprises computer readable code to:

determine a current distance from the system to a particular point of interest, and

display the particular point of interest when the current distance satisfies a distance parameter associated with the particular point of interest.

13. The computer readable medium of claim 8 , wherein the uncertainty value is based on whether the current pose is determined using an optical pose determination method.

14. The computer readable medium of claim 8 , wherein the uncertainty parameter for each of the plurality of points of interest is one of a plurality of uncertainty parameters for each of the plurality of points of interest, and

wherein the computer readable code to determine the display method comprises computer readable code to determine the display method for each of the plurality of points of interest based on the uncertainty value and each point of interest's corresponding plurality of uncertainty parameters.

15. A system for displaying virtual information in a view of a real environment, the system comprising:

one or more processors; and

a memory comprising computer readable code executable by the one or more processors to:

determine a current pose of a system that displays virtual information in a real environment, wherein the current pose is determined relative to at least one part of the real environment;

determine an uncertainty value for the current pose indicative of an accuracy of a location of the current pose;

obtain, based on the current pose, a plurality of points of interest and an uncertainty parameter for each of the plurality of points of interest; and

determine a display method for each of the plurality of points of interest based on the uncertainty value and each point of interest's corresponding uncertainty parameter, wherein the uncertainty parameter indicates an accuracy of a location of a corresponding point of interest.

16. The system of claim 15 , wherein at least one of the determined display methods results in a point of interest not being displayed.

17. The system of claim 15 , wherein the display method for each of the plurality of points of interest is selected from a group including a 3D model and an image overlay.

18. The system of claim 15 , wherein the computer readable code to determine a display method comprises computer readable code to:

determine, for a particular point of interest, whether the uncertainty value satisfies an uncertainty threshold associated with the particular point of interest, and

display the particular point of interest in a view of the real environment when the uncertainty value is determined to satisfy the uncertainty threshold for the particular point of interest.

19. The system of claim 15 , wherein the computer readable code to determine a display method comprises computer readable code to:

determine a current distance from the system to a particular point of interest, and

display the particular point of interest when the current distance satisfies a distance parameter associated with the particular point of interest.

20. The system of claim 15 , wherein the uncertainty value is based on whether the current pose is determined using an optical pose determination method.

21. The system of claim 15 , wherein the uncertainty parameter for each of the plurality of points of interest is one of a plurality of uncertainty parameters for each of the plurality of points of interest, and

wherein the computer program code to determine the display method comprises computer readable code to determine the display method for each of the plurality of points of interest based on the uncertainty value and each points of interest's corresponding plurality of uncertainty parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2016
From: METAIO GMBH
To: APPLE INC.
Reel/Frame 040821/0462 →
Priority Claims (1)
EP 10155026 · Mar 1, 2010 · regional
Continuity (2)
Continuation 13582314
Related Publication 20160210786A1 · Jul 21, 2016