IP Library Granted Patent US 8,837,779
Granted Patent B2
US 8,837,779 · App. 13/502,959 · Granted Sep 16, 2014

Method for determining the pose of a camera and for recognizing an object of a real environment

Inventor: Peter Meier (Munich, DE)
Assignee: Metaio GmbH
G06T7/004G06T7/2033G06T9/3216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,837,779
App. No.
13/502,959
Granted
Sep 16, 2014
Kind
B2
Abstract

A method for determining the pose of a camera ( 22, 32 ) relative to a real environment ( 40 ) comprises the following steps: taking at least one image ( 50 ) of a real environment by means of a camera ( 22, 32 ), the image containing at least part of a real object ( 41 ), performing a tracking method that evaluates information with respect to correspondences between features associated with the real object ( 41 ) and corresponding features of the real object ( 41 ) as it is contained in the image ( 50 ) of the real environment, so as to obtain conclusions about the pose of the camera ( 22, 32 ), determining at least one parameter of an environmental situation, and performing the tracking method in accordance with the at least one parameter. Analogously, the method can also be utilized in a method for recognizing an object of a real environment in an image taken by a camera.

Claims (35)

1. A method for determining the pose of a camera relative to a real environment, said method comprising the following steps:

taking at least one image of a real environment by means of a camera, the image containing at least part of a real object;

determining at least one parameter of an environmental situation;

performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera; and

performing the tracking method in accordance with the at least one parameter;

wherein calculation of an uncertainty is effected in determining the camera pose by determination of a distance measure of the at least one parameter of the environmental situation and known ideal parameters of the tracking method with respect to a known environmental situation.

2. The method of claim 1 , wherein the tracking method makes use of at least one optical tracking method that is initialized in accordance with the at least one parameter.

3. The method of claim 1 , wherein by means of the tracking method the pose of the camera is determined with one, two, three, four, five or six degrees of freedom.

4. The method of claim 1 , wherein the tracking method employs several tracking sensors that are different with respect to the sensor type.

5. The method of claim 4 , wherein the tracking method includes weighting of at least one of the tracking sensors in accordance with the at least one parameter.

6. The method of claim 1 , wherein the tracking method includes prioritization of method partial steps in accordance with the at least one parameter.

7. The method of claim 1 , wherein a data model is employed that is compared with data of the image taken by the camera, the data model being selected in consideration of the at least one parameter of the environmental situation.

8. The method of claim 1 , wherein, for determining the at least one parameter of the environmental situation, at least one or more of the following items of information are used: time of the day; time of the year; weather, in particular rain, clouds, sun and fog; position of the moon; snow conditions; foliage of the trees; altitude above sea level; public events, in particular strong movements; traffic situation; position; orientation in a world coordinate system; histogram of the image; temperature; maintenance step.

9. The method of claim 1 , wherein the at least one parameter has an effect on minimum requirements with respect to successful retrieval of model data in the image, in particular determines a number of features associated with the real object, which are to be brought info into conformity with corresponding features of the image of the real object in the image of the real environment.

10. The method of claim 1 , wherein a data model is used that is compared with data of the image of the real object ( 41 ) in the image of the real environment; and

wherein the at least one parameter has influence on the preparation and/or use of the data model.

11. The method of claim 10 , wherein a parameter vector per data model or per feature within the data model is stored.

12. The method of claim 10 , wherein the at least one parameter influences the kind of preparation of the data model, in particular influences the preparation of feature detectors and descriptors.

13. The method of claim 1 , wherein a data model is extracted from a number of individual models, said data model containing an intersection of information present in several environmental situations.

14. The method of claim 1 , wherein several partial systems respectively perform the method, wherein not each of the partial systems calculates a tracking situation itself, but finished situation information is made available to all partial systems, advantageously even a finished recognition or initialization model is made available to the partial systems.

15. The method of claim 14 , wherein the partial systems provide information on the environmental situation to a server provided or to a peer-to-peer process.

16. The method of claim 1 , wherein, by determining the camera pose, depth information with respect to at least one real object contained in the image is calculated, the depth information being used to blend in, in a display device, an occlusion model for occluding a real object, when virtual information superimposed on the real environment is occluded in the view by the real object to be occluded.

17. A method for determining the pose of a camera relative to a real environment, said method comprising the following steps:

taking at least one image of a real environment by means of a camera, the image containing at least part of a real object;

determining at least one parameter of an environmental situation;

performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera; and

performing the tracking method in accordance with the at least one parameter;

wherein the at least one parameter has an effect on minimum requirements with respect to successful retrieval of model data in the image, in particular determines a number of features associated with the real object, which are to be brought into conformity with corresponding features of the image of the real object in the image of the real environment.

18. The method of claim 17 , wherein calculation of an uncertainty is effected in determining the camera pose by determination of a distance measure of the at least one parameter of the environmental situation and known ideal parameters of the tracking method with respect to a known environmental situation.

19. A method for determining the pose of a camera relative to a real environment, said method comprising the following steps:

taking at least one image of a real environment by means of a camera, the image containing at least part of a real object;

determining at least one parameter of an environmental situation;

performing a tracking method that evaluates information with respect to correspondences between features associated with the real object and corresponding features of the real object as it is contained in the image of the real environment, so as to obtain conclusions about the pose of the camera; and

performing the tracking method in accordance with the at least one parameter;

wherein, by determining the camera pose, depth information with respect to at least one real object contained in the image is calculated, the depth information being used to blend in, in a display device, an occlusion model for occluding a real object, when virtual information superimposed on the real environment is occluded in the view by the real object to be occluded.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2016
From: METAIO GMBH
To: APPLE INC.
Reel/Frame 040821/0462 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 14, 2012
From: MEIER, PETER
To: METAIO GMBH
Reel/Frame 028204/0446 →
Priority Claims (1)
DE 10 2009 049 849 · Oct 19, 2009 · national
Continuity (1)
Related Publication 20120237085A1 · Sep 20, 2012