Augmented viewing of a scenery and subsurface infrastructure

1. A method for providing an augmented view of a real world scenery and of an occluded subsurface infrastructure, the method comprising:

taking at least one image of the real world scenery by a camera with an image sensor;

providing image reference information to the at least one image, wherein the image reference information is comprising a camera position and a camera orientation in the real world scenery at the time of taking the at least one image;

providing three dimensional information of a subsurface infrastructure; and

deriving from the three dimensional information of the subsurface infrastructure a two dimensional projection on the image sensor, wherein the two dimensional projection is made by using the reference information of the at least one image,

wherein a projection position of a two dimensional anchor element projection of at least one anchor element of the subsurface infrastructure being visible on the at least one image of the real world scenery is compared with an image position of the anchor element image of the anchor element on the at least one image, and

wherein a difference between the image position and the projection position of the same anchor element is compensated for matching and overlaying the two dimensional projection derived from the three dimensional information of the subsurface infrastructure with the at least one image and thereby providing an improved augmented view,

wherein the at least one anchor element of the subsurface infrastructure, being visible on the at least one image of the real world scenery and/or in the two dimensional projection at the image sensor, is assigned to one of several predefined anchor element categories based on a machine learned identifier and/or classifier unit, preferably comprising a deep learning e.g. with convolutional neural networks, in such a way that the anchor element can be assigned to an anchor element category independently of its construction state.

2. The method according to claim 1 , further comprising matching and overlaying the two dimensional projection derived from the three dimensional information of the subsurface infrastructure with the at least one image and thereby providing an improved augmented view comprises translating or rotating the two dimensional projection derived from the three dimensional information of the subsurface infrastructure.

3. The method according to claim 1 , wherein the matching and overlaying the two dimensional projection derived from the three dimensional information of the subsurface infrastructure with the at least one image and thereby providing an improved augmented view comprises:

improving image reference information to the at least one image by adjusting the camera position or the camera orientation in the image reference information of the real world scenery based on the difference between the image position and the projection position of the same anchor element, and

deriving from the three dimensional information of the subsurface infrastructure a two dimensional projection at the image sensor, wherein the two dimensional projection is made by using the improved reference information of the at least one image.

4. The method according to claim 3 , wherein improving image reference information to the at least one image by adjusting the camera position or the camera orientation in the real world scenery comprises:

deriving the position of the anchor element from the three dimensional information of the subsurface infrastructure, and

using the position of the anchor element as reference position for improving the camera position or the camera orientation in the real world scenery.

5. The method according to claim 1 , wherein the camera position and/or the camera orientation in the real world scenery is deduced from data of a navigation system like a navigation satellite system (GPS), a base station navigation, an accelerometer, a gyroscope, an inertial measurement unit (IMU), a solid state compass, a structure-from-motion unit, a bundle adjustment unit, a Simultaneous Localization and Mapping (SLAM) unit, or of a spatial reference recognition unit.

6. A mobile augmented reality viewing device configured to execute a method according to claim 1 .

7. A computer program product with program code being stored on a non-transitory machine readable medium, the program code being configured for the execution of at least one of the methods according to claim 1 .

8. A method for providing an augmented view of a real world scenery and of an occluded subsurface infrastructure, the method comprising:

taking at least one image of the real world scenery by a camera with an image sensor;

providing image reference information to the at least one image, wherein the image reference information is comprising a camera position and a camera orientation in the real world scenery at the time of taking the at least one image;

providing three dimensional information of a subsurface infrastructure; and

deriving from the three dimensional information of the subsurface infrastructure a two dimensional projection on the image sensor, wherein the two dimensional projection is made by using the reference information of the at least one image,

wherein a projection position of a two dimensional anchor element projection of at least one anchor element of the subsurface infrastructure being visible on the at least one image of the real world scenery is compared with an image position of the anchor element image of the anchor element on the at least one image, and

wherein a difference between the image position and the projection position of the same anchor element is compensated for matching and overlaying the two dimensional projection derived from the three dimensional information of the subsurface infrastructure with the at least one image and thereby providing an improved augmented view, and

wherein the at least one anchor element of the subsurface infrastructure is visible on the at least one image of the real world scenery is assigned to one of several predefined anchor element categories with corresponding position and shape characteristics.

9. The method according to claim 8 , wherein the shape characteristics include shape information in different scales and/or in different projections.

10. The method according to claim 8 , wherein the position and shape characteristics of the predefined anchor element categories comprise a central position and a shape information, wherein the shape information is comprising at least one point located at a circumference line of the anchor element.

11. The method according to claim 8 , wherein:

the position and shape characteristics of at least one anchor element of the subsurface infrastructure are used to determine a projection position and an image position of the at least one anchor element on the two dimensional projection of the image sensor and on the at least one image, respectively,

the difference between the image position and the projection position is used for providing a matched two dimensional projection, only if the difference between the image position and the projection position is below a predefined maximum difference.

12. The method according to claim 8 , wherein:

the position and shape characteristics of at least one anchor element of the subsurface infrastructure are used to determine a projection orientation and an image orientation of the at least one anchor element on the two dimensional projection of the image sensor and on the at least one image with corresponding position and shape characteristics,

the difference between the image orientation and the projection orientation is used for providing a matched two dimensional projection, only if the difference between the image orientation and the projection orientation is below a predefined maximum difference.

13. The method according to claim 11 , wherein determining the projection position of the anchor element, the projection orientation, the image position, or image orientation comprises edge extraction and fitting of the extracted edge information to shape characteristics of one of the predefined anchor element categories.

14. The method according to claim 11 , wherein determining the projection position of the anchor element, the projection orientation, the image position, or image orientation comprises least-squares template matching.