Granted Patent B2

US 9,558,559 · App. 13/857,688 · Granted Jan 31, 2017

Abstract

Claims (56)

1. A method comprising:

processing one or more images to determine camera location information and camera pose information associated with at least one camera capturing the one or more images, wherein the camera location information and the camera pose information are represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system;

processing the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to a local 3D Cartesian coordinate system;

matching the local three-dimensional point cloud represented according to the local 3D Cartesian coordinate system to one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine the camera location information and the camera pose information according to the ECEF 3D Cartesian coordinate system;

associating the camera location information and the camera pose information represented according to the ECEF Cartesian coordinate system with the one or more images as meta-data information to form an augmented content; and

rendering the augmented content, based on the associated camera location information and the camera pose information,

wherein the rendering of the augmented content is more accurate than a related rendering based upon rough sensor data of the one or more images.

2. A method of claim 1 , further comprising:

mapping the local three-dimensional point cloud to the ECEF Cartesian coordinate system based, at least in part, on the matching.

3. A method of claim 1 , further comprising:

determining the one or more reference point clouds based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera.

4. A method of claim 1 , further comprising:

determining overlay information based, at least in part, on the camera location information and the camera pose information; and

a rendering of the overlay information in the one or more images.

5. A method of claim 4 , wherein the overlay information includes, at least in part, point-of-interest information, social messaging information, location-based service information, or a combination thereof.

6. A method of claim 1 , wherein the one or more images are part of at least one video sequence, the method further comprising:

selecting the one or more images from among one or more frames of the at least one video sequence, wherein the one or more images represent, at least in part, a subset of the one or more frames.

7. A method of claim 1 , further comprising:

determining one or more panoramic images based, at least in part, on the camera location information and the camera pose information;

determining field of view information for the one or more images based, at least in part, on the camera location information and the camera pose information; and

causing, at least in part, an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

8. A method of claim 1 , further comprising:

determining one or more other images associated with other camera location information and other camera pose information based, at least in part, on the camera location information and the camera pose information; and

associating the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

9. A method of claim 8 , wherein the one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein one or more contextual criteria include, at least in part, one or more temporal criteria.

10. A method of claim 1 , wherein the local 3D Cartesian coordinate system has an origin point on the Earth's surface and covers a limited space; and wherein the ECEF Cartesian coordinate system has an origin point at the center of the mass of the Earth.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

process and/or facilitate a processing of one or more images to determine camera location information and camera pose information associated with at least one camera capturing the one or more images, wherein the camera location information and the camera pose information are represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system,

process and/or facilitate a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to a local 3D Cartesian coordinate system,

match the local three-dimensional point cloud represented according to the local 3D Cartesian coordinate system to one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine the camera location information and the camera pose information according to the ECEF 3D Cartesian coordinate system, and

associate the camera location information and the camera pose information represented according to the ECEF Cartesian coordinate system with the one or more images as meta-data information to form an augmented content, and

cause, at least in part, a rendering of the augmented content, based on the associated camera location information and the camera pose information,

wherein the rendering of the augmented content is more accurate than a related rendering based upon rough sensor data of the one or more images.

12. An apparatus of claim 11 , wherein the apparatus is further caused to:

map the local three-dimensional point cloud to the ECEF Cartesian coordinate system based, at least in part, on the matching.

13. An apparatus of claim 11 , wherein the apparatus is further caused to:

determine the one or more reference point clouds based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera.

14. An apparatus of claim 11 , wherein the apparatus is further caused to:

determine overlay information based, at least in part, on the camera location information and the camera pose information, and

cause, at least in part, a rendering of the overlay information in the one or more images.

15. An apparatus of claim 14 , wherein the overlay information includes, at least in part, point-of-interest information, social messaging information, location-based service information, or a combination thereof.

16. An apparatus of claim 11 , wherein the one or more images are part of at least one video sequence, and wherein the apparatus is further caused to:

select the one or more images from among one or more frames of the at least one video sequence,

wherein the one or more images represent, at least in part, a subset of the one or more frames.

17. An apparatus of claim 11 , wherein the apparatus is further caused to:

determine one or more panoramic images based, at least in part, on the camera location information and the camera pose information,

determine field of view information for the one or more images based, at least in part, on the camera location information and the camera pose information, and

cause, at least in part, an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

18. An apparatus of claim 11 , wherein the apparatus is further caused to:

determine one or more other images associated with other camera location information and other camera pose information based, at least in part, on the camera location information and the camera pose information, and

cause, at least in part, an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

19. An apparatus of claim 18 , wherein the one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein one or more contextual criteria include, at least in part, one or more temporal criteria.

20. An apparatus of claim 11 , wherein the local 3D Cartesian coordinate system has an origin point on the Earth's surface and covers a limited space; and wherein the ECEF Cartesian coordinate system has an origin point at the center of the mass of the Earth.

Assignments (12)