Granted Patent B2

US 9,699,375 · App. 14/015,610 · Granted Jul 4, 2017

Abstract

Claims (61)

1. A method comprising facilitating a processing of, or processing, (1) data, (2) information, or (3) at least one signal, the (1) data, (2) information, (3) at least one signal based, at least in part, on the following:

a selection of one or more images from among one or more frames of at least one video sequence, wherein the one or more images represent, at least in part, a subset of the one or more frames;

a processing of the one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a local three-dimensional (3D) Cartesian coordinate system;

a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to the local 3D Cartesian coordinate system;

at least one determination of one or more reference point clouds represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera;

a processing of at least one of the one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine a correspondence of the local point cloud represented in the local 3D Cartesian coordinate system, wherein the correspondence maps the camera location information and the camera pose information from the local 3D Cartesian coordinate system to the ECEF 3D Cartesian coordinate system;

an association of the camera location information or the camera pose information represented according to the ECEF 3D Cartesian coordinate system with the one or more images as metadata information; and

post-processing of the at least one video sequence to extract 3D geo-augmented metadata from the one or more frames of the at least one video sequence based on the camera location or the camera pose information as represented according to the ECEF 3D Cartesian coordinate system and as associated in the metadata information,

wherein results of the post-processing of the at least one video sequence are displayed on a user display.

2. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:

a matching of the local three-dimensional point cloud to the one or more reference point clouds; and

a definition of the camera location information, the camera pose information, or a combination thereof according to the global coordinate system based, at least in part, on the matching.

3. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:

at least one determination of overlay information based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

a rendering of the overlay information in the one or more images.

4. A method of claim 1 , wherein the (1) data, (2) information, (3) at least one signal are further based, at least in part, on the following:

at least one determination of popularity information associated with the camera location information, the camera pose information, or a combination thereof; and

a presentation of one or more representations of the popularity information in at least one user interface of the at least one camera.

5. A method of claim 4 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:

at least one determination of one or more representative images associated with the camera location, the camera pose information, or a combination thereof; and

a presentation of (a) the one or more representative images; (b) guidance information for navigating to one or more locations, for pointing the at least one camera to one or more targets, or a combination thereof associated with the one or more representative images; (c) one or more indicators for confirming that the at least one camera is at or near the one or more locations associated with the one or more representative images, pointing to the one or more targets, or a combination thereof; (d) or a combination thereof.

6. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:

at least one determination of one or more panoramic images based, at least in part, on the camera location information, the camera pose information, or a combination thereof;

at least one determination of field of view information for the one or more images based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

7. A method of claim 1 , wherein the (1) data, (2) information, or (3) at least one signal are further based, at least in part, on the following:

at least one determination of the one or more other images associated with other camera location information, other camera pose information, or a combination thereof based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

8. A method of claim 1 , wherein one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein the one or more contextual criteria include, at least in part, one or more temporal criteria.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

cause, at least in part, a selection of one or more images from among one or more frames of the at least one video sequence, wherein the one or more images represent, at least in part, a subset of the one or more frames;

process and/or facilitate a processing of the one or more images to determine camera location information, camera pose information, or a combination thereof associated with at least one camera capturing the one or more images, wherein the camera location information, the camera pose information, or a combination thereof is represented according to a local three-dimensional (3D) Cartesian coordinate system and the one or more images are part of at least one video sequence;

process and/or facilitate a processing of the one or more images to determine a local point cloud of one or more objects depicted in the one or more images, wherein the local point cloud is represented according to the local 3D Cartesian coordinate system;

determine one or more reference point clouds represented according to an Earth Centered, Earth Fixed (ECEF) 3D Cartesian coordinate system based, at least in part, on the camera location information, location sensor data, or a combination thereof associated with the at least one camera;

process and/or facilitate a processing of at least one of the one or more reference point clouds represented according to the ECEF 3D Cartesian coordinate system to determine a correspondence of the local point cloud represented in the local 3D Cartesian coordinate system, wherein the correspondence maps the camera location information and the camera pose information from the local 3D Cartesian coordinate system to the ECEF 3D Cartesian coordinate;

cause, at least in part, an association of the camera location information, the camera pose information represented according to the ECEF Cartesian coordinate system, or a combination thereof with the one or more images as metadata information; and

cause, at least in part, a post-processing of the at least one video sequence to extract 3D geo-augmented metadata from the one or more frames of the at least one video sequence based on the camera location or the camera pose information as represented according to the ECEF 3D Cartesian coordinate system and as associated in the metadata information,

wherein results of the post-processing of the at least one video sequence are displayed on a user display.

10. An apparatus of claim 9 , wherein the apparatus is further caused to:

cause, at least in part, a matching of the local three-dimensional point cloud to one or more reference point clouds; and

cause, at least in part, a definition of the camera location information, the camera pose information, or a combination thereof according to the global coordinate system based, at least in part, on the matching.

11. An apparatus of claim 9 , wherein the apparatus is further caused to:

determine overlay information based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, a rendering of the overlay information in the one or more images.

12. An apparatus of claim 9 , wherein the one or more images represent one or more viewfinder images of the at least one camera, and wherein the apparatus is further caused to:

determine popularity information associated with the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, a presentation of one or more representations of the popularity information in at least one user interface of the at least one camera.

13. An apparatus of claim 12 , wherein the apparatus is further caused to:

determine one or more representative images associated with the camera location, the camera pose information, or a combination thereof; and

cause, at least in part, a presentation of (a) the one or more representative images; (b) guidance information for navigating to one or more locations, for pointing the at least one camera to one or more targets, or a combination thereof associated with the one or more representative images; (c) one or more indicators for confirming that the at least one camera is at or near the one or more locations associated with the one or more representative images, pointing to the one or more targets, or a combination thereof; (d) or a combination thereof.

14. An apparatus of claim 9 , wherein the apparatus is further caused to:

determine one or more panoramic images based, at least in part, on the camera location information, the camera pose information, or a combination thereof;

determine field of view information for the one or more images based, at least in part, on the camera location information, the camera pose information, or a combination thereof; and

cause, at least in part, an extension of the one or more images with at least a portion of the one or more panoramic images based, at least in part, on the field of view information.

15. An apparatus of claim 9 , wherein the apparatus is further caused to:

determine one or more other images associated with other camera location information, other camera pose information, or a combination thereof based, at least in part, on the camera location information, the camera pose information, or a combination; and

cause, at least in part, an association of the one or more other images with the one or more images for interaction by at least one user during playback of the one or more images.

16. An apparatus of claim 9 , wherein one or more other images are further determined based, at least in part, on one or more contextual criteria, and wherein the one or more contextual criteria include, at least in part, one or more temporal criteria.

Assignments (12)