Interactive mixed reality platform utilizing geotagged social media

1. An end-user system comprising:

a communication device configured to communicate with a server;

a display screen;

one or more processors; and

at least one memory storing instructions which, when executed by the one or more processors, cause the end-user system to:

access a physical world geographical location from a user;

access at least two 360-degree street view images and depth maps for a region surrounding the physical world geographical location;

render, for display on the display screen, a three-dimensional mirrored world portion based on the at least two 360-degree street view images and the depths maps, the rendering comprising fusing the at least two 360-degree street view images by:

generating spherical geometries to represent the at least two 360-degree street view images,

correcting a depth value for each vertex of the spherical geometries by sampling the depth maps with spherical coordinates,

discarding pixels in intersections of the spherical geometries,

texturing each spherical geometry with the corresponding 360-degree street view image,

texturing seams where the spherical geometries meet by a weighted average of textures of the spherical geometries which meet at the seams,

texturing the ground plane, and

applying Gaussian filters in occluded regions which are not captured by any of the at least two 360-degree street view images;

render an avatar at a mirrored world location corresponding to the physical world geographical location;

access geotagged social media posts which have geotags in the region and which the user is permitted to view; and

render the geotagged social media posts as three-dimensional objects in the mirrored world portion.

2. The end-user system of claim 1 , wherein the rendered three-dimensional mirrored world portion corresponds to a 360-degree field of view about the mirrored world location.

3. The end-user system of claim 2 , wherein other mirrored world portions outside the 360-degree field of view are not rendered until the avatar is moved toward the other mirrored world portions.

4. The end-user system of claim 3 , wherein the instructions, when executed by the one or more processors, further cause the end-user system to:

communicate, via the communication device, movements of the avatar to the server;

receive, from the server, further 360-degree street view images corresponding to a new mirrored world location of the avatar; and

render, for display on the display screen, a second mirrored world portion based on the further 360-degree street view images.

5. The end-user system of claim 1 , wherein in rendering the three-dimensional mirrored world portion based on the at least two 360-degree street view images, the instructions, when executed by the one or more processors, cause the end-user system to:

determine whether the mirrored world location is outside or inside a boundary of a mesh corresponding to the at least two 360-degree street view images;

in case the mirrored world location is inside the boundary, render the three-dimensional mirrored world portion at a first level of detail; and

in case the mirrored world location is outside the boundary, render the three-dimensional mirrored world portion at a second level of detail,

wherein the first level of detail has greater detail than the second level of detail.

6. The end-user system of claim 5 , wherein the instructions, when executed by the one or more processors, further cause the end-user system to access two-dimensional physical world map data.

7. The end-user system of claim 6 , wherein in rendering the three-dimensional mirrored world portion at the second level of detail, the instructions, when executed by the one or more processors, cause the end-user system to:

access polygon data and height data in the two-dimensional physical world map data, the polygon data and the height data relating to buildings;

render three-dimensional polyhedrons corresponding to the buildings based on the polygon data and the height data; and

texture sides of the polyhedrons corresponding to the at least two 360-degree street view images based on the at least two 360-degree street view images.

8. The end-user system of claim 5 , wherein in rendering the three-dimensional mirrored world portion at the first level of detail, the instructions, when executed by the one or more processors, cause the end-user system to perform the fusing of the at least two 360-degree street view images.

9. The end-user system of claim 1 , wherein in rendering the geotagged social media posts as three-dimensional objects in the mirrored world portion, the instructions, when executed by the one or more processors, cause the end-user system to render at least one of:

a balloon object displaying a social media post, or

a frame displaying one or more selected from a social media image, text, videos, and a collaborative drawing board.

10. The end-user system of claim 1 , wherein in rendering the geotagged social media posts as three-dimensional objects in the mirrored world portion, the instructions, when executed by the one or more processors, cause the end-user system to render at least one of:

a stack of objects,

a poster grid of objects, or

a temporal transition of objects.

11. A method in an end-user system, the method comprising:

accessing a physical world geographical location from a user;

accessing at least two 360-degree street view images and depths maps for a region surrounding the physical world geographical location;

rendering, for display on a display screen, a three-dimensional mirrored world portion based on the at least two 360-degree street view images and the depths maps, the rendering comprising fusing the at least two 360-degree street view images by:

generating spherical geometries to represent the at least two 360-degree street view images,

correcting a depth value for each vertex of the spherical geometries by sampling the depth maps with spherical coordinates,

discarding pixels in intersections of the spherical geometries,

texturing each spherical geometry with the corresponding 360-degree street view image,

texturing seams where the spherical geometries meet by a weighted average of textures of the spherical geometries which meet at the seams,

texturing the ground plane, and

applying Gaussian filters in occluded regions which are not captured by any of the at least two 360-degree street view images;

rendering an avatar at a mirrored world location corresponding to the physical world geographical location;

accessing geotagged social media posts which have geotags in the region and which the user is permitted to view; and

rendering the geotagged social media posts as three-dimensional objects in the mirrored world portion.

12. The method of claim 11 , wherein the rendered three-dimensional mirrored world portion corresponds to a 360-degree field of view about the mirrored world location.

13. The method of claim 12 , wherein other mirrored world portions outside the 360-degree field of view are not rendered until the avatar is moved toward the other mirrored world portions.

14. The method of claim 13 , further comprising:

communicating, via a communication device, movements of the avatar to a server;

receiving, from the server, further 360-degree street view images corresponding to a new mirrored world location of the avatar; and

rendering, for display on the display screen, a second mirrored world portion based on the further 360-degree street view images.

15. The method of claim 11 , wherein rendering the three-dimensional mirrored world portion based on the at least two 360-degree street view images includes:

determining whether the mirrored world location is outside or inside a boundary of a mesh corresponding to the at least two 360-degree street view images;

in case the mirrored world location is inside the boundary, rendering the three-dimensional mirrored world portion at a first level of detail; and

in case the mirrored world location is outside the boundary, rendering the three-dimensional mirrored world portion at a second level of detail,

wherein the first level of detail has greater detail than the second level of detail.

16. The method of claim 15 , further comprising accessing two-dimensional physical world map data.

17. The method of claim 16 , wherein rendering the three-dimensional mirrored world portion at the second level of detail includes:

accessing polygon data and height data in the two-dimensional physical world map data, the polygon data and the height data relating to buildings;

rendering three-dimensional polyhedrons corresponding to the buildings based on the polygon data and the height data; and

texturing sides of the polyhedrons corresponding to the at least two 360-degree street view images based on the at least two 360-degree street view images.

18. The method of claim 15 , wherein rendering the three-dimensional mirrored world portion at the first level of detail includes performing the fusing of the at least two 360-degree street view images.

19. The method of claim 11 , wherein rendering the geotagged social media posts as three-dimensional objects in the mirrored world portion includes rendering at least one of:

a balloon object displaying a social media post, or

a frame displaying one or more selected from a social media image, text, videos, and a collaborative drawing board.

20. The method of claim 11 , wherein rendering the geotagged social media posts as three-dimensional objects in the mirrored world portion includes rendering at least one of:

a stack of objects,

a poster grid of objects, or

a temporal transition of objects.