System and method for sharing virtual and augmented reality scenes between users and viewers
View Patent ↗A preferred method for sharing user-generated virtual and augmented reality scenes can include receiving at a server a virtual and/or augmented reality (VAR) scene generated by a user mobile device. Preferably, the VAR scene includes visual data and orientation data, which includes a real orientation of the user mobile device relative to a projection matrix. The preferred method can also include compositing the visual data and the orientation data into a viewable VAR scene; locally storing the viewable VAR scene at the server; and in response to a request received at the server, distributing the processed VAR scene to a viewer mobile device.
1. A method comprising:
capturing a VAR scene with a mobile device at a location, wherein the VAR scene comprises a plurality of images captured by the mobile device, wherein each image of the plurality of images is associated with orientation data corresponding to a first orientation of the mobile device relative to a three-dimensional frame of reference;
correlating the plurality of images based on each image's orientation data at the mobile device;
compressing at least the correlated plurality of images at the mobile device to generate a processed VAR scene; and
transmitting the processed VAR scene to a server.
2. The method of claim 1 , further comprising creating a projection matrix representing an orientation of the device in a three-dimensional external frame of reference.
3. The method of claim 1 , wherein the orientation data further comprises a second orientation of the mobile device relative to a nodal point separate from the mobile device.
4. The method of claim 1 , wherein compressing at least the correlated plurality of images comprises down-sampling the one or more still images and fitting the down-sampled one or more still images onto a single texture.
5. The method of claim 1 , further comprising blending the correlated plurality of images.
6. The method of claim 5 , wherein blending the correlated plurality of images comprises premultiplying an image with a one of a Gaussian kernel or a smoothstep kernel.
7. The method of claim 1 , wherein the plurality of images comprises a video comprising a series of frames.
8. The method of claim 1 , wherein the VAR scene further comprises audio data.
9. The method of claim 1 , further comprising locally storing the processed VAR scene on the mobile device.
10. The method of claim 1 , wherein transmitting the processed VAR scene comprises transmitting metadata about the VAR scene to the server.
11. The method of claim 1 , wherein transmitting the processed VAR scene comprises asynchronously transmitting the correlated plurality of images and the orientation data to the server.
12. The method of claim 11 , further comprising associating a universally unique identifier with at least the correlated plurality of images, wherein the universally unique identifier for an image is based on a hash of a pathname associated with the image.
13. A method comprising:
receiving at a server a VAR scene generated by a mobile device, wherein the VAR scene comprises a plurality of images captured by the mobile device, wherein each image of the plurality of images is associated with orientation data corresponding to a real orientation of the mobile device in three-dimensional space relative to a projection matrix;
compositing the plurality of images based on each image's orientation data into a viewable VAR scene;
locally storing the viewable VAR scene at the server; and
in response to a request received at the server, distributing the processed VAR scene to a viewer mobile device.
14. The method of claim 13 , wherein compositing the plurality of images based on each image's orientation data comprises aligning one or more still images into a substantially uniform spherical image.
15. The method of claim 14 , wherein compositing the plurality of images based on each image's orientation data comprises associating one or more projection matrix parameters with the substantially uniform spherical image.
16. The method of claim 13 , further comprising associating a first viewer device with the viewer mobile device.
17. The method of claim 16 , wherein distributing the viewable VAR scene comprises pushing the VAR scene to the viewer mobile device in response to a request received from the first viewer device.
18. A method comprising:
from a viewer mobile device, requesting a VAR scene from a server, wherein the VAR scene comprises a plurality of images captured by a capture mobile device, wherein each image of the plurality of images is associated with orientation data corresponding to an orientation of the capture mobile device relative to a three-dimensional frame of reference;
receiving the VAR scene at the viewer mobile device;
determining a first orientation of the viewer mobile device relative to a three-dimensional frame of reference;
determining a second orientation of the viewer mobile device relative to a nodal point;
using the orientation data of the capture mobile device associated with each of the plurality of images in the VAR scene, orienting the VAR scene displayable on the viewer mobile device to the viewer based on the first orientation and the second orientation of the viewer mobile device; and
displaying the VAR scene on the viewer mobile device.
19. The method of claim 18 , wherein the VAR scene is processed by the server prior to receipt by the viewer mobile device.
20. The method of claim 18 , further comprising creating a projection matrix at the viewer mobile device representing an orientation of the viewer mobile device in a three-dimensional external frame of reference.
21. The method of claim 18 , further comprising adapting the VAR scene displayable on the viewer mobile device to the viewer in response to a change in one of the first orientation or the second orientation.
22. The method of claim 18 , wherein the capture mobile device and the viewer mobile device are the same device.