Offloading slam processing to a remote device
A request to add an overlay at a pixel located at (x, y) coordinates in images is received. Waypoints in a three-dimensional space are identified with respective two-dimensional projections onto the images falling inside a circle with a given radius r and centered at (x, y). An overlay z-coordinate is determined for the overlay as a weighted sum of respective z coordinates of the waypoints. The overlay is assigned a location (x, y, overlay z-coordinate).
1 . A method, comprising:
receiving a request to add an overlay at a pixel located at (x, y) coordinates in images;
identifying waypoints in a three-dimensional space with respective two-dimensional projections onto the images falling inside a circle with a given radius r and centered at (x, y);
determining an overlay z-coordinate for the overlay as a weighted sum of respective z coordinates of the waypoints; and
assigning the overlay a location (x, y, overlay z-coordinate).
2 . The method of claim 1 , further comprising:
providing the overlay to a device so that that overlay is visible on the device.
3 . The method of claim 2 , further comprising:
requesting log in information for a remote device so that the remote device is able to display the overlay, wherein the remote device connects to a web browser so that the overlay is capable of being remotely shared with one or more remote device.
4 . The method of claim 2 , wherein the overlay is generated on a remote server from the image generated on the device.
5 . The method of claim 2 , wherein the overlay includes interactive elements that respond to a user input.
6 . The method of claim 1 , wherein identifying waypoints further comprises filtering waypoints based on their relevance to a predefined context associated with the overlay.
7 . The method of claim 1 , wherein the identifying of the waypoints include simultaneous localization and mapping (SLAM) of the images.
8 . The method of claim 7 , wherein the SLAM of the images are outputted to a web browser so that the SLAM of the images are accessible by one or more remote device.
9 . A method, comprising:
transmitting, from a user device and to a server, an encoded video frame of a video captured by a camera of the user device;
receiving, at the user device and from the server, data comprising a camera pose, waypoints, and data related to augmented reality overlays that are visible given the camera pose;
overlaying, at the user device, the augmented reality overlays over the video frame to obtain an augmented reality frame or receiving an augmented reality frame from the server; and
displaying or storing the augmented reality frame at the user device.
10 . The method of claim 9 , further comprising:
transmitting the augmented reality frame to a media server, a web browser, a client device or both.
11 . The method of claim 10 , further comprising:
requesting log in information so that one or more remote users are connectable to the web browser.
12 . The method of claim 10 , further comprising:
generating the augmented reality frame on the server with a regular mode.
13 . The method of claim 9 , wherein the augmented reality frame is formed as a tag or an annotation on the video frame.
14 . The method of claim 13 , wherein the tag or the annotations are pinned to a physical location located within the video frame.
15 . The method of claim 14 , further comprising:
automatically generating the tag or the annotation when the physical location is captured by an imaging device of the user device.
16 . A system comprising:
a device configured to be worn by a user, the device configured to:
capture frames or images and
transmit the frames or the images;
a server in communication with the device, the server comprising a processor configured to:
receive the frames or the images;
process the frames or the images to generate simultaneous location and mapping (SLAM) information related to the frames or the images;
overlay augmented reality overlays over the frames or the images; and
transmitting the overlays to the device to be displayed to the user,
wherein the server stores historical data and automatically displays the historical data upon a previously captured image being viewed by the device and the historical data is saved as a waypoint such that when the device arrives at the waypoint the historical data is automatically displayed.
17 . The system of claim 16 , further comprising:
a web browser in communication with the server so that the overlays may be viewed remotely by one or more remote devices.
18 . The system of claim 16 , wherein the device comprises: a camera, speakers, and a microphone so that audio and video can be shared between the device and the server.
19 . The system of claim 16 , wherein the overlay is at a pixel located at (x, y) coordinates in the frames or the images.
20 . The system of claim 16 , wherein the overlay comprises an overlay z-coordinate for the overlay as a weighted sum of respective z coordinates of the waypoints.