Augmented reality video stream synchronization
A platform for synchronizing augmented reality (AR) views and information between two or more network connected devices is disclosed. A first device captures a video stream and associated essential meta-data, embeds the essential meta-data into the video stream and transmits it to a second device. The second device receives the video stream, extracts the essential meta-data, inserts one or more AR objects into the video stream with reference to the enhanced meta-data, and transmits to the first device the AR objects and reference to the essential meta-data. The first device renders the one or more AR objects into the video stream, using the essential meta-data references to locate the AR objects in each video stream frame. The second device may also determine and transmit a modified video stream to the first device.
1. A method for a service provider to synchronize augmented reality (AR) objects across a network with a device, in which essential meta-data and nonessential data facilitate first and second rendering of an AR presentation, comprising:
receiving, over the network from the device, a video stream associated with the device, the video stream including embedded in-band therein the essential meta-data corresponding to at least motion of the device while capturing the video stream to facilitate the first rendering, and an associated separate non-embedded out-of-band nonessential data to facilitate the second rendering, wherein the device determines selection of information that is embedded in-band and information that is non-embedded out-of-band nonessential data dynamically based at least in part on analysis of communication session conditions, the essential meta-data embedded in the video stream such that the essential meta-data is non-visible;
extracting the essential meta-data information from the video stream;
determining, based at least in part on the essential meta-data, initial AR information including placement of one or more AR objects in the video stream;
determining the first rendering based at least in part on the initial AR information;
determining an updated AR information based at least in part on updating the initial AR information based at least in part on the separate nonessential data; and
determining the second rendering based at least in part on refining the initial rendering based at least in part on the updated AR information.
2. The method of claim 1 , further comprising:
receiving, over the network, the separate nonessential data corresponding to the video stream over the network contemporaneously with the video stream;
determining an updated placement of the one or more AR objects in the video stream based at least in part on the updated AR information.
3. The method of claim 2 , further comprising:
providing the updated AR information over the network.
4. The method of claim 2 , further comprising:
receiving the video stream with a transport having at least a first and a second synchronized data stream, wherein the first data stream includes the video stream and the second data stream includes the nonessential data.
5. The method of claim 1 , wherein extracting the essential meta-data comprises:
receiving essential meta-data represented as non-visible data within the video stream; and
deobfuscating the essential meta-data within the video stream to facilitate the extracting.
6. The method of claim 1 , further comprising:
determining, for a selected one or more frame of the video stream, a corresponding essential meta-data for the selected one or more frame; and
calculating at least the initial AR information for the one or more frame based at least in part on the corresponding essential meta-data.
7. A non-transitory computer readable medium (CRM) comprising instructions for a service provider to synchronize augmented reality (AR) objects across a network with a device, in which essential meta-data and nonessential data facilitate first and second rendering of an AR presentation, the instructions executable by an apparatus, that when executed cause the apparatus to:
receive a video stream from the device, the video stream including embedded in-band therein the essential meta-data corresponding to at least motion of the device while capturing the video stream to facilitate the first rendering, and an associated separate non-embedded out-of-band nonessential data to facilitate the second rendering, wherein the device determines selection of information that is embedded in-band and information that is non-embedded out-of-band nonessential data is determined dynamically based at least in part on analysis of communication session conditions, the essential meta-data embedded in the video stream such that the essential meta-data is non-visible;
extract the essential meta-data information from the video stream;
determine, based at least in part on the essential meta-data, initial AR information including placement of one or more AR objects in the video stream;
provide the initial AR information to the device;
determining the first rendering based at least in part on the initial AR information;
determine an updated AR information based at least in part on updating the initial AR information based at least in part on the separate nonessential data; and
determining the second rendering based at least in part on refining the initial rendering based at least in part on the updated AR information.
8. The CRM of claim 7 , wherein the instructions are to further cause the apparatus to:
receive the separate nonessential data corresponding to the video stream over the network contemporaneously with the video stream; and
determine an updated placement of the one or more AR objects in the video stream based at least in part on the updated AR information.