Alignment of 3D graphics extending beyond frame in augmented reality system with remote presentation

1 . A method, comprising:

receiving, by an augmented reality viewing device, frames of video of a live view of an event at an event venue;

receiving, by the augmented reality viewing device, augmented reality content for display over the view of the venue during the event within the frames of video as viewed through the augmented reality viewing device;

receiving, by the augmented reality viewing device, the augmented reality content for display over the view of the venue during the event extending beyond the frames of video as viewed through the augmented reality viewing device;

displaying by the augmented reality viewing device of the frames of video at a first location within a first field of view of the augmented reality viewing device, the first field of view including at least of portion of the frames of video and extending beyond the frames of video;

displaying by the augmented reality viewing device of the augmented reality content over the view of the venue during the event within the frames of video and extending beyond the frames of video, the augmented reality content within the frames of video aligning in a first alignment with the augmented reality content extending beyond the frames of video, including;

determining a placement of the augmented reality content within the frames of video;

determining a placement of the augmented reality content outside of the frames of video; and

aligning the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video; and

maintaining the first alignment of the augmented reality content within the frames and the augmented reality content extending beyond the frames of video as the first field of view of the augmented reality viewing device changes.

2 . The method of claim 1 , wherein determining the placement of the augmented reality content within the frames of video comprises determining a location of the augmented reality content within the frames of video; and

wherein aligning the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video comprises, subsequent to determining a location of the augmented reality content within the frames of video, aligning the augmented reality content extending beyond the frames of video with the augmented reality content within the frames of video to have the first alignment along a boundary of the frames of the video.

3 . The method of claim 2 , wherein maintaining the first alignment comprises:

generating for each of a plurality of field of views other than the first field of view a corresponding projection for display of the frames of video and the augmented reality content within the frames of video and extending beyond the frames of video;

determining a change in the field of view from the first field of view to a second field of view; and

displaying by the augmented reality viewing device of the augmented reality content within the frames of video and extending beyond the frames of video for the projection corresponding to the second field of view.

4 . The method of claim 3 , further comprising:

receiving from the augmented reality viewing device data on the augmented reality viewing device's field of view; and

determining from the data on the augmented reality viewing device's field of view the change in the field of view.

5 . The method of claim 3 , wherein the second field of view does not include the frames of video.

6 . The method of claim 1 , wherein determining the placement of the augmented reality content within the frames of video comprises determining a location of the augmented reality content within the frames of video; and

wherein aligning the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video comprises translating the content used to form the frames of video and the augmented reality content together to place the content used to form the frames of video within the frames of video and to place a portion of the augmented reality content within the frame and a portion of the augmented reality content extending beyond the frame.

7 . The method of claim 6 , wherein translating the content used to form the frames of video and the augmented reality content together includes a scaling operation.

8 . The method of claim 6 , wherein the content used to form the frames of video includes segmented video that fills only a portion of the frames of video.

9 . The method of claim 1 , wherein determining the placement of the augmented reality content within the frames of video comprises generating an image by the augmented reality viewing device of the first field of view; and

wherein aligning the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video comprises aligning the augmented reality content within the frames of video with the augmented reality content extending beyond the frames of video based on the image of the first field of view.

10 . The method of claim 9 , wherein maintaining the first alignment comprises:

determining whether a field of view for the augmented reality viewing device has changed from the first field of view;

in response to the field of view changing from the first field of view, generating an updated image by the augmented reality viewing device of the changed field of view; and

aligning the augmented reality content within the frames of video with the augmented reality content extending beyond the frames of video based on the updated image.

11 . A system, comprising:

one or more servers configured to receive data from and transmit data to an augmented reality viewing device and to:

receive frames of video of a live view of an event at an event venue;

receive information on a first field of view of the augmented reality viewing device;

provide to the augmented reality viewing device of the frames of video for display at a first location within the first field of view of the augmented reality viewing device, the first field of view including at least of portion of the frames of video and extending beyond the frames of video;

provide to the augmented reality viewing device augmented reality content for display over the view of the venue during the event within the frames of video;

provide to the augmented reality viewing device augmented reality content extending beyond the frames of video;

provide to the augmented reality viewing device information to align the augmented reality content within the frames of video in a first alignment with the augmented reality content extending beyond the frames of video, including information to:

determine a placement of the augmented reality content within the frames of video;

determine a placement of the augmented reality content outside of the frames of video; and

align the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video; and

provide to the augmented reality viewing device information to maintain the first alignment of the augmented reality content within the frames and the augmented reality content extending beyond the frames of video as the first field of view of the augmented reality viewing device changes.

12 . The system of claim 11 , wherein the augmented reality viewing device is at a location remote from the event venue.

13 . The system of claim 11 , wherein the information to determine the placement of the augmented reality content within the frames of video includes information to determine a location of the augmented reality content within the frames of video; and

wherein the information to align the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video includes information to, subsequent to determining a location of the augmented reality content within the frames of video, align the augmented reality content extending beyond the frames of video with the augmented reality content within the frames of video to have the first alignment along a boundary of the frames of the video.

14 . The system of claim 13 , wherein, to provide augmented reality content to the augmented reality viewing device, the one or more servers are further configured to:

generate for each of a plurality of field of views other than the first field of view a corresponding projection for display of the frames of video and the augmented reality content within the frames of video and extending beyond the frames of video;

determine a change in the field of view from the first field of view to a second field of view; and

provide to the augmented reality viewing device of the augmented reality content within the frames of video and extending beyond the frames of video for the projection corresponding to the second field of view.

15 . The system of claim 14 , wherein the one or more servers are further configured to:

receive from the augmented reality viewing device data on the augmented reality viewing device's field of view; and

determine from the data on the augmented reality viewing device's field of view the change in the field of view.

16 . The system of claim 11 , wherein the information to determine the placement of the augmented reality content within the frames of video includes information to determine a location of the augmented reality content within the frames of video; and

wherein the information to align the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video includes information to translate the content used to form the frames of video and the augmented reality content together to place the content used to form the frames of video within the frames of video and to place a portion of the augmented reality content within the frame and a portion of the augmented reality content extending beyond the frame.

17 . The system of claim 16 , wherein the content used to form the frames of video includes segmented video that fills only a portion of the frames of video.

18 . The system of claim 11 , wherein the information to determine the placement of the augmented reality content within the frames of video includes information to generate an image by the augmented reality viewing device of the first field of view; and

wherein the information to align the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video includes information to align the augmented reality content within the frames of video with the augmented reality content extending beyond the frames of video based on the image of the first field of view.

19 . The system of claim 18 , wherein, to provide augmented reality content to the augmented reality viewing device, the one or more servers are further configured to:

determine whether a field of view for the augmented reality viewing device has changed from the first field of view;

in response to the field of view changing from the first field of view, generate an updated image by the augmented reality viewing device of the changed field of view; and

align the augmented reality content within the frames of video with the augmented reality content extending beyond the frames of video based on the updated image.

20 . A method, comprising:

receiving, at a system of one or more servers, frames of video of a live view of an event at an event venue;

receiving, at the system of one or more servers, information on a first field of view of an augmented reality viewing device;

providing from the system of one or more servers to the augmented reality viewing device of the frames of video for display at a first location within the first field of view of the augmented reality viewing device, the first field of view including at least of portion of the frames of video and extending beyond the frames of video;

providing, from the system of one or more servers to the augmented reality viewing device, augmented reality content for display over the view of the venue during the event within the frames of video;

providing, from the system of one or more servers to the augmented reality viewing device, augmented reality content extending beyond the frames of video;

providing, from the system of one or more servers to the augmented reality viewing device, information to align the augmented reality content within the frames of video in a first alignment with the augmented reality content extending beyond the frames of video, including information for:

determining a placement of the augmented reality content within the frames of video;

determining a placement of the augmented reality content outside of the frames of video; and

aligning the augmented reality content within the frames of video with the augmented reality content outside of the frames of video along a boundary of the frames of video; and

providing, from the system of one or more servers to the augmented reality viewing device, information to maintain the first alignment of the augmented reality content within the frames and the augmented reality content extending beyond the frames of video as the first field of view of the augmented reality viewing device changes.