Augmented reality system for viewing an event with distributed computing

1. A method, comprising:

generating by a mobile device of image data of a venue and image metadata for the image in a coordinate system for the mobile device;

providing image data of the venue and image metadata from the mobile device to a server system;

generating by the server system of a transformation between the mobile device's coordinate system and the real world coordinate system from the image data and image metadata;

transmitting the transformation between the mobile device's coordinate system and the real world coordinate system from the server system to the mobile device;

receiving at the mobile device from the server system of graphics to be displayed by the mobile device over a view of the venue;

converting by the mobile device of the graphics into the mobile device's coordinate system based on the transformation;

displaying by the mobile device of the graphics over the view of the venue;

generating by the server system a set of reference images including one or more tracking points;

transmitting the set of reference images from the server system to the mobile device;

comparing by the mobile device of images generated by the mobile device with the reference images;

updating by the mobile device of the transformation between the mobile device's coordinate system and the real world coordinate system based on comparing the images generated by the mobile device with the reference images;

determining by the mobile device of an accuracy of the transformation between the mobile device's coordinate system and the real world coordinate system; and

based on the accuracy, requesting by the mobile device from the one or more servers an updated transformation between the mobile device's coordinate system and the real world coordinate system.

2. The method of claim 1 , wherein the graphics to be displayed by the mobile device over the view of the venue as received from the one or more servers are specified by location and orientation in the real world coordinate system.

3. The method of claim 1 , wherein displaying of the graphics includes displaying three dimensional contours of the venue over a two dimensional view of the venue.

4. The method of claim 1 , further comprising:

determining by the mobile device of an amount of latency in transmissions between the mobile device and the one or more servers,

wherein requesting by the mobile device from the one or more servers the updated transformation is further based on the amount of latency.

5. The method of claim 1 , wherein the mobile device includes a display having a screen and displaying the graphics includes displaying the view of the venue as image data generated by the camera.

6. The method of claim 1 , wherein the mobile device includes a head mounted display and displaying the graphics includes displaying the graphics over the view of venue as viewed by through the head mounted display.

7. A system, comprising:

a mobile device configured to:

generate image data of a venue and image metadata for the image data in a coordinate system for the mobile device;

receive a transformation between the mobile device's coordinate system and the real world coordinate system;

determine an accuracy of the transformation between the mobile device's coordinate system and the real world coordinate system;

based on the accuracy, request an updated transformation between the mobile device's coordinate system and the real world coordinate system;

receive graphics to be displayed by the mobile device over a view of the venue specified by location and orientation in the real world coordinate system;

convert the graphics into a coordinate system of the mobile device based on the transformation between the mobile device's coordinate system and the real world coordinate system; and

present the converted the graphics over the view of the venue; and

one or more servers configured to exchange data with the mobile device and to:

receive the image data of the venue and the image metadata;

generate the transformation between the mobile device's coordinate system and the real world coordinate system from the image data and image metadata;

provide the transformation and the updated transformation between the mobile device's coordinate system and the real world coordinate system; and

provide the graphics to be displayed over the view of the venue specified by location and orientation in the real world coordinate system.

8. The system of claim 7 , wherein the one or more servers are further configured to:

generate a set of reference images including one or more tracking points; and

transmit the set of reference images to the mobile device, and

wherein the mobile device is further configured to:

compare images generated by the mobile device with the reference images; and

update the transformation between the mobile device's coordinate system and the real world coordinate system based on comparing the images generated by the mobile device with the reference images.

9. The system of claim 8 , wherein the mobile device is further configured to:

determine an amount of latency in transmissions between the mobile device and the one or more servers, wherein the requesting for the updated transformation is further based on the amount of latency.

10. The system of claim 7 , wherein the graphics to be displayed by the mobile device over the view of the venue as received from the one or more servers are specified by location and orientation in the real world coordinate system.

11. The system of claim 7 , wherein displaying of the graphics includes displaying three dimensional contours of the venue over a two dimensional view of the venue.

12. The system of claim 7 , wherein the mobile device includes a display having a screen and displaying the graphics includes displaying the view of the venue as image data generated by the camera.

13. A method, comprising:

generating image data by a camera of a first mobile device;

providing image metadata in a coordinate system for first mobile device for the image data by one or more sensors of the first mobile;

transmitting the image data and the image metadata from the first mobile device to one or more servers;

transmitting a request for graphics to be displayed by the first mobile device over a view of a venue from the first mobile device to one or more servers;

receiving at the first mobile device from the one or more servers a transformation between a coordinate system the first mobile device and a real world coordinate system;

receiving at the first mobile device from the one or more servers the requested graphics;

converting by the first mobile device of the requested graphics into the first mobile device's coordinate system based on the transformation between the first mobile device's coordinate system and a real world coordinate system;

presenting the requested graphics over the view of the venue by the mobile device;

receiving a set of reference images including one or more tracking points from the one or more servers at the first mobile device, and

comparing by the first mobile device of image data from the camera to the reference images;

based on comparing the image data from the camera to the reference images, updating by the first mobile device of the transformation between the first mobile device's coordinate system and the real world coordinate system;

determining by the first mobile device of whether the update of the transformation between the first mobile device's coordinate system and the real world coordinate system has an accuracy within a threshold value; and

in response to determining that the accuracy of the update of the transformation is not within a threshold value, issuing by the first mobile device of a request to the one or more servers for an updated transformation from the first mobile device's coordinate system to the real world coordinate system.

14. The method of claim 13 , further comprising:

receiving by the one or more servers of the image data and the image metadata from the first mobile device;

calculating by the one or more servers of the transformation between the first mobile device's coordinate system and the real world coordinate system from the image data and image metadata from the first mobile device;

transmitting the transformation between the first mobile device's coordinate system and the real world coordinate system from the one or more servers to the first mobile device; and

transmitting the requested graphics from the one or more servers to the first mobile device.

15. The method of claim 14 , further comprising

retrieving point features of the venue for an event in a first coordinate system and locations of a set of fiducials features for the venue in the real world coordinate system by the one or more servers from one or more databases, wherein the transformation between the first mobile device's coordinate system and the real world coordinate system is further calculated from the point features of the venue and the set of fiducials features for the venue.

16. The method of claim 14 , further comprising:

receiving image data from a second mobile device at the one or more servers;

receiving image metadata from the second mobile device in a coordinate system of the second mobile device at the one or more servers;

calculating by the one or more servers of a transformation between the second mobile device's coordinate system and the real world coordinate system from the image data and image metadata from the second mobile device; and

transmitting the transformation between the second mobile device's coordinate system and the real world coordinate system to the second mobile device from the one or more servers to the second mobile device.

17. The method of claim 13 , wherein the graphics to be displayed by the first mobile device over the view of the venue as received from the one or more servers are specified by location and orientation in the real world coordinate system.

18. The method of claim 13 , wherein the graphics include three dimensional contours of the venue to be displayed over a two dimensional view of the venue.

19. The method of claim 13 , further comprising:

determining by the first mobile device of an amount of latency in transmissions between the first mobile device and the one or more servers, wherein requesting an updated transformation is further based on the amount of latency.

20. The method of claim 13 , providing image metadata in a coordinate system for the first mobile device for the image data includes receiving global positioning satellite (GPS) data.