Adding augmented reality to a sub-view of a high resolution central video feed
Techniques are disclosed to add augmented reality to a sub-view of a high resolution central video feed. In various embodiments, a central video feed is received from a first camera on a first recurring basis and time-stamped position information is received from a tracking system on a second recurring basis. The central video feed is calibrated against a spatial region encompassed by the central video feed. The received time-stamped position information and a determined plurality of tiles associated with at least one frame of the central video feed are used to define a first sub-view of the central video feed. The first sub-view and a homography defining placement of augmented reality elements on the at least one frame of the central video feed are provided as output to a device configured to use the first sub-view and the homography display the first sub-view.
1. A system, comprising:
a communication interface configured to:
receive, on a first recurring basis, a transmission of a central video feed from a first camera; and
receive, on a second recurring basis, a respective time-stamped position information from a tracking system; and
a processor coupled to the communication interface and configured to:
calibrate the central video feed against a spatial region encompassed by the central video feed including by:
partitioning at least one frame of the central video feed into a plurality of tiles at a first resolution; and
determining a homography defining placement of augmented reality elements on the at least one frame of the central video feed;
use the received time-stamped position information and the plurality of tiles associated with at least one frame of the central video feed to define a first sub-view of the central video feed, wherein the first sub-view:
includes a subset of the plurality of tiles associated with at least one frame of the central video feed; and
is associated with a first subject of a plurality of subjects in the spatial region; and
output the first sub-view and the determined homography to a device configured to use the first sub-view and the homography to display the first sub-view.
2. The system of claim 1 , wherein the first subject of the plurality of subjects in the spatial region includes at least one player or object of interest.
3. The system of claim 1 , wherein the first subject of the plurality of subjects in the spatial region includes a plurality of subjects.
4. The system of claim 1 , wherein the subset of the plurality of tiles is selected based at least in part on the first subject.
5. The system of claim 1 , wherein the central video feed includes a full view of the spatial region and the spatial region includes a competition space.
6. The system of claim 1 , wherein the first sub-view of the central video feed includes an isolation shot of a portion of the spatial region.
7. The system of claim 1 , wherein the first sub-view includes a plurality of video frames depicting the first subject.
8. The system of claim 1 , wherein the first sub-view includes a plurality of video frames centered on the first subject.
9. The system of claim 1 , wherein the first sub-view is zoomable and a level of zoom is based at least in part on padding surrounding the first subject.
10. The system of claim 1 , wherein partitioning at least one frame of the central video feed into a plurality of tiles at a first resolution includes dividing the at least one frame into a predetermined number of tiles based at least in part on a desired resolution.
11. The system of claim 1 , wherein the augmented reality elements include a scrimmage line.
12. The system of claim 1 , wherein the augmented reality elements include statistics associated with a competition corresponding to the central video feed.
13. The system of claim 1 , wherein the augmented reality elements includes advertising content customized to a user.
14. The system of claim 1 , wherein the processor is configured to:
capture a blank scene; and
determine augmented reality elements to add to the scene.
15. The system of claim 1 , wherein the device is configured to use the first sub-view and the homography to display the first sub-view including by reconstructing an image by compositing a blank scene, the augmented reality elements, and the first subject.
16. The system of claim 1 , wherein the first camera is fixed.
17. The system of claim 1 , wherein the first camera captures the central video feed at a resolution of at least 12K.
18. The system of claim 1 , wherein the device is configured to use the first sub-view and the homography to display the first sub-view including by displaying a graphical user interface including a plays panel, a video panel including the first sub-view, and a player panel.
19. A method, comprising:
receiving, on a first recurring basis, a transmission of a central video feed from a first camera,
receiving, on a second recurring basis, a respective time-stamped position information from a tracking system;
calibrating the central video feed against a spatial region encompassed by the central video feed including by:
partitioning at least one frame of the central video feed into a plurality of tiles at a first resolution; and
determining a homography defining placement of augmented reality elements on the at least one frame of the central video feed;
using the received time-stamped position information and the plurality of tiles associated with at least one frame of the central video feed to define a first sub-view of the central video feed, wherein the first sub-view:
includes a subset of the plurality of tiles associated with at least one frame of the central video feed; and
is associated with a first subject of a plurality of subjects in the spatial region; and
providing the first sub-view and the determined homography as output to a device configured to use the first sub-view and the homography to display the first sub-view.
20. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving, on a first recurring basis, a transmission of a central video feed from a first camera;
receiving, on a second recurring basis, a respective time-stamped position information from a tracking system;
calibrating the central video feed against a spatial region encompassed by the central video feed including by:
partitioning at least one frame of the central video feed into a plurality of tiles at a first resolution; and
determining a homography defining placement of augmented reality elements on the at least one frame of the central video feed;
using the received time-stamped position information and the plurality of tiles associated with at least one frame of the central video feed to define a first sub-view of the central video feed, wherein the first sub-view:
includes a subset of the plurality of tiles associated with at least one frame of the central video feed; and
is associated with a first subject of a plurality of subjects in the spatial region; and
providing the first sub-view and the determined homography as output to a device configured to use the first sub-view and the homography to display the first sub-view.