Dynamic client-server video tiling streaming
A system and a method for communicating video data associated with content across a network are disclosed. A server partitions the content into a plurality of tiles where each tile includes a subset of the content and index data describing a relationship between a tile and one or more adjacent tiles. Each tile is then encoded as a video frame, for example each tile is encoded according to the H.264 standard. The plurality of tiles are then transmitted to a client which combines the plurality of tiles using the index data to locally recreate the content. The client associates a predetermined viewport tile with a client viewport representing the area of the content that is visible in a client display device to allow viewing of the content while tiles not associated with the client viewport are cached to expedite subsequent retrieval.
1. A method for communicating content across a network using video frames, the method comprising:
partitioning the content into a plurality of tiles, each tile including a subset of the content and index data describing a relationship between a tile and one or more adjacent tiles;
encoding the content associated with each tile in the plurality of tiles as a reference frame;
transmitting the plurality of encoded tiles to a client to allow the client to combine the plurality of encoded tiles using the index data;
responsive to a modification to the content associated with a tile, generating a difference frame, the difference frame describing the modification to the content associated with the tile in relation to the reference frame;
comparing a storage size of the difference frame to a threshold specifying a maximum storage size of the difference frame;
responsive to the storage size of the difference frame being less than the threshold, transmitting the index data of the tile whose content has been modified and the difference frame to the client, the client configured to combine the difference frame with the reference frame associated with the tile identified by the index data; and
responsive to the storage size of the difference frame being greater than the threshold,
generating a replacement reference frame, the replacement reference frame including the modified content associated with the tile, and
transmitting the index data of the tile whose content has been modified and the replacement reference frame to the client to allow the client to combine the replacement reference frame with one or more of the plurality of encoded tiles using the index data.
2. The method of claim 1 , wherein the index data further comprises a tile identifier, a tile height and width, or clipping data, the clipping data describing editing of the content to not exceed the tile height and width.
3. The method of claim 1 , wherein partitioning the content into a plurality of tiles comprises:
segmenting the content into a grid, the grid comprising a plurality of tiles having a fixed height and width and each tile adjacent to one or more surrounding tiles.
4. The method of claim 1 , further comprising:
caching the reference frame associated with each tile in the plurality of tiles.
5. The method of claim 1 , further comprising:
caching the difference frame and the index data associating the difference frame with the reference frame including the modification to content.
6. A non-transitory computer-readable storage medium configured to store instructions for communicating content across a network using video frames, the instructions when executed by a processor causing the processor to:
partition the content into a plurality of tiles, each tile including a subset of the content and index data describing a relationship between a tile and one or more adjacent tiles;
encode the content associated with each tile in the plurality of tiles as a reference frame;
transmit the plurality of encoded tiles to a client to allow the client to combine the plurality of encoded tiles using the index data;
responsive to a modification to the content associated with a tile, generate a difference frame, the difference frame describing the modification to the content associated with the tile in relation to the reference frame;
compare a storage size of the difference frame to a threshold specifying a maximum storage size of the difference frame;
responsive to the storage size of the difference frame being less than the threshold, transmit the index data of the tile whose content has been modified and the difference frame to the client, the client configured to combine the difference frame with the reference frame associated with the tile identified by the index data; and
responsive to the storage size of the difference frame being greater than the threshold,
generate a replacement reference frame, the replacement reference frame including the modified content associated with the tile, and
transmit the index data of the tile whose content has been modified and the replacement reference frame to the client to allow the client to combine the replacement reference frame with one or more of the plurality of encoded tiles using the index data.
7. The non-transitory computer-readable storage medium of claim 6 , wherein the index data further comprises a tile identifier, a tile height and width, or clipping data, the clipping data describing editing of the content to not exceed the tile height and width.
8. The non-transitory computer-readable storage medium of claim 6 , wherein the instructions that when executed cause the processor to partition the content into a plurality of tiles further comprises instructions that when executed cause the processor to segment the content into a grid, the grid comprising a plurality of tiles having a fixed height and width and each tile adjacent to one or more surrounding tiles.
9. The non-transitory computer-readable storage medium of claim 6 , further comprising instructions that when executed cause the processor to cache the reference frame associated with each tile in the plurality of tiles.
10. The non-transitory computer-readable storage medium of claim 6 , further comprising instructions that when executed cause the processor to cache the difference frame and the index data associating the difference frame with the reference frame including the modification to content.
11. A system for communicating content across a network using video frames, the system comprising:
a processor configured to execute computer program instructions; and
a non-transitory computer readable storage medium storing the computer program instructions executable by the processor, the instructions for:
partitioning the content into a plurality of tiles, each tile including a subset of the content and index data describing a relationship between a tile and one or more adjacent tiles;
encoding the content associated with each tile in the plurality of tiles as a reference frame;
transmitting the plurality of encoded tiles to a client to allow the client to combine the plurality of encoded tiles using the index data;
responsive to a modification to the content associated with a tile, generating a difference frame, the difference frame describing the modification to the content associated with the tile in relation to the reference frame;
comparing a storage size of the difference frame to a threshold specifying a maximum storage size of the difference frame;
responsive to the storage size of the difference frame being less than the threshold, transmitting the index data of the tile whose content has been modified and the difference frame to the client, the client configured to combine the difference frame with the reference frame associated with the tile identified by the index data; and
responsive to the storage size of the difference frame being greater than the threshold:
generating a replacement reference frame, the replacement reference frame including the modified content associated with the tile, and
transmitting the index data of the tile whose content has been modified and the replacement reference frame to the client to allow the client to combine the replacement reference frame with one or more of the plurality of encoded tiles using the index data.
12. The system of claim 11 , wherein the index data further comprises a tile identifier, a tile height and width, or clipping data, the clipping data describing editing of the content to not exceed the tile height and width.
13. The system of claim 11 , wherein the instructions for partitioning the content into a plurality of tiles further comprises instructions for segmenting the content into a grid, the grid comprising a plurality of tiles having a fixed height and width and each tile adjacent to one or more surrounding tiles.
14. The system of claim 11 , further comprising instructions for caching the reference frame associated with each tile in the plurality of tiles.
15. The system of claim 11 , further comprising instructions for caching the difference frame and the index data associating the difference frame with the reference frame including the modification to content.