Systems and methods for seeking within multimedia content during streaming playback
A receiver driven approach for playback of remote content is described. One embodiment includes obtaining information concerning the content of the media file from the remote server, identifying a starting location within the media sequence, identifying byte ranges of the media file corresponding to media required to play the media sequence from the starting location, requesting the byte ranges required to play the media sequence from the starting location, buffering received bytes of information pending commencement of playback, playing back the buffered bytes of information, receiving a user instruction, identifying byte ranges of the media file corresponding to media required to play the media sequence in accordance with the user instruction, flushing previous byte range requests, and requesting the byte ranges required to play the media in accordance with the user instruction.
1. A playback device, comprising:
a processor; and
a non-volatile storage containing an application for causing the processor to perform the steps of:
establishing at least one connection for communicating with a remote server system;
obtaining information from a remote server system describing at least one video track, multiple audio tracks, and multiple subtitle tracks;
selecting a video track from the at least one video track;
requesting a header describing the selected video track;
selecting an audio track from the multiple audio tracks;
obtaining index information indicating the locations of audio and video data within the selected audio and video tracks;
determining byte ranges to request from the selected audio and video tracks using the index information;
requesting byte ranges from the selected video track and the selected audio track from the remote server system;
buffering received bytes of information comprising audio and video data;
checking that sufficient data is buffered to commence playback and playing back the buffered audio and video data;
responding to a received seek instruction by:
pausing playback;
determining byte ranges to request from the selected audio and video tracks based upon a new playback location using the index information;
requesting byte ranges required to play the selected audio and video tracks from the new playback location from the remote server;
buffering received bytes of information comprising audio and video data pending resumption of playback; and
checking that sufficient data is buffered to commence playback and playing back the buffered audio and video data.
2. The playback device of claim 1 , wherein the application is further capable of causing the processor to asynchronously request byte ranges from the selected video track and the selected audio track from the remote server.
3. The playback device of claim 1 , wherein the step of requesting a header describing the selected video track comprises the step of requesting a DRM header.
4. The playback device of claim 3 , wherein the application is further capable of causing the processor to perform the step of decrypting the DRM header.
5. The playback device of claim 3 , wherein:
the DRM header identifies at least one key; and
the application is further capable of causing the processor to perform the steps of decrypting encrypted frames of video using the at least one key prior to decoding.
6. The playback device of claim 5 , wherein:
encrypted frames of video are only partially encrypted;
the received bytes of information contain references to portions of video frames that are encrypted; and
the step of decrypting encrypted frames of video using the at least one key prior to decoding further comprises decrypting encrypted portions of frames of video using the references to the portions of the video frames that are encrypted.
7. The playback device of claim 1 , wherein playing back the buffered video data from the requested byte ranges provides fast forward trick play.
8. The playback device of claim 1 , wherein playing back the buffered video data from the requested byte ranges provides fast rewind trick play.
9. The playback device of claim 1 , wherein the application is further capable of causing the processor to perform the step of discarding audio and video data contained within the buffer in response to a received seek instruction.
10. The playback device of claim 1 , wherein the application is further capable of causing the processor to perform the steps of:
selecting a subtitle track from the multiple subtitle tracks;
requesting data from the selected subtitle track from the remote server system, where the data contains subtitle text; and
superimposing the subtitle text over video during playback.
11. The playback device of claim 1 , wherein the application is further capable of causing the processor to perform the step of flushing previous byte range requests in response to the received seek instruction.
12. A playback device, comprising:
a processor; and
a non-volatile storage containing an application for causing the processor to perform the steps of:
establishing at least one connection for communicating with a remote server system;
obtaining information from a remote server system describing at least one video track, and at least one audio track;
selecting a video track from the at least one video track;
requesting a header describing the selected video track, where the requested header comprises a DRM header;
decrypting the DRM header;
selecting an audio track from the at least one audio track;
obtaining index information indicating the locations of audio and video data within the selected audio and video tracks;
determining byte ranges to request from the selected audio and video tracks using the index information;
creating a buffer;
requesting byte ranges from the video track and the audio track from the remote server system;
buffering received bytes of information comprising audio and video data;
checking that sufficient data is buffered to commence playback;
decrypting encrypted frames of video using information from the decrypted DRM header;
playing back the buffered audio and the decrypted video data;
responding to a received seek instruction by:
pausing playback;
discarding buffered audio and video data;
determining byte ranges to request from the selected audio and video tracks based upon a new playback location using the index information;
requesting byte ranges required to play the selected audio and video tracks from the new playback location from the remote server;
buffering received bytes of information comprising audio and video data pending resumption of playback;
checking that sufficient data is buffered to commence playback;
decrypting encrypted frames of video using information from the decrypted DRM header; and
playing back the buffered audio and decrypted video data.
13. The playback device of claim 12 , wherein:
the DRM header identifies at least one key; and
the step of decrypting encrypted frames of video using information from the decrypted DRM header comprises decrypting encrypted frames of video using the at least one key prior to decoding.
14. The playback device of claim 13 , wherein:
encrypted frames of video are only partially encrypted;
the received bytes of information contain references to portions of video frames that are encrypted; and
the step of decrypting encrypted frames of video using the at least one key prior to decoding further comprises decrypting encrypted portions of frames of video using the references to the portions of the video frames that are encrypted.
15. The playback device of claim 12 , wherein the application is further capable of causing the processor to asynchronously request byte ranges from the selected video track and the selected audio track from the remote server.
16. The playback device of claim 12 , wherein playing back the buffered video data from the requested byte ranges provides fast forward trick play.
17. The playback device of claim 12 , wherein playing back the buffered video data from the requested byte ranges provides fast rewind trick play.
18. The playback device of claim 12 , wherein the application is further capable of causing the processor to perform the steps of:
obtaining information from a remote server system describing at least one subtitle track;
selecting a subtitle track from the at least one subtitle track;
requesting data from the selected subtitle track from the remote server system, where the data contains subtitle text; and
superimposing the subtitle text over video during playback.
19. The playback device of claim 12 , wherein the application is further capable of causing the processor to perform the step of flushing previous byte range requests in response to the received seek instruction.
20. A method of playing back content on a playback device, comprising:
establishing at least one connection for communicating with a remote server system using a playback device;
obtaining information from a remote server system using the playback device, where the obtained information describes at least one video track, multiple audio tracks, and multiple subtitle tracks;
selecting a video track from the at least one video track;
requesting a header describing the at least one video track using the playback device;
selecting an audio track from the multiple audio tracks using the playback device;
obtaining index information indicating the locations of audio and video data within the selected audio and video tracks;
determining byte ranges to request from the selected audio and video tracks using the index information;
requesting byte ranges from the selected video track and the selected audio track from the remote server system using the playback device;
buffering received bytes of information comprising audio and video data on the playback device;
checking that sufficient data is buffered to commence playback and playing back the buffered audio and video data using the playback device;
responding to receipt of a seek instruction at the playback device by:
pausing playback on the playback device;
determining byte ranges to request from the selected audio and video tracks based upon a new playback location using the index information;
requesting byte ranges required to play the selected audio and video tracks from the new playback location from the remote server using the playback device;
buffering received bytes of information comprising audio and video data pending resumption of playback using the playback device; and
checking that sufficient data is buffered to commence playback and playing back the buffered audio and video data using the playback device.
21. The method of claim 20 , further comprising asynchronously requesting byte ranges from the selected video track and the selected audio track from the remote server using the playback device.
22. The method of claim 20 , wherein requesting a header describing the selected video track further comprises requesting a DRM header using the playback device.
23. The method of claim 22 , further comprising decrypting the DRM header.
24. The method of claim 23 , wherein:
the DRM header identifies at least one key; and
the method further comprises decrypting encrypted frames of video using the at least one key prior to decoding using the playback device.
25. The method of claim 24 , wherein:
encrypted frames of video are only partially encrypted;
the received bytes of information contain references to portions of video frames that are encrypted; and
decrypting encrypted frames of video using the at least one key prior to decoding further comprises decrypting encrypted portions of frames of video using the references to the portions of the video frames that are encrypted using the playback device.
26. The method of claim 20 , wherein playing back the buffered video data from the requested byte ranges provides fast forward trick play.
27. The method of claim 20 , wherein playing back the buffered video data from the requested byte ranges provides fast rewind trick play.
28. The method of claim 20 , further comprising discarding audio and video data contained within the buffer in response to a received seek instruction.
29. The method of claim 20 , further comprising:
selecting a subtitle track from the multiple subtitle tracks using the playback device;
requesting data from the selected subtitle track from the remote server system using the playback device, where the data contains subtitle text; and
superimposing the subtitle text over video during playback using the playback device.
30. The method of claim 20 , further comprising flushing previous byte range requests in response to the received seek instruction.