INTRA-REFRESH FOR VIDEO STREAMING
Embodiments relate to encoding and decoding frames of a video stream. Video frames are encoded as intra-coded frames (Iframes) and predictive coded frames (P/Bframes) and transmitted. When a receiver of the encoded frames is unable to decode a frame, due to transmission problems or otherwise, the encoded video stream can be recovered without requiring a full Iframe to be generated at one time. Instead, intra-coded data is provided by the transmitter in slices. Specifically, frames with only portions of intra-coded data (Islices) are transmitted in sequence until enough intra-coded data is provided to the receiver to recover a frame and resume decoding. The intra-refresh frames may also contain slices predictively encoded (Pslices) based on restricted search spaces of preceding intra-refresh frames.
1 . A method encoding recovery performed by a first computing device that is transmitting a video stream to a second computing device, the method comprising:
intra-frame encoding a frame of the video stream to generate an Iframe and transmitting the Iframe to the second computing device;
inter-frame encoding a plurality of frames to generated Pframes, a first of the Pframes encoded from the Iframe, and a second of the Pframes encoded based on first Pframe;
receiving an indication from the second computing device that the second Pframe was not properly received or decodable by the second computing device; and
responsive to the indication, encoding and transmitting a sequence of intra-refresh frames, each intra-refresh frame comprising a single intra-refresh slice (Islice).
2 . A method according to claim 1 , wherein the intra-refresh frames are configured such that if the second computing device receives each of the intra-refresh frames a full frame is guaranteed to be recoverable from the intra-refresh frames.
3 . A method according to claim 1 , wherein a second intra-refresh frame is encoded immediately after a first intra-refresh frame, and wherein the method further comprises performing predictive encoding for one or more predictive slices (Pslices) of the second intra-refresh frame by restricting predictive scanning of an encoder to a validly encoded region of the first intra-refresh frame.
4 . A method according to claim 3 , wherein the validly encoded region comprises at least the Islice of the first intra-refresh frame.
5 . A method according to claim 4 , wherein the region consists of the Islice of the first intra-refresh frame and only any Pslices of the first intra-refresh frames that were encoded from a previous intra-refresh frame.
6 . A method according to claim 3 , wherein the predictive scanning comprises a motion search.
7 . A method according to claim 1 , wherein each intra-refresh frame includes one more Pslice than the immediately preceding intra-refresh frame.
8 . A method according to claim 1 , wherein the first computing device comprises an H.264 encoder and the second computing device comprises an H.264 decoder.
9 . A method according to claim 1 , further comprising smoothing the video quality of decoded intra-refresh frames by randomizing which blocks/portions of the intra-refresh frames are intra-encoded.
10 . A method according to claim, further comprising performing dithering on a slice of an intra-refresh frame.
11 . A computing device comprising:
processing hardware configured to generated video frames;
a framebuffer configured to store the generated video frames;
a multiplexer; and
a hardware encoder coupled with the processing hardware and configured to encode the video frames which are then formed into a video stream by the multiplexer for transmission, wherein the computing device is configured to respond to a request to refresh the video stream by encoding a sequence of intra-refresh frames, wherein the intra-refresh frames together are capable of being decoded to produce a complete frame of the video stream.
12 . A computing device according to claim 11 , wherein each intra-refresh frame comprises N slices, and wherein there are exactly N intra-refresh frames.
13 . A computing device according to claim 12 , wherein each intra-refresh frame comprises only one intra-encoded slice (Islice), and wherein each intra-refresh frame's respective Islice is at a different position than each of the other intra-refresh frames.
14 . A computing device according to claim 11 , wherein searching for motion for inter-frame encoding of slices is restricted to a region that increases from one intra-refresh frame to the next.
15 . A method performed by a computing device comprising:
encoding video frames as a sequence of at least an Iframe and a chain of Pframes;
transmitting the encoded video frames via a communication interface of the computing device;
determining that a transmitted frame was not able to be decoded, and in response encoding first, second, and third intra-refresh frames, in order respectively, wherein each intra-refresh frame comprises a first, second, and third slice,
wherein the first intra-refresh frame comprises a first Islice as its first slice,
wherein the second intra-refresh frame comprises a first Pslice as its first slice and a second Islice as its second slice, and wherein first Pslice was encoded based only on the first Islice, and
wherein the third intra-refresh frame comprises a second Pslice as its first slice, a third Pslice as its second slice, and a second Islice as its third slice, wherein the second and third Pslices are encoded based only on the second Islice and the first Pslice.
16 . A method according to claim 15 , further comprising decoding, by a device receiving the intra-refresh frames, a frame, wherein at least three slices of the frame are fully recovered by decoding based on the first and second Islices and based on the first, second, and third Pslices.
17 . A method according to claim 15 , wherein the intra-refresh frames are transmitted via wireless channel, and wherein the method further comprises dynamically adjusting sizes of the slices based on information about a condition of the channel.
18 . A method according to claim 15 , wherein which of the slices will receive Islices is randomized.
19 . A method according to claim 15 , wherein the first Pslice is encoded using a motion search on only the first Islice, and wherein the second and third Pslices are encoded using a motion search on only the second Islice and the first Pslice.
20 . A method according to claim 15 , wherein the first intra-refresh frame further comprises a Pslice of a prior frame.