Method and system for switching between video streams in a continuous presence conference
View Patent ↗An MRC system includes an MRM and an MRE that handle needs for Intra frames in an efficient way, improves the conferees experience, and reduces the load on the resources associated with that MRC videoconference by reducing the number of Intra frames and lowering the impact of Intra frames when they are needed. In some embodiments, when a requiring MRE requests an Intra frame for a video stream received from a presenting MRE, an MRM may respond by requesting the presenting MRE to send a temporary video stream toward the requiring MRE while sending in parallel a normal stream toward the rest of the MREs.
1. A method comprising:
sending a first compressed video stream from a first media relay endpoint to be transferred toward a second media relay endpoint;
creating a second compressed video stream by the first media relay endpoint to be transferred toward a third media relay endpoint, comprising:
synchronizing a first reference frame in a first encoder of the first media relay endpoint with a second reference frame in a second encoder of the first media relay endpoint;
sending a first compressed video frame in the first compressed video stream, encoded by the first encoder from the first reference frame; and
sending a second compressed video frame in the second compressed video stream, encoded by the second encoder from the second reference frame.
2. The method of claim 1 , further comprising:
receiving a request for an intra frame to be sent toward the third media relay endpoint,
wherein the act of creating the second compressed video stream by the first media relay endpoint to be transferred toward the third media relay endpoint is performed responsive to the request for an intra frame.
3. The method of claim 1 , wherein the act of synchronizing a first reference frame in a first encoder of the first media relay endpoint with a second reference frame in a second encoder of the first media relay endpoint comprises:
creating a first sequence of reference frames in the first encoder by encoding a first sequence of video frames in the first video stream by the first encoder; and
creating a second sequence of reference frames in the second encoder by encoding a second sequence of video frames in the second video stream by the second encoder, the second sequence of video frames encoded based on the first sequence of reference frames,
wherein the second sequence of reference frames is successively more synchronized with the first sequence of reference frames, and
wherein a last reference frame of the second sequence of reference frames is synchronized with a last reference frame of the first sequence of reference frames.
4. The method of claim 3 , wherein the second sequence of video frames comprises:
an intra frame of lower quality than a video frame of the first video stream; and
a sequence of inter frames of increasing quality.
5. The method of claim 3 , wherein a last video frame of the second sequence of video frames is a unique compressed frame.
6. The method of claim 3 , wherein a difference between a current frame of the first sequence of reference frames and an immediately preceding reference frame of the second sequence of reference frames is losslessly compressed to create the last frame of the second sequence of video frames.
7. The method of claim 6 , wherein the difference is losslessly compressed using ZIP.
8. The method of claim 3 , wherein the first sequence of video frames is a sequence of inter frames.
9. The method of claim 1 , wherein the first compressed video stream and the second compressed video stream comply with H.264.
10. The method of claim 1 , further comprising:
terminating the second video stream; and
sending the first video stream to be transferred toward the second media relay endpoint and the third media relay endpoint.
11. The method of claim 1 , wherein the first video stream is sent for relaying toward a plurality of media relay endpoints.
12. The method of claim 1 , wherein the act of synchronizing a first reference frame in a first encoder of the first media relay endpoint with a second reference frame in a second encoder of the first media relay endpoint comprises:
creating a first sequence of reference frames in the first encoder while encoding the first compressed video stream; and
creating a second sequence of reference frames in the second encoder while encoding the second compressed video stream,
wherein the first compressed video stream and the second compressed video stream are encoded from a same sequence of scaled video images.
13. The method of claim 12 , wherein the second compressed video stream comprises:
an intra frame of lower quality than a video frame of the first compressed video stream; and
a sequence of inter frames of increasing quality.
14. The method of claim 13 , wherein the first compressed video stream comprises a stream of inter frames.
15. The method of claim 12 , wherein a current video frame of the same sequence of scaled video images is encoded by the first encoder as a switching predictive (SP) frame and by the second encoder as a secondary switching predictive (SSP) frame.
16. The method of claim 15 , further comprising terminating the second compressed video stream after sending the SSP frame, wherein a video frame of the second compressed video stream is an SSP frame.
17. A media relay multipoint control unit for providing a continuous presence layout at a first media relay endpoint during a multipoint video conference between the first media relay endpoints and a plurality of media relay endpoints, the continuous presence layout comprising a plurality of segments, each segment displaying video from a selected media relay endpoint of the plurality of media relay endpoints, the media relay multipoint control unit comprising:
a network interface that receives relay compressed video data chunks from the plurality of media relay endpoints and sends relayed compressed video data chunks to the first media relay endpoint;
a signaling and control module that selects two or more media relay endpoints from the plurality of media relay endpoints; and
a compressed video processor that
parses the received relay compressed video data chunks;
organizes the parsed relay compressed video data chunks received from the selected two or more media relay endpoints into a group of two or more streams of relayed compressed video data chunks; and
transfers the group of two or more streams of relayed compressed video data chunks toward the first media relay endpoint via the network interface,
wherein the compressed video processor, in response to receiving instructions from the signaling and control module that an intra frame is requested by a requiring media endpoint from a presenting media relay endpoint of the selected two or more media relay endpoints:
obtains a temporary stream of relay compressed video data chunks in parallel to obtaining a normal stream of relay compressed video data chunks, both from the presenting media relay endpoint;
parses the obtained relay compressed video data chunks over the temporary stream;
organizes the parsed relay compressed video data chunks into a temporary stream of relayed compressed video data chunks; and
transfers the temporary stream of relayed compressed video data chunks toward the requiring media relay endpoint via the network interface.
18. The media relay multipoint control unit of claim 17 , wherein the compressed video processor is further configured to respond to instructions received from the signaling and control module that the temporary stream is terminated by relaying normal relayed compressed video data chunks obtained from the presenting media relay endpoint toward the requiring media relay endpoint.
19. A media relay endpoint, comprising:
a video processor comprising:
a first encoder; and
a temporary encoder,
wherein the video processor is configured to:
send a first video stream compressed by the first encoder for relaying toward a second media relay endpoint;
send a temporary video stream compressed by the temporary encoder for relaying toward a third media relay endpoint, when receiving an indication that an intra frame is needed by the third media relay endpoint;
synchronize a first reference frame in the first encoder with a second reference frame in the temporary encoder; and
terminate the temporary video stream.
20. The media relay endpoint of claim 19 , wherein the indication is an intra request received from the third media relay endpoint.
21. The media relay endpoint of claim 19 , wherein the video processor synchronizes by creating a switching predictive (SP) frame by the first encoder and a secondary switching predictive (SSP) frame by the temporary encoder.
22. A media relayed endpoint (MRE), comprising:
an MRE video processor that:
receives a group of streams of relayed compressed video data chunks originating from a group of selected MREs;
organizes the received relayed compressed video data chunks into a plurality of groups, each group corresponding to an MRE of the group of selected MREs;
decodes each group of organized received relayed compressed video data chunks; and
assembles a continuous presence layout from the decoded organized received relayed compressed video data chunks,
wherein the MRE video processor is configured to:
obtain a temporary stream of relayed compressed video data chunks sent from a first MRE;
organize, decode, and assemble video from the temporary stream in the layout;
synchronize a reference frame in a decoder allocated to decode a common video stream received from the first MRE; and
organize, decode, and assemble video from the common stream in the layout.