IP Library Granted Patent US 9,596,504
Granted Patent B2
US 9,596,504 · App. 13/143,628 · Granted Mar 14, 2017

Method for encoding of a video stream

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,596,504
App. No.
13/143,628
Granted
Mar 14, 2017
Kind
B2
Abstract

A temporal sequence of pictures is generated in a method for encoding of a first video stream. To do so, a synchronization signal is used which is derived from a second video stream independently of the first video stream, or the encoding of a second video stream independent of the first video stream is based on the same principle as for the encoding of the first video stream.

Claims (42)

1. A method for mixing at least two video streams comprising:

generating a first video stream by a first end point, the first video stream comprising a first temporal sequence of frames;

generating a second video stream by a second end point, the second video stream comprising a second temporal sequence of frames;

the first end point sending the first temporal sequence of frames to a mixing device;

the second end point sending the second temporal sequence of frames to the mixing device;

the mixing device deriving a synchronization signal from at least one of: the first temporal sequence of frames, the second temporal sequence of frames, and a timing signal;

the mixing device sending the synchronization signal to at least one of the first end point and the second end point so that subsequent frames of the first temporal sequence and subsequent frames of the second temporal sequence sent to the mixing device are synchronized with each other via synchronized encoding of the first and second subsequent frames by encoders of the first and second end points;

at least one of the first end point and second end point receiving the synchronization signal;

at least one of the first end point and the second end point, in response to the received synchronization signal, adjusting encoding of the frames of the first video stream and the frames of the second video stream such that subsequent frames of the first temporal sequence sent to the mixing device and subsequent frames of the second temporal sequence sent to the mixing device are synchronized with each other to have a same predictive structure based on the received synchronization signal;

the mixing device receiving the subsequent frames of the first temporal sequence and the subsequent frames of the second temporal sequence after having sent the synchronization signal and mixes the received subsequent frames of the first temporal sequence with the received subsequent frames of the second temporal sequence to generate a mixed video stream.

2. The method of claim 1 , wherein the synchronized predictive structure of the subsequent frames of the first temporal sequence sent to the mixing device and the subsequent frames of the second temporal sequence sent to the mixing device is configured so that each P-frame of a sequence of P-frames in the subsequent frames of the first temporal sequence has a position corresponding to a respective P-frame of a sequence of P-frames in the subsequent frames of the second temporal sequence.

3. The method of claim 1 , wherein the mixing device is a server of a video teleconference system, the first end point is a subscriber terminal and the second end point is a subscriber terminal.

4. The method of claim 1 , wherein the synchronization signal contains an information bit identifying a time offset between the positions of I-frames within the subsequent frames of the first temporal sequence sent to the mixing device or within the subsequent frames of the second temporal sequence sent to the mixing device.

5. The method of claim 1 , wherein the synchronization signal contains an information item identifying a number of P-frames or number of B-frames to follow an I-frame in at least one of the subsequent frames of the first temporal sequence sent to the mixing device and the subsequent frames of the second temporal sequence sent to the mixing device.

6. The method of claim 1 wherein the synchronization signal is only sent to the second end point and is derived from the first temporal sequence of frames.

7. The method of claim 1 wherein the synchronization signal is sent to both the first end point and the second end point.

8. The method of claim 1 , wherein the first end point, second end point, and mixing device are elements of a video conference system and wherein the first and second end points are within a video conference, the method comprising: upon arrival of an additional participant in the video conference, encoding a third video stream from a third end point of the additional participant as non-synchronized for a first time period and subsequently synchronizing said third video stream via communications with the mixing device after the mixing device generates a synchronization signal for the third end point for synchronizing encoding of the third video steam with the encoding of the first video stream and the encoding of the second video stream so that subsequent encoding of frames of the third video stream are encoded to have the same predictive structure as the encoding of the first and second video streams performed by the encoders of the first and second end points.

9. The method of claim 8 , comprising signaling before or during synchronization, a desired prediction structure to an encoder of the third end point.

10. The method of claim 1 wherein the synchronization signal is sent to both the first end point and the second end point and the synchronization signal is based on the timing signal.

11. The method of claim 1 wherein the predictive structure defines a group of pictures for encoding of at least one of the first video stream and the second video stream such that the subsequent frames of the first temporal sequence sent to the mixing device and the subsequent frames of the second temporal sequence sent to the mixing device have a same length for individual picture groups.

12. The method of claim 1 , wherein the mixing device is a central server and the first end point is a terminal and the second end point is a terminal and wherein the mixed video stream is a lone output of the central server for a teleconference involving the first and second end points.

13. The method of claim 1 , wherein the synchronization signal is configured so that the encoder of the first endpoint encodes the subsequent frames of the first temporal sequence and the encoder of the second end point encodes the subsequent frames of the second temporal sequence in a corresponding manner so that I-frames within the subsequent frames of the first temporal sequence correspond with I-frames within the subsequent frames of the second temporal sequence and P-frames within the subsequent frames of the first temporal sequence correspond with P-frames within the subsequent frames of the second temporal sequence.

14. The method of claim 1 , wherein the synchronization signal is configured so that each of the subsequent frames of the first temporal sequence is dependent on no more than one preceding frame within the first temporal sequence.

15. The method of claim 14 , wherein the synchronization signal is also configured so that each of the subsequent frames of the second temporal sequence is dependent on no more than one preceding frame within the first temporal sequence.

16. The method of claim 1 , wherein the synchronization signal is also configured to identify that each group of pictures within the subsequent frames of the first temporal sequence is to start with an I-frame.

17. A video conference system comprising:

a first end point;

a second end point; and

a mixing device;

the first end point generating a first video stream comprising a first temporal sequence of frames;

the second end point generating a second video stream comprising a second temporal sequence of frames;

the first end point sending the first temporal sequence of frames to the mixing device;

the second end point sending the second temporal sequence of frames to the mixing device;

the mixing device deriving a synchronization signal from at least one of: the first temporal sequence of frames, the second temporal sequence of frames, and a timing signal;

the mixing device sending the synchronization signal to at least one of the first end point and the second end point so that subsequent frames of the first temporal sequence and subsequent frames of the second temporal sequence sent to the mixing device are synchronized with each other via synchronized encoding of the first and second subsequent frames by encoders of the first and second end points;

at least one of the first end point and second end point receiving the synchronization signal;

at least one of the first end point and the second end point, in response to the received synchronization signal, adjusting encoding such that subsequent frames of the first temporal sequence sent to the mixing device and subsequent frames of the second temporal sequence sent to the mixing device are synchronized with each other to have a same predictive structure based on the received synchronization signal;

the mixing device receiving the subsequent frames of the first temporal sequence and the subsequent frames of the second temporal sequence after having sent the synchronization signal and the mixing device mixing the received subsequent frames of the first temporal sequence with the received subsequent frames of the second temporal sequence to generate a mixed video stream.

18. The system of claim 17 , wherein the mixing device is a server or a central server, the first end point is a terminal and the second end point is a terminal.

19. The system of claim 18 , wherein the predictive structure defines a group of pictures for encoding of at least one of the first video stream and the second video stream such that the subsequent frames of the first temporal sequence sent to the mixing device and the subsequent frames of the second temporal sequence sent to the mixing device have a same length for individual picture groups; and

wherein the synchronization signal is configured so that the encoder of the first endpoint encodes the subsequent frames of the first temporal sequence and the encoder of the second end point encodes the subsequent frames of the second temporal sequence in a corresponding manner so that I-frames within the subsequent frames of the first temporal sequence correspond with I-frames within the subsequent frames of the second temporal sequence and P-frames within the subsequent frames of the first temporal sequence correspond with P-frames within the subsequent frames of the second temporal sequence.

20. The system of claim 18 , wherein the synchronization signal contains an information bit identifying a time offset between the positions of I-frames within the subsequent frames of the first temporal sequence sent to the mixing device or within the subsequent frames of the second temporal sequence sent to the mixing device.

Assignments (7)
SECURITY INTEREST Recorded Feb 14, 2023
From: RINGCENTRAL, INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 062973/0194 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2022
From: RINGCENTRAL IP HOLDINGS, INC.
To: RINGCENTRAL, INC.
Reel/Frame 058856/0711 →
CONFIDENTIAL PATENT AGREEMENT Recorded Dec 21, 2020
From: UNIFY GMBH & CO. KG
To: UNIFY PATENTE GMBH & CO. KG
Reel/Frame 054822/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2020
From: RINGCENTRAL IP HOLDINGS, INC.
To: RINGCENTRAL, INC.
Reel/Frame 053353/0484 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2020
From: UNIFY SOFTWARE AND SOLUTIONS GMBH & CO. KG; UNIFY PATENTE GMBH & CO. KG
To: RINGCENTRAL IP HOLDINGS, INC.
Reel/Frame 053353/0111 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2017
From: AMON, PETER; OERTEL, NORBERT; AGTHE, BERNHARD
To: SIEMENS ENTERPRISE COMMUNICATIONS GMBH & CO. KG
Reel/Frame 041539/0312 →
CHANGE OF NAME Recorded Dec 8, 2014
From: SIEMENS ENTERPRISE COMMUNICATIONS GMBH & CO. KG
To: UNIFY GMBH & CO. KG
Reel/Frame 034537/0869 →