System and method for automatic encoder adjustment based on transport data
View Patent ↗A system and method for transmission of a video stream are provided. The system may include: an encoder adapted to generate a video stream comprising a plurality of encoded frames, encoded according to at least one encoding parameter; a comparator in communication with the encoder, the comparator adapted to compare encoded frames of the plurality of encoded frames with input frames to determine a fitness metric reflective of visual quality of the encoded frames; and a controller in communication with the comparator, the controller adapted to adjust the at least one encoding parameter based on the fitness metric.
1. A system for transmission of a video stream across a plurality of connected networks, each network having a corresponding available bandwidth, the system comprising:
a connection controller circuit controlling a plurality of interfaces for transmitting the video stream over the plurality of connected networks, each interface corresponding to a specific network of the plurality of connected networks;
an encoder circuit adapted to generate individual video streams comprising a plurality of encoded frames, encoded according to at least one encoding parameter, each individual video stream transmitted to a corresponding interface of the plurality of interfaces for transmission across a corresponding network at an estimated maximum bit rate derived based on a predicted available bandwidth available across all of the plurality of connected networks;
a comparator circuit in communication with the encoder circuit, the comparator adapted to compare encoded frames of the plurality of encoded frames with input frames to determine a fitness metric reflective of visual quality of the encoded frames upon detecting a change in the predicted available bandwidth determined from feedback data obtained by the connection controller circuit corresponding to each network; and
an encoding controller circuit in cooperation with the comparator, the encoding controller circuit configured to:
receive data corresponding to the available bandwidth for each network of the plurality of connected networks, and
control the encoder circuit to adjust the at least one encoding parameter for each of the individual video streams based on the available bandwidth of the corresponding network and the fitness metric, the encoder circuit and the comparator cooperating to iteratively adjust the at least one encoding parameter until the fitness metric meets or exceeds a pre-defined minimum value to optimize visual quality of the video stream for a bit rate determined based at least on the available bandwidth of the corresponding network of the plurality of networks.
2. The system of claim 1 , wherein the input frames are provided to the comparator circuit from a video capture device.
3. The system of claim 1 , further comprising a pre-processor that processes frames prior to encoding, and wherein the input frames are provided to the comparator circuit from the pre-processor.
4. The system of claim 3 , wherein the input frames are scaled versions of frames captured by a video capture device.
5. The system of claim 1 , further comprising a transmitter adapted to transmit encoded frames over a network.
6. The system of claim 1 , wherein the comparator circuit applies at least one of Structural Similarity (SSIM), Peak Signal to Noise Ratio (PSNR), Mean Squared Error (MSE), and Multi-Scale SSIM (MS-SSIM) to determine the fitness metric.
7. The system of claim 1 , wherein the encoding controller circuit is adapted to adjust the at least one encoding parameter based on a complexity level of the video stream.
8. The system of claim 7 , wherein the complexity level of the video stream is determined based on motion estimation analysis of the input frames of the video stream.
9. The system of claim 7 , wherein the complexity level is based on an indicator identifying a type of broadcast being encoded to generate the video stream.
10. The system of claim 7 , wherein when the video stream has a high complexity level, adjusting the at least one encoding parameter comprises: increasing a frame rate, and decreasing a frame size to accommodate the available bandwidth.
11. The system of claim 1 , wherein the encoding controller circuit is configured for adjusting the fitness metric based on a complexity level of the video stream.
12. The system of claim 1 , wherein the input frames compared by the comparator circuit to determine the fitness metric are pre-processed and scaled down input frames.
13. The system of claim 1 , wherein, the encoded frames compared by the comparator circuit to determine the fitness metric are encoded frames re-scaled to an original size of the input frames.
14. A method for transmitting a video stream across a plurality of connected networks, each network having a corresponding available bandwidth, the method comprising:
controlling a plurality of interfaces for transmitting the video stream over the plurality of connected networks, each interface corresponding to a specific network of the plurality of connected networks;
generating individual video streams each comprising a plurality of the encoded frames;
encoding the frames of the video stream according to at least one encoding parameter for each of network of the plurality of connected networks for transmission across a corresponding network at an estimated maximum bit rate derived based on a predicted available bandwidth available across all of the plurality of connected networks;
comparing the encoded frame with an input frame to determine a fitness metric reflective of visual quality of the encoded frame upon detecting a change in the predicted available bandwidth determined from feedback data obtained by the connection controller circuit corresponding to each network;
receiving data corresponding to the available bandwidth for each network of the plurality of connected networks over which the video stream is transmitted; and
controlling an encoder to adjust the at least one encoding parameter for each of the individual video streams based on the available bandwidth of the corresponding network and the fitness metric, the encoder cooperating with a comparator to iteratively adjust the at least one encoding parameter until the fitness metric meets or exceeds a pre-defined minimum value to optimize visual quality of the video stream for a bit rate determined based at least on the available bandwidth of the corresponding network of the plurality of networks.
15. The method of claim 14 , further comprising:
encoding a further frame of the video stream according to the adjusted at least one encoding parameter.
16. The method of claim 14 , further comprising:
pre-processing the input frame, the pre-processing comprising scaling the input frame.
17. The method of claim 14 , wherein the fitness metric is determined based on at least one of Structural Similarity (SSIM), Peak Signal to Noise Ratio (PSNR), Mean Squared Error (MSE), and Multi-Scale SSIM (MS-SSIM).
18. The method of claim 14 , further comprising adjusting the at least one encoding parameter based on a correlated parameter.
19. The method of claim 18 , wherein the correlated parameter is a packet loss rate.
20. The method of claim 14 , further comprising adjusting the at least one encoding parameter based on a maximum bit rate.
21. The method of claim 14 , further comprising adjusting the at least one encoding parameter based on a complexity level of the video stream.
22. The method of claim 21 , wherein the complexity level of the video stream is determined based on motion estimation analysis of the input frames of the video stream.