System to optimize video streaming of USB camera via bulk endpoint
A system and method to optimize video streaming of a plurality of USB cameras streaming via a bulk interface of a universal serial bus (USB), the system comprising a host unit communicating on universal serial bus (USB) via the bulk interface, a throughput analyzer computer program residing in the host unit, at least a USB Camera with a varying throughput, a dynamically programmable image sensor varies a throughput of the USB camera, a throughput control block of the USB camera receives input from the throughput analyzer computer program and adjusts the throughput of the camera, the USB controller regulates a transmission of USB packet buffers from each of the USB Camera to the host unit. The method comprises assessing by the throughput analyzer computer program of the universal serial bus utilization by the host unit by including all connected devices, and includes selectively dropping a plurality of data packet/frames.
1 . A system to optimize video streaming of a plurality of universal serial bus (USB) cameras, streaming via a universal serial bus (USB) having a maximum throughput in a direction, the system comprising:
a USB camera capable of capturing video data, having,
a throughput control block for adjusting the throughput of the USB camera, having an image sensor capturing video data according to a throughput control signal, connected to the image sensor and providing thereto an image sensor control signal,
the image sensor providing, in response to said image sensor control signal from said throughput control block, an image sensor output of said image sensor captured video data adjusted according to throughput management parameters having at least one of a variable complimentary metal-oxide-semiconductor pixel clock, a controllable horizontal blanking time, and a selective packet drop, and
a USB controller, having a statistics block, connected to receive said image sensor output and to receive a streaming control signal from said throughput control block, and providing a USB output; and
a host unit including,
a USB application responsive to said USB Camera USB output, and
a throughput analyzer which provides said throughput control signal to said throughput control block, in response to said USB output, to selectively control at least one of said throughput control block to provide said USB output no greater than said USB maximum throughput.
2 . The system to optimize video streaming of the plurality of USB cameras as claimed in claim 1 , wherein the throughput of the USB camera is varied by the throughput control block varying the CMOS pixel clock of the USB camera.
3 . The system to optimize video streaming of the plurality of USB cameras as claimed in claim 1 , wherein the throughput of the USB camera is varied by the throughput control block varying the horizontal blanking line time.
4 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the throughput of the USB camera is varied by the throughput control block dropping a plurality of frames.
5 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the throughput of the USB camera is varied by the throughput control block configurably dropping a plurality of non-critical adjacent frames while a plurality of critical adjacent frames are retained.
6 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 5 , wherein the plurality of USB cameras is comprised of a USB camera or a USB Video Class (UVC) device or a combination thereof, wherein the plurality of critical adjacent frames are referenced with a prescribed object, and identified by monitoring an image statistics from a statistics block of the USB camera or a USB Video Class (UVC) Device.
7 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the throughput analyzer computer program continuously monitors a USB bus throughput and provides real-time updates of an available bandwidth of the USB bus characterized as real-time USB bus throughput.
8 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 7 , wherein the throughput analyzer computer program monitors an ongoing video streaming data from the USB camera and compares against the real-time USB bus throughput, wherein a difference thereof is provided as a continuous feedback to the throughput control block of the USB camera.
9 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the throughput analyzer computer program assesses a USB bus throughput on a demand from the throughput control block to communicate available bandwidth.
10 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 9 , wherein the throughput analyzer computer program enables a static test data from the USB camera and validates multiple data rates, wherein a data rate without an involuntary packet drop is provided as feedback to the throughput control block of the USB camera.
11 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the throughput analyzer periodically monitors a USB bus throughput and provides updates of an available bandwidth of the USB.
12 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 11 , wherein the throughput analyzer computer program enables a static test data from the USB camera and compares against the USB bus current throughput sent from the USB camera device, wherein a difference thereof is provided as feedback to the throughput control block of the device.
13 . The system to optimize video streaming of a plurality of USB cameras as claimed in claim 1 , wherein the system further comprises a circular buffer memory device, situated between the plurality of USB cameras and a streaming application of the host unit, incorporating a circular buffer architecture temporarily parking a plurality of data packets of video streaming.
14 . A method to optimize video streaming of one or more of universal serial bus (USB) cameras streaming via a BULK interface of universal serial bus (USB) to a host unit, the method comprising the steps of:
a) providing a video streaming data transfer between one or more USB cameras and said host unit at data transfer rate,
b) evaluating, by monitoring a USB bus connected to one or more USB cameras and providing updates on available USB bandwidth according to a throughput analyzer computer program, a USB version supported by the host unit and a corresponding maximum transfer rate for the BULK interface,
c) assessing, by the throughput analyzer computer program, a utilization amount of the USB bus by the host unit by including all devices connected to the USB bus,
d) monitoring as usage amounts, a memory usage and a CPU usage of the host unit to correspondingly alter a transfer rate, a resolution, and or a frame rate of the one or more USB cameras,
e) providing a throughput control signal in response to one or more of said evaluating USB version, assessing utilization amount of the USB bus, and monitoring usage amounts, and
f) restricting a data transfer rate of the one or more USB cameras in response to said throughput control signal.
15 . The method to optimize video streaming as claimed in claim 14 , wherein the restricting the data transfer rate is different for each of the one or more USB cameras.
16 . The method to optimize video streaming as claimed in claim 14 , wherein the restricting the data transfer rate is by varying a complementary metal oxide semiconductor (CMOS) pixel clock of an image sensor of the one or more USB cameras.
17 . The method to optimize video streaming as claimed in claim 14 , wherein the restricting the data transfer rate is by varying the horizontal blanking line time of an image sensor of the one or more USB cameras.
18 . The method to optimize video streaming as claimed in claim 14 , wherein the restricting the data transfer rate is by selectively dropping at least one data packet of the one or more USB cameras.
19 . The method to optimize video streaming as claimed in claim 14 , wherein the throughput analyzer computer program tracks free buffers zones in a circular buffer architecture wherein a data overflow occurs when a free buffer zone count between a source pointer and a sink pointer is at zero of the circular buffer.
20 . The method to optimize video streaming as claimed in claim 19 , including providing a low threshold, a medium threshold and a high threshold, wherein the multiple threshold of the free buffer zone count are created for a corresponding low packet drop rate, a medium packet drop rate and a high packet drop rate.
21 . The method to optimize video streaming as claimed in claim 19 , wherein the throughput analyzer tracks a time taken for a data packet to travel from the source pointer to the sink pointer of the circular buffer.
22 . The method to optimize video streaming as claimed in claim 18 , further providing dropping a data packet of non-critical adjacent frames.
23 . The method to optimize video streaming as claimed in claim 18 , wherein the restricting the data transfer rate is by dropping a data packet of unreferenced non-critical adjacent frames.
24 . The method to optimize video streaming as claimed in claim 18 , wherein the restricting the data transfer rate is by dropping a data packet of non-critical adjacent frames while critical adjacent frames identified with respect to a prescribed object are retained.
25 . The method to optimize video streaming as claimed in claim 24 , wherein the one or more USB cameras consist of a USB camera or a USB Video Class (UVC) Device or a combination thereof, and wherein the identifying of the critical adjacent frames is by monitoring a luma based and a chroma based image statistics from a statistics block of the USB Video Class (UVC) Camera/UVC Device.