Video system
A system configuration suitable for carrying out video recognition by machine learning in a video system including an imaging device and a video processing device is provided. One aspect of the video system according to the present invention includes an imaging device and a video processing device. The imaging device includes an imaging unit configured to generate video data by imaging, a front side NN unit including multiple layers of neural networks from an input layer to a predetermined intermediate hidden layer (hereinafter referred to as “intermediate hidden layer”) that are configured to perform recognition processing of the video data; and an imaging side control unit configured to transmit an output of the intermediate hidden layer of the front side NN unit (hereinafter referred to as “intermediate feature quantity”). The video processing device includes a video processing side control unit configured to acquire the intermediate feature quantity transmitted from the imaging side control unit, and a rear side NN unit including the multiple layers of the neural networks from a layer subsequent the intermediate hidden layer to an output layer, the rear side NN unit configured to perform the remaining recognition processing regarding the intermediate feature quantity acquired by the video processing side control unit.
1. A video system comprising an imaging device and a video processing device, wherein
the imaging device comprises:
an imaging unit configured to generate video data by imaging;
a front side NN unit comprising multiple layers of neural networks from an input layer to a predetermined intermediate hidden layer (hereinafter referred to as “intermediate hidden layer”) that are configured to perform recognition processing of the video data; and
an imaging side control unit configured to transmit an output of the intermediate hidden layer of the front side NN unit (hereinafter referred to as “intermediate feature quantity”) comprising an identifier configured to identify abnormality according to the intermediate feature quantity, and a transmission data selection unit configured to not output the intermediate feature quantity to the video processing device in a case where the identifier identifies that there is no abnormality, and the video processing device comprises:
a video processing side control unit configured to acquire the intermediate feature quantity transmitted from the imaging side control unit; and
a rear side NN unit comprising the multiple layers of the neural networks from a layer subsequent the intermediate hidden layer to an output layer, the rear side NN unit configured to perform a remaining recognition processing regarding the intermediate feature quantity acquired by the video processing side control unit.
2. The video system according to claim 1 , wherein
the transmission data selection unit is configured not to output the video data to the video processing device in a case where the identifier identifies that there is no abnormality.
3. The video system according to claim 1 , wherein
at least one of the imaging side control unit and the video processing side control unit comprises
an identifier learning unit configured to perform learning of the identifier based on the intermediate feature quantity and a processing result of the rear side NN unit.
4. The video system according to claim 2 , wherein
at least one of the imaging side control unit and the video processing side control unit comprises
an identifier learning unit configured to perform learning of the identifier based on the intermediate feature quantity and a processing result of the rear side NN unit.
5. A video system comprising an imaging device and a video processing device, wherein
the imaging device comprises:
an imaging unit configured to generate video data by imaging;
a front side NN unit comprising multiple layers of neural networks from an input layer to a predetermined intermediate hidden layer (hereinafter referred to as “intermediate hidden layer”) that are configured to perform recognition processing of the video data; and
an imaging side control unit configured to transmit an output of the intermediate hidden layer of the front side NN unit (hereinafter referred to as “intermediate feature quantity”), and
the video processing device comprises:
a video processing side control unit configured to acquire the intermediate feature quantity transmitted from the imaging side control unit; and
a rear side NN unit comprising the multiple layers of the neural networks from a layer subsequent the intermediate hidden layer to an output layer, the rear side NN unit configured to perform a remaining recognition processing regarding the intermediate feature quantity acquired by the video processing side control unit, and
the video processing device or an exterior thereof comprises:
an entire NN model configured to implement an entire node of the neural networks, and
an NN learning unit configured to perform a learning processing with respect to the entire NN model and determine a configuration parameter of each node of the entire NN model,
the imaging side control unit is configured to
acquire the configuration parameter corresponding to each node of the front side NN unit from the NN learning unit and set the same to each node of the front side NN unit, and
the video processing side control unit is configured to
acquire the configuration parameter corresponding to each node of the rear side NN unit from the NN learning unit and set the same to each node of the rear side NN unit.
6. The video system according to claim 5 , wherein
at least one of the imaging side control unit and the video processing side control unit comprises:
a node allocation unit configured to change allocation of a number of nodes of the front side NN unit and a number of nodes of the rear side NN unit, and
wherein the front side NN unit and the rear side NN unit communicates a configuration parameter of a node being transferred according to the allocation of the number of nodes being changed by the node allocation unit.
7. The video system according to claim 6 , wherein
the node allocation unit is configured to acquire a load status of the imaging device or the video processing device and change the allocation of the number of nodes of the front side NN unit and the number of nodes of the rear side NN unit in response to the load status.
8. A video system comprising an imaging device and a video processing device, wherein
the imaging device comprises:
an imaging unit configured to generate video data by imaging;
a front side NN unit comprising multiple layers of neural networks from an input layer to a predetermined intermediate hidden layer (hereinafter referred to as “intermediate hidden layer”) that are configured to perform recognition processing of the video data; and
an imaging side control unit configured to transmit an output of the intermediate hidden layer of the front side NN unit (hereinafter referred to as “intermediate feature quantity”), and
the video processing device comprises:
a video processing side control unit configured to acquire the intermediate feature quantity transmitted from the imaging side control unit; and
a rear side NN unit comprising the multiple layers of the neural networks from a layer subsequent the intermediate hidden layer to an output layer, the rear side NN unit configured to perform a remaining recognition processing regarding the intermediate feature quantity acquired by the video processing side control unit, and
at least one of the imaging side control unit and the video processing side control unit comprises
a node allocation unit configured to change allocation of a number of nodes of the front side NN unit and a number of nodes of the rear side NN unit, and
wherein the front side NN unit and the rear side NN unit communicates a configuration parameter of a node being transferred according to the allocation of the number of nodes being changed by the node allocation unit.
9. The video system according to claim 8 , wherein
the node allocation unit is configured to acquire a load status of the imaging device or the video processing device and change the allocation of the number of nodes of the front side NN unit and the number of nodes of the rear side NN unit in response to the load status.
10. The video system according to claim 9 , wherein
a relative load balance between the imaging device and the video processing device is acquired, wherein in a state where a load of the imaging device is increased, a large number of nodes is allocated to the rear side NN unit, and in a state where a load of the video processing device is increased, a large number of nodes is allocated to the front side NN unit.
11. The video system according to claim 10 , wherein
the node allocation unit is configured to move a position of the intermediate hidden layer in a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby cut down an amount of communicated information of the intermediate feature quantity.
12. The video system according to claim 10 , wherein
the node allocation unit is configured to
acquire information regarding a communication traffic that varies according to a number of imaging devices connected to the video processing device, an amount of information entered from the imaging devices and a frequency thereof, and
in a state where the communication traffic is excessive, move a position of the intermediate hidden layer toward a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby reduce the amount of communicated information of the intermediate feature quantity.
13. The video system according to claim 9 , wherein
the node allocation unit is configured to move a position of the intermediate hidden layer in a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby cut down an amount of communicated information of the intermediate feature quantity.
14. The video system according to claim 9 , wherein
the node allocation unit is configured to
acquire information regarding a communication traffic that varies according to a number of imaging devices connected to the video processing device, an amount of information entered from the imaging devices and a frequency thereof, and
in a state where the communication traffic is excessive, move a position of the intermediate hidden layer toward a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby reduce the amount of communicated information of the intermediate feature quantity.
15. The video system according to claim 8 , wherein
the node allocation unit is configured to acquire a load status of the imaging device based on at least one of a group consisting of temperature detection, current detection, video processing delay, and change of processing frame rate.
16. The video system according to claim 8 , wherein
a relative load balance between the imaging device and the video processing device is acquired, wherein in a state where a load of the imaging device is increased, a large number of nodes is allocated to the rear side NN unit, and in a state where a load of the video processing device is increased, a large number of nodes is allocated to the front side NN unit.
17. The video system according to claim 16 , wherein
the node allocation unit is configured to move a position of the intermediate hidden layer in a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby cut down an amount of communicated information of the intermediate feature quantity.
18. The video system according to claim 16 , wherein
the node allocation unit is configured to
acquire information regarding a communication traffic that varies according to a number of imaging devices connected to the video processing device, an amount of information entered from the imaging devices and a frequency thereof, and
in a state where the communication traffic is excessive, move a position of the intermediate hidden layer toward a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby reduce the amount of communicated information of the intermediate feature quantity.
19. The video system according to claim 8 , wherein
the node allocation unit is configured to move a position of the intermediate hidden layer in a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby cut down an amount of communicated information of the intermediate feature quantity.
20. The video system according to claim 8 , wherein
the node allocation unit is configured to
acquire information regarding a communication traffic that varies according to a number of imaging devices connected to the video processing device, an amount of information entered from the imaging devices and a frequency thereof, and
in a state where the communication traffic is excessive, move a position of the intermediate hidden layer toward a direction to reduce a product of the number of nodes of the intermediate hidden layer and a transmission frequency of the intermediate feature quantity to thereby reduce the amount of communicated information of the intermediate feature quantity.