Forward propagation apparatus, learning apparatus, information processing system, processing method, and non-transitory computer readable medium storing program
A forward propagation apparatus is a forward propagation apparatus for a neural network, including: a mask generation unit that generates a binary mask; and a layer execution unit that performs an operation for a sparse convolutional layer according to a value at each coordinate of the binary mask, in which the mask generation unit: generates heat maps by performing an operation for a convolutional layer on an input feature map; generates a composite heat map obtained by combining the heat maps, into one heat map by summing up values of heat maps on a coordinate-by-coordinate basis; and generates the binary mask by binarizing a value at each coordinate of the composite heat map by using a predetermined threshold.
1 . A forward propagation apparatus for a neural network, comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
generate heat maps by performing an operation for a convolutional layer on an input feature map, the number of the heat maps being equal to the number of types of objects to be detected by the neural network;
generate a composite heat map obtained by combining the heat maps, the number of which is equal to the number of types of objects to be detected, into one heat map by summing up values of the heat maps on a coordinate-by-coordinate basis;
generate a binary mask by binarizing a value at each coordinate of the composite heat map using a predetermined threshold, wherein the binary mask indicates areas of the composite heat map where the objects to be detected are present; and
perform an operation for a sparse convolutional layer according to a value at each coordinate of the binary mask, wherein the operation for the sparse convolutional layer comprises performing product-sum operations only on the areas of the composite heat map where the objects to be detected are present.
2 . The forward propagation apparatus according to claim 1 , wherein a value of a weight of the convolutional layer for generating heat maps the number of which is equal to the number of types of objects to be detected, is a value that is machine-learned by using a heat map generated based on a correct answer label.
3 . The forward propagation apparatus according to claim 1 , wherein the at least one processor is further configured to execute the instructions to change a resolution of the composite heat map according to a resolution of the sparse convolutional layer.
4 . The forward propagation apparatus according to claim 1 , wherein the at least one processor is further configured to execute the instructions to generate the binary mask for an operation for a final layer of consecutive sparse convolutional layers, and further generate a binary mask for an operation for a sparse convolutional layer preceding the final layer from the generated binary mask.
5 . The forward propagation apparatus according to claim 1 , wherein the number of types of objects to be detected is the number of types of joint points of a human being coordinates of which are detected by the neural network.
6 . The forward propagation apparatus according to claim 1 , wherein the number of types of objects to be detected is the number of classes used in object detection by the neural network.
7 . The forward propagation apparatus according to claim 1 , wherein the number of types of objects to be detected is the number of classes used in semantic segmentation by the neural network.
8 . A processing method for a forward propagation apparatus for a neural network, comprising:
generating a binary mask, and
performing an operation for a sparse convolutional layer according to a value at each coordinate of the binary mask, wherein
the generating the binary mask comprises:
generating heat maps by performing an operation for a convolutional layer on an input feature map, the number of the heat maps being equal to the number of types of objects to be detected by the neural network;
generating a composite heat map obtained by combining the heat maps, the number of which is equal to the number of types of objects to be detected, into one heat map by summing up values of the heat maps on a coordinate-by-coordinate basis; and
generating the binary mask by binarizing a value at each coordinate of the composite heat map by using a predetermined threshold, wherein the binary mask indicates areas of the composite heat map where the objects to be detected are present, and
wherein the operation for the sparse convolutional layer comprises performing product-sum operations only on the areas of the composite heat map where the objects to be detected are present.