Determination of a convolutional neural network (CNN) for automatic target recognition in a resource constrained environment
Methods and structures are presented for implementing an automatic target recognition system as a convolutional neural network (CNN) in a satellite or other environment with constrained resources, such as limited memory capacity and limited processing capability. For example, this allows for the automatic target recognition to be implemented on a field programmable gate array (FPGA). Image data is split into subsets of contiguous pixels, with the subsets processed in parallel in a CNN of a corresponding processing node using quantized weight values that are determined in a training process that accounts for the constraints of the automatic target recognition system. The results of the automatic target recognition process is based on the combined output of the processing nodes.
1. A method, comprising:
receiving data on available memory capacity of a target recognition system;
receiving data on available processing capability of the target recognition system;
receiving training image data for a two dimensional array of pixel values; and
determining a neural network to perform image identification on the target recognition system, comprising:
determining from one or both of the available memory capacity of the target recognition system and the available processing capability of the target recognition system a number N of a plurality of subsets of the two dimensional array of pixel values, each of the subsets comprising pixel values of a plurality of contiguous pixel locations;
determining from one or both of the available memory capacity of the target recognition system and the available processing capability of the target recognition system a network structure of N processing nodes, each including a neural network of a plurality of layers configured to process a corresponding one of the subsets of pixel values;
separating the training image data into the plurality of subsets of pixel values for the training image data; and
performing a training operation to determine a set of weight values for the neural network of each of the processing nodes using the corresponding subset of pixel values for the training image data.
2. The method of claim 1 , wherein each of the subsets of pixel values correspond to an M by M square of contiguous pixel locations, where M is an integer greater than 1.
3. The method of claim 1 , wherein each of N processing nodes is configured to provide a corresponding intermediate result and determining the neural network further comprises:
determining one or more shared layers configured to receive the intermediate results and generate therefrom a target recognition result.
4. The method of claim 3 , wherein the one or more shared layers includes a softmax layer.
5. The method of claim 3 , wherein the one or more shared layers includes a sigmoid layer.
6. The method of claim 1 , wherein performing the training operation to determine the set of weight values comprises:
training the neural network using floating point values for the weight values; and
quantizing the floating point weight values.
7. The method of claim 1 , wherein determining the neural network further comprises:
determining a hardware structure for the target recognition system from the determined network structure of N processing nodes.
8. The method of claim 7 , wherein determining the neural network further comprises:
determining an instruction set for the hardware structure.
9. The method of claim 7 , wherein determining the neural network further comprises:
configuring target recognition system according to the determined hardware structure.
10. The method of claim 9 , wherein target recognition system is implemented on a field programmable gate array (FPGA) circuit.
11. The method of claim 1 , wherein determining the neural network further comprises:
loading the determined sets of weight values into memory of the target recognition system.
12. The method of claim 11 , wherein the target recognition system is on a satellite and loading the determined sets of weight values into memory of the target recognition system comprises:
transmitting the loading the determined sets of weight values through an antenna of the satellite.
13. The method of claim 11 , wherein determining the neural network further comprises:
subsequent to loading the determined sets of weight values into memory of the target recognition system, determining an updated set of weight values; and
loading the updated set of weight values into the memory of the target recognition system.
14. The method of claim 1 , wherein, subsequent to performing the training operation, determining the neural network further comprises:
determining an updated value N′ for the number of the plurality of subsets of pixel values;
determining an updated network structure of N′ processing nodes;
separating the training image data into the plurality of N′ subsets of pixel values for the training image data; and
performing an updated training operation to determine a set of weight values for the neural network of each of the processing nodes using the corresponding N′ subset of pixel values for the training image data.
15. The method of claim 1 , wherein, subsequent to performing the training operation, determining the neural network further comprises:
determining an updated network structure of the N processing nodes; and
performing an updated training operation to determine a set of weight values for the neural network of each of the processing nodes of the updated network structure of the N processing nodes.
16. A system, comprising:
one or more interfaces configured to:
receive training image data for a two dimensional array of pixel values;
receive a number N of a plurality of subsets of a two dimensional array of pixel values, each of the subsets comprising pixel values of a plurality of contiguous pixel locations; and
receive a convolutional neural network (CNN) structure of a plurality of layers that is configured to process an input corresponding one of the subsets of pixel values; and
one or more processors connected to the one or more interfaces and configured to:
determine a neural network for a target recognition system that is configured to perform image identification on an image from the two dimensional array of pixel values, the neural network comprising N processing nodes each configured to process in parallel by the CNN of a corresponding one of the subsets of pixel values;
separate the training image data into the plurality of subsets of pixel values for the training image data; and
perform a training operation to determine a set of weight values for the CNN of each of the processing nodes using the corresponding subset of pixel values for the training image data.
17. The system of claim 16 , wherein the one or more processors are further configured to:
determine a hardware structure for implementing the neural network.
18. The system of claim 17 , wherein the target recognition system is implemented on a field programmable gate array and the one or more processors are further configured to:
configure the field programmable gate array to implement the neural network according to the determined hardware structure.
19. A method, comprising:
receiving data on one or both of an available memory capacity and an available processing capability of a field programmable gate array (FPGA);
receiving training image data for a two dimensional array of pixel values;
determining from one or both of the available memory capacity and the available processing capability of the FPGA a number N of a plurality of subsets of the two dimensional array of pixel values, each of the subsets comprising pixel values of a plurality of contiguous pixel locations;
determining from one or both of the available memory capacity and the available processing capability of the FPGA a convolutional neural network (CNN) structure of a plurality of layers that is configured to process an input corresponding one of the subsets of pixel values;
determine a neural network for a target recognition system implemented on the FPGA that is configured to perform image identification on an image from the two dimensional array of pixel values, the neural network comprising N processing nodes each configured to process in parallel by the CNN of a corresponding one of the subsets of pixel values;
separating the training image data into the plurality of subsets of pixel values for the training image data; and
performing a training operation to determine a set of weight values for the CNN of each of the processing nodes using the corresponding subset of pixel values for the training image data.
20. The method of claim 19 , further comprising:
determining a hardware structure for implementing the neural network on the FPGA; and
configuring the FPGA according to the determined hardware structure.