Look-up convolutional layer in convolutional neural network
Embodiments provide for a processor including logic to accelerate convolutional neural network processing, the processor including first logic to apply a convolutional layer to an image to generate a first convolution result and second logic to apply a look-up convolutional layer to the first convolution result to generate a second convolution result, the second convolution result associated with a location of the first convolution result within a global filter kernel.
1. An apparatus comprising:
one or more processors to accelerate convolutional neural network processing, the one or more processors to:
apply a convolutional layer to an image to generate a first convolution result;
apply a look-up convolutional layer to the first convolution result to generate a second convolution result, the second convolution result associated with a location of the first convolution result within a global filter kernel; and
estimate a position and an orientation of a camera associated with the image.
2. The apparatus as in claim 1 , wherein the global filter kernel is significantly larger than the first convolution result and the second convolution result is an enlarged feature map.
3. The apparatus as in claim 2 , wherein the global filter kernel is associated with an environment filter map for use in camera pose estimation.
4. The apparatus as in claim 3 , wherein the environmental filter map is represented by a global filter kernel having a shape defined by a view angle of an imaging module used to capture the image.
5. The apparatus as in claim 3 , wherein the one or more processors are further to process the second convolution result to estimate a camera pose for a camera associated with the image.
6. The apparatus as in claim 5 , wherein the one or more processors are further to apply one or more fully-connected neural network layers to the enlarged feature map to generate a value associated with a camera pose estimate.
7. The apparatus as in claim 5 , wherein the one or more processors are further to apply one or more fully-connected neural network layers to a subsample of the enlarged feature map to generate a value associated with a camera pose estimate.
8. The apparatus as in claim 5 , wherein the one or more processors are further to estimate the camera pose for a camera associated with an autonomous robot.
9. The apparatus as in claim 5 , wherein the one or more processors are further to estimate the camera pose for a camera associated with an autonomous vehicle.
10. The apparatus as in claim 1 , wherein the one or more processors comprise a graphics or image processor core to receive one or more convolutional kernels associated with the convolutional layer and at least one look-up convolutional kernel associated with the look-up convolution layer.
11. A computer processing method executed by one or more processors, the method comprising:
applying a convolutional layer of a convolutional neural network to an image to generate a first feature map;
applying a look-up convolution layer to the first feature map to generate a second feature map, the second feature map associated with a location of the first feature map within a look-up convolution kernel; and
estimating a position and an orientation of a camera associated with the image.
12. The method as in claim 11 , wherein the image is a multi-dimensional image of an environment and the look-up convolution kernel is a global feature kernel associated with a localization environment map.
13. The method as in claim 12 , wherein the global feature kernel is selected based on a camera configuration of an imaging unit associated with a localization module of an autonomous vehicle or autonomous robot.
14. At least one computer-readable medium having stored thereon instructions which, when executed, cause a computing device to perform operations comprising:
applying a convolutional layer of a convolutional neural network to an image to generate a first feature map;
applying a look-up convolution layer to the first feature map to generate a second feature map, the second feature map associated with a location of the first feature map within a look-up convolution kernel; and
estimating a position and an orientation of a camera associated with the image.
15. The computer-readable medium as in claim 14 , wherein applying a look-up convolution layer to the first feature map includes applying multiple look-up convolution kernels to the image.
16. The computer-readable medium as in claim 15 , wherein the image is a multi-dimensional image having one or more of a red, green, and blue channel.
17. The computer-readable medium as in claim 14 , wherein the image is a multi-dimensional image of an environment and the look-up convolution kernel is a global feature kernel associated with a localization environment map.
18. The computer-readable medium as in claim 17 , wherein the global feature kernel is selected based on a camera configuration of an imaging unit associated with a localization module of an autonomous vehicle or autonomous robot.
19. The computer-readable medium as in claim 18 , wherein the global feature kernel is associated with a cylindrical environment map.
20. The computer-readable medium as in claim 18 , wherein the global feature kernel is associated with a spherical environment map.