Method and apparatus for feature transforming and processing based on artificial neural network
Disclosed herein are a method and apparatus for processing feature information based on an artificial neural network. According to an embodiment of the present disclosure, the apparatus for processing feature information based on an artificial neural network may include a memory for storing data and a processor for controlling the memory, and the processor may further be configured to extract a graph, which includes vertices, based on a feature map of an image, to extract a feature vector corresponding to the vertices and to process the graph and the feature vector based on an artificial neural network, and the graph may include positions of the vertices and information on a connection relationship between the vertices.
1. An apparatus for processing feature information based on an artificial neural network, the apparatus comprising:
a memory configured to store data; and
a processor configured to control the memory and perform an encoding operation and a decoding operation,
in the encoding operation, the processor is configured to:
obtain a first feature map of a first image through a first convolution layer,
obtain an importance map for the first feature map by passing the first feature map through a second convolution layer,
obtain first vertices, required for extracting a graph feature for the first image, using the first feature map and the importance map,
extract a first graph, which includes the first vertices, based on the first feature map,
obtain first feature vectors corresponding to the first vertices by passing per-pixel feature vectors extracted from the first feature map through a graph convolution layer, and
process the first graph and the first feature vectors based on an artificial neural network, and
wherein the first graph includes positions of the first vertices and information on a connection relationship between the first vertices,
in the decoding operation, the processor is configured to:
obtain a second graph including second vertices, and second feature vectors, which are processed based on the artificial neural network,
output a second feature map of a second image based on the second graph and the second feature vectors, and
obtain a per-pixel analysis result of the second image based on the second feature map, and
wherein the second graph includes positions of the second vertices and information on a connection relationship between the second vertices.
2. The apparatus of claim 1 , wherein the artificial neural network is a graph convolution network (GCN).
3. The apparatus of claim 1 , wherein the information on the connection relationship between the first vertices includes an edge set and an adjacency matrix for the first vertices.
4. The apparatus of claim 3 , wherein the edge set and the adjacency matrix are obtained using a k-nearest neighbors algorithm, a random sampling algorithm, or a Delaunay triangulation algorithm.
5. The apparatus of claim 1 , wherein the first feature vectors are obtained using a k-nearest neighbors algorithm or a random sampling algorithm.
6. The apparatus of claim 1 , wherein the second feature map of the second image is output through a third convolution layer.
7. The apparatus of claim 1 , wherein the per-pixel analysis result is obtained through fully-connected layers.
8. A method for processing feature information based on an artificial neural network, the method comprising:
performing an encoding operation comprising:
obtaining a first feature map of a first image through a first convolution layer;
obtaining an importance map for the first feature map by passing the first feature map through a second convolution layer;
obtaining first vertices, required for extracting a graph feature for the first image, using the first feature map and the importance map;
extracting a first graph consisting of the vertices based on the first feature map;
obtaining first feature vectors corresponding to the first vertices by passing per-pixel feature vectors extracted from the first feature map through a graph convolution laver; and
processing the first graph and the first feature vectors based on an artificial neural network,
wherein the first graph includes positions of the first vertices and information on a connection relationship between the first vertices, and
performing a decoding operation comprising:
obtaining a second graph including second vertices, and second feature vectors, which are processed based on the artificial neural network,
outputting a second feature map of a second image based on the second graph and the second feature vectors, and
obtaining a per-pixel analysis result of the second image based on the second feature map, and
wherein the second graph includes positions of the second vertices and information on a connection relationship between the second vertices.
9. The method of claim 8 , wherein the artificial neural network is a graph convolution network (GCN).
10. The method of claim 8 , wherein the information on the connection relationship between the first vertices includes an edge set and an adjacency matrix for the first vertices.
11. The method of claim 10 , wherein the edge set and the adjacency matrix are obtained using a k-nearest neighbors algorithm, a random sampling algorithm, or a Delaunay triangulation algorithm.
12. The method of claim 10 , wherein the first feature vectors are obtained using a k-nearest neighbors algorithm or a random sampling algorithm.