Server for classifying image and operating method thereof
A server for classifying an image and a method of operating the server are provided. The method includes obtaining classification probability values of the image, by applying the image to an image classification model including a plurality of parallel multi-layer perceptron (MLP) layers, and classifying the image, based on the classification probability values, wherein each of the plurality of parallel MLP layers includes a first MLP and a second MLP, an operation using the first MLP and an operation using the second MLP are performed in parallel, and dimensions of data before and after an operation of each of the plurality of parallel MLP layers are same by combining an operation result of the first MLP with an operation result of the second MLP.
1. A computer vision method of classifying an image by a server, the computer vision method comprising:
obtaining, by the server, classification probability values of the image by applying the image to an image classification model including a plurality of parallel multi-layer perceptron (MLP) layers sequentially connected to each other; and
based on the classification probability values, classifying, by the server, the image,
wherein each of the plurality of parallel MLP layers comprises:
a first MLP and a second MLP, an operation using the first MLP and an operation using the second MLP being performed in parallel, and
dimensions of data before and after an operation of each of the plurality of parallel MLP layers are a same dimension by transforming an operation result of the first MLP to match a predefined original dimension, transforming an operation result of the second MLP to match the predefined original dimension, and then combining the transformed operation result of the first MLP with the transformed operation result of the second MLP to maintain the same dimension before and after the operation of each of the plurality of parallel MLP layers.
2. The computer vision method of claim 1 , wherein an operation of a first parallel MLP layer that is an initial parallel MLP layer among the plurality of parallel MLP layers comprises:
receiving image patches, as an input, obtained by splitting the image;
obtaining a first feature map by using the first MLP in the first parallel MLP layer for learning a local characteristic of each of the image patches;
obtaining a second feature map by using the second MLP in the first parallel MLP layer for learning a relationship between the image patches;
generating an output feature map based on the first feature map and the second feature map; and
transferring the output feature map to a second parallel MLP layer that is a next parallel MLP layer.
3. The computer vision method of claim 2 , wherein the generating of the output feature map comprises:
applying, by the server, an attention weight to each of the first feature map and the second feature map; and
adding, by the server, the first feature map and the second feature map to which the attention weight is applied.
4. The computer vision method of claim 2 , wherein the first MLP and the second MLP each comprise an active function layer and one or more fully connected layers.
5. The computer vision method of claim 2 , wherein the obtaining of the first feature map by using the first MLP in the first parallel MLP layer for learning the local characteristic of each of the image patches comprises:
obtaining, by the server, first vectors, in which all pixels in multi-channel image patches are linearly embedded, of the image with respect to each of the multi-channel image patches; and
applying, by the server, the first vectors to the first MLP.
6. The computer vision method of claim 5 , wherein the obtaining of the second feature map by using the second MLP in the first parallel MLP layer for learning the relationship between the image patches comprises:
obtaining, by the server, second vectors, in which pixels at a same location are linearly embedded in each of the image patches of a same channel, of the image; and
applying, by the server, the second vectors to the second MLP.
7. The computer vision method of claim 2 ,
wherein each of the plurality of parallel MLP layers further comprises a third MLP, and
wherein the third MLP performs operations in parallel with the first MLP and the second MLP.
8. The computer vision method of claim 7 ,
wherein an operation of a first parallel MLP layer that is an initial parallel MLP layer among the plurality of parallel MLP layers further comprises:
obtaining, by the server, a third feature map by using a third MLP for learning the relationship between the image patches by reflecting channel information of the image, and
wherein the generating of the output feature map of the initial parallel MLP layer comprises:
generating, by the server, the output feature map based on the first feature map, the second feature map, and the third feature map.
9. The computer vision method of claim 8 , wherein the obtaining of the third feature map by using the third MLP for learning the relationship between the image patches by reflecting the channel information of the image comprises:
identifying, by the server, pixels, of the image in each of multi-channel image patches, at a same location in the multi-channel image patches;
obtaining, by the server, third vectors in which the pixels are linearly embedded; and
applying, by the server, the third vectors to the third MLP.
10. A server for classifying an image, the server comprising:
a communication interface;
memory storing instructions; and
one or more processors communicatively coupled to the communication interface and the memory,
wherein the instructions, when executed by the one or more processors individually or collectively, cause the server to:
obtain classification probability values of the image by applying the image to an image classification model including a plurality of parallel multi-layer perceptron (MLP) layers sequentially connected to each other, and
based on the classification probability values, classify the image,
wherein each of the plurality of parallel MLP layers comprises a first MLP and a second MLP,
wherein an operation using the first MLP and an operation using the second MLP are performed in parallel, and
wherein dimensions of data before and after an operation of each of the plurality of parallel MLP layers are a same dimension by transforming an operation result of the first MLP to match a predefined original dimension, transforming an operation result of the second MLP to match the predefined original dimension, and then combining the transformed operation result of the first MLP and the transformed operation result of the second MLP to maintain the same dimension before and after the operation of each of the plurality of parallel MLP layers.
11. The server of claim 10 ,
wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
perform an operation of a first parallel MLP layer that is an initial parallel MLP layer among the plurality of parallel MLP layers, and wherein the operation of the first parallel MLP layer comprises:
receiving image patches, as an input, obtained by splitting the image;
obtaining a first feature map by using the first MLP in the first parallel MLP layer for learning a local characteristic of each of the image patches;
obtaining a second feature map by using the second MLP in the first parallel MLP layer for learning a relationship between the image patches;
generating an output feature map, based on the first feature map and the second feature map; and
transferring the output feature map to a second parallel MLP layer that is a next parallel MLP layer.
12. The server of claim 11 , wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
apply an attention weight to each of the first feature map and the second feature map; and
add the first feature map and the second feature map to which the attention weight is applied.
13. The server of claim 11 , wherein the first MLP and the second MLP each comprise an active function layer and one or more fully connected layers.
14. The server of claim 11 , wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
obtain first vectors, in which all pixels in multi-channel image patches are linearly embedded, of the image with respect to each of the multi-channel image patches; and
apply the first vectors to the first MLP.
15. The server of claim 14 , wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
obtain second vectors, in which pixels at a same location are linearly embedded in each of the image patches of a same channel, of the image; and
apply the second vectors to the second MLP.
16. The server of claim 11 ,
wherein each of the plurality of parallel MLP layers further comprises a third MLP, and
wherein the third MLP performs operations in parallel with the first MLP and the second MLP.
17. The server of claim 16 ,
wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to;
obtain the operation of the initial parallel MLP layer among the plurality of parallel MLP layers, and
wherein the operation of the initial parallel MLP layer comprises:
obtaining a third feature map by using a third MLP for learning the relationship between the image patches by reflecting channel information of the image; and
generating the output feature map based on the first feature map, the second feature map, and the third feature map.
18. The server of claim 17 , wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
identify pixels, of the image in each of multi-channel image patches, at a same location in the multi-channel image patches;
obtain third vectors in which the pixels are linearly embedded; and
apply the third vectors to the third MLP.
19. The server of claim 17 ,
wherein the instructions, when executed by the one or more processors individually or collectively, further cause the server to:
identify pixels, of the image in each of multi-channel image patches, using a random sampling method, and
wherein the pixels are embedded as one unit vector.
20. One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executed instructions that, when executed by one or more processors of a server individually or collectively, cause the server to perform computer vision operations for classifying an image by using an image classification model including a plurality of parallel multi-layer perceptron (MLP) layers sequentially connected to each other, the plurality of parallel MLP layers comprising an input parallel MLP layer, one or more hidden parallel MLP layers, and an output parallel MLP layer, each of the plurality of parallel MLP layers comprising a first MLP and a second MLP and dimensions of data before and after an operation of each of the plurality of parallel MLP layers are a same dimension by transforming an operation result of the first MLP to match a predefined original dimension, transforming an operation result of the second MLP to match the predefined original dimension, and then combining the transformed operation result of the first MLP with the transformed operation result of the second MLP to maintain the same dimension before and after the operation of each of the plurality of parallel MLP layers, the operations comprising:
generating, by the server, a plurality of image patches by splitting the image;
generating, by the server, first vectors for recognizing a local characteristic of each of the image patches;
generating, by the server, second vectors for recognizing a characteristic of a relationship between the image patches;
inputting, by the server, the first vectors into the first MLP in the input parallel MLP layer;
inputting, by the server, the second vectors into the second MLP in the input parallel MLP layer;
performing, by the server, an operation of the first MLP in the input parallel MLP layer and an operation of the second MLP in the input parallel MLP layer in parallel;
extracting, by the server, features related to the image by using the one or more hidden parallel MLP layers; and
outputting, by the server, a classification probability value of the image from the features related to the image by using the output parallel MLP layer.