Method for generating a reconstructed image
A method for generating reconstruction a reconstructed image is adapted to an input image having a target object. The method comprises converting the input image into a feature map with vectors by an encoder; performing a training procedure according to training images of reference objects to generate feature prototypes associated with the training images and store the feature prototypes to a memory; selecting a part of feature prototypes from the feature prototypes stored in the memory according to similarities between the feature prototypes and the feature vectors; generating a similar feature map according the part of feature prototypes and weights, wherein the weights represents similarities between the part of feature prototypes and the feature vectors; and converting the similar feature map into the reconstructed image by a decoder; wherein the encoder, the decoder and the memory form an auto-encoder.
1. A method for generating a reconstructed image adapted to an input image having a target object, comprising:
converting the input image into a feature map with a plurality of feature vectors by an encoder;
performing a training procedure according to a plurality of training images of a plurality of reference objects to generate a plurality of feature prototypes associated with the plurality of training images and storing the plurality of feature prototypes to a memory;
selecting a part of feature prototypes from the plurality of feature prototypes stored in the memory according to a plurality of similarities between the plurality of feature prototypes and the plurality of feature vectors;
generating a similar feature map according the part of feature prototypes and a plurality of weights, wherein each of the plurality of weights represents each of the similarities between the part of feature prototypes and the plurality of feature vectors; and
converting the similar feature map into the reconstructed image by a decoder, wherein the encoder, the decoder and the memory form an auto-encoder.
2. The method for generating the reconstructed image of claim 1 , wherein selecting the part of feature prototypes from the plurality of feature prototypes stored in the memory according to the plurality of similarities between the plurality of feature prototypes and the plurality of feature vectors comprises:
selecting a part of weights from large to small in the plurality of weights;
selecting the plurality of feature prototypes corresponding to the part of weights;
calculating a plurality of sparse similarities according to a number of the part of weights; and
serving the plurality of sparse similarities as the plurality of weights.
3. The method for generating the reconstructed image of claim 1 , wherein performing the training procedure according to the plurality of training images of the plurality of reference objects comprises:
setting a default feature prototype;
for each of the plurality of training images, dividing the training image into a plurality of patches;
converting the plurality of patches into a plurality of patch features respectively by the encoder;
calculating a plurality of distances between the plurality of patch features and the default feature prototype;
saving at least one patch feature whose distance is smaller than a threshold; and
updating the default feature prototype as one of the plurality of feature prototypes according to the saved at least one patch features.
4. The method for generating the reconstructed image of claim 1 , wherein performing the training procedure according to the plurality of training images of the plurality of reference objects comprises:
setting a default feature prototype;
for each of the plurality of training images, converting the training image into a plurality of patch features by the encoder;
calculating a plurality of distances between the plurality of patch features and the default feature prototype;
saving at least one patch feature whose distance is smaller than a threshold; and
updating the default feature prototype as one of the plurality of feature prototypes according to the saved at least one patch feature.
5. The method for generating the reconstructed image of claim 1 , wherein the plurality of weights is calculated by the following equation:
ω
i
=
exp
(
-
z
-
M
i
2
)
∑
j
=
1
M
exp
(
-
z
-
M
j
2
)
,
i
=
{
1
,
…
,
M
}
,
where w i is the i th weight, z is one of the plurality of feature vectors, M i is the i th feature prototype stored in the memory, and M is a number of slots of the memory.
6. The method for generating the reconstructed image of claim 3 , wherein the threshold is an average of the plurality of distances.
7. The method for generating the reconstructed image of claim 3 , wherein the auto-encoder has a loss function associated with the following equation:
L trust =r ( z,M (1) )∥ z−M (1) ∥ 2 ,
wherein L trust is one term in the loss function, z is one of the plurality of feature vectors, M (1) is one of the plurality of feature prototypes that is closest to z, and r(z, M (1) ) is configured to indicate whether a distance between z and M is smaller than the threshold.
8. The method for generating the reconstructed image of claim 7 , wherein r(z, M (1) ) is calculated by the following equation:
r
(
z
,
M
(
1
)
)
=
{
1
z
-
M
(
1
)
2
≤
δ
1
-
1
δ
1
<
z
-
M
(
1
)
2
≤
δ
2
0
otherwise
,
wherein δ 1 is the threshold and δ 2 is another threshold greater than δ 1 .