Method and device for compressing image and neural network using hidden variable
A method and apparatus for image compression using a latent variable are provided. The multiple components of the latent variable may be sorted in order of importance. Through sorting, when the feature information of only some of the multiple components is used, the quality of a reconstructed image may be improved. In order to generate a latent variable, the components of which are sorted in order of importance, learning may be performed in various manners. Also, less important information may be eliminated from the latent variable, and processing, such as quantization, may be applied to the latent variable. Through elimination and processing, the amount of data for the latent variable may be reduced.
1. A processing method, comprising:
acquiring a latent variable; and
performing selection or processing related to the latent variable,
wherein multiple components of the latent variable are sorted in a specific order, and
wherein the latent variable is acquired through an encoder neural network that is trained in advance.
2. The processing method of claim 1 , wherein the multiple components are sorted in order of importance.
3. The processing method of claim 1 , wherein loss in learning in a neural network includes reconstruction loss and a sparsity penalty, and the sparsity penalty is a sum of values of the multiple components of the latent variable to which weights are assigned, respectively.
4. The processing method of claim 3 , wherein the weight assigned to each of the multiple components is related to a frequency with which feature information represented by the component is used in input images that are used for learning.
5. The processing method of claim 1 , wherein, in learning in a neural network configured with multiple learning steps, elements of only some components having high-ranking indices, among the multiple components of the latent variable, are activated in each of the multiple steps.
6. The processing method of claim 5 , wherein a number of activated components having the high-ranking indices is randomly set.
7. The processing method of claim 5 , wherein whether to activate each of the multiple components of the latent variable is determined, whereby a probability that the component affects a magnitude of the reconstruction loss is adjusted.
8. The processing method of claim 1 , wherein the selection is eliminating, from the latent variable, information that is less important for reconstruction of an image.
9. The processing method of claim 1 , wherein the processing is reducing an amount of data for the latent variable by eliminating a specific component, among the multiple components of the latent variable.
10. The processing method of claim 9 , wherein the specific component is at least one component having a lowest importance.
11. The processing method of claim 1 , wherein the processing is quantization for the latent variable.
12. The processing method of claim 1 , wherein:
the multiple components are sorted in order of importance,
the processing is neural network pruning, and
the neural network pruning is eliminating nodes having a lowest importance, among nodes of a latent layer of a neural network.
13. The processing method of claim 12 , wherein multiple neural networks are derived from the neural network using a parameter that indicates a degree of pruning in the neural network pruning.
14. A processing method, comprising:
generating a reconstructed image using a decoder neural network to which a latent variable is applied,
wherein multiple components of the latent variable are sorted in a specific order, and
wherein the latent variable is acquired through an encoder neural network that is trained in advance.
15. The processing method of claim 14 , wherein the multiple components are sorted in order of importance.
16. The processing method of claim 14 , further comprising:
generating the latent variable using latent variable generation information.
17. The processing method of claim 14 , wherein the reconstructed image is generated using feature information of only some components having high-ranking indices, among the multiple components of the latent variable.
18. The processing method of claim 17 , wherein a ranking of each of the multiple components is determined based on a component index thereof.
19. A computer-readable recording medium in which a bitstream is stored, the bitstream comprising:
latent variable generation information,
wherein:
a latent variable is generated using the latent variable generation information,
a reconstructed image is generated using a decoder neural network to which the latent variable is applied, and
multiple components of the latent variable are sorted in a specific order, and
wherein the latent variable is acquired through an encoder neural network that is trained in advance.