Method and device for processing picture
View Patent ↗Embodiments of the present disclosure provide a method and a device for processing a picture and an electronic device. The method includes scaling a raw picture to obtain N scaled pictures with different resolutions, where N is a natural number greater than or equal to 2; mapping the N scaled pictures respectively into N content picture each with a same predetermined style feature through a neural algorithm network; scaling the N content pictures to have the same resolution; and encoding and combining the N content images of the same resolution into a dynamic video.
1. A method for processing a picture, applied for an electronic device including a processor, the method comprising:
scaling a raw picture to obtain N scaled pictures having different resolutions, where N is a natural number equal to or greater than 2;
mapping the N scaled pictures respectively into N content pictures each having a same predetermined style feature by using a neural algorithm network;
scaling the N content pictures to a same resolution; and
encoding and combining the N content pictures having the same resolution into a dynamic video;
wherein the neural algorithm network comprises an image conversion network and a loss network, and mapping the N scaled pictures respectively into the N content pictures each having the same predetermined style feature by using the neural algorithm network comprises:
calculating a reconstruction loss via the loss network, wherein the reconstruction loss is a loss between a raw image and an output image of the image conversion network with the raw image as an input;
determining a mapping weight parameter of the neural algorithm network based on the reconstruction loss; and
mapping the N scaled pictures respectively into the N content pictures each having the same predetermined style feature based on the mapping weight parameter.
2. The method of claim 1 , wherein N is a product of a number of frames contained in the dynamic video and a playing time length of the dynamic video.
3. The method of claim 1 , wherein mapping the N scaled pictures respectively into the N content pictures each having the same predetermined style feature by using the neural algorithm network comprises:
adding a noisy background picture into each of the N scaled pictures; and
mapping the N scaled pictures each added with the noisy background picture respectively into the N content pictures each having the same predetermined style feature.
4. The method of claim 1 , wherein the loss network is configured to define a feature reconstruction loss function and a style reconstruction loss function, and calculating the reconstruction loss via the loss network comprises:
calculating a first high-order semantic loss between the raw image and the output image of the image conversion network with the raw image as an input, via the feature reconstruction loss; and
calculating a second high-order semantic loss between a style image and an output image of the image conversion network with the style image as an input, via the style reconstruction loss function.
5. The method of claim 4 , wherein determining the mapping weight parameter of the neural algorithm network based on the reconstruction loss comprises:
determining the mapping weight parameter based on the first high-order semantic loss and the second high-order semantic loss.
6. The method of claim 1 , wherein scaling the raw picture to obtain the N scaled pictures having different resolutions comprises:
scaling the raw picture to obtain the N scaled pictures having different resolutions by modifying a picture property of the raw picture, wherein the picture property comprises at least one of picture size, picture format, and picture compression mode.
7. An electronic device, comprising a processor, a memory, a communication interface and a bus,
wherein the processor, the memory and the communication interface are connected to each other and communicate with each other via the bus;
the memory is a non-transitory computer readable medium configured to store executable program codes;
when reading the executable program codes stored in the memory to run a program corresponding to the executable program codes, the processor is configured to:
scale a raw picture to obtain N scaled pictures having different resolutions, where N is a natural number equal to or greater than 2;
map the N scaled pictures respectively into N content pictures each having a same predetermined style feature by using a neural algorithm network;
scale the N content pictures to a same resolution; and
encode and combine the N content pictures having the same resolution into a dynamic video;
wherein the neural algorithm network comprises an image conversion network and a loss network, and the processor is further configured to:
calculate a reconstruction loss via the loss network, wherein the reconstruction loss is a loss between a raw image and an output image of the image conversion network with the raw image as an input;
determine a mapping weight parameter of the neural algorithm network based on the reconstruction loss; and
map the N scaled pictures respectively into the N content pictures each having the same predetermined style feature based on the mapping weight parameter.
8. The electronic device of claim 7 , wherein N is a product of a number of frames contained in the dynamic video and a playing time length of the dynamic video.
9. The electronic device of claim 7 , wherein the processor is further configured to:
add a noisy background picture into each of the N scaled pictures; and
map the N scaled pictures each added with the noisy background picture respectively into the N content pictures each having the same predetermined style feature.
10. The electronic device of claim 7 , wherein the processor is further configured to:
scale the raw picture to obtain the N scaled pictures having different resolutions by modifying a picture property of the raw picture, wherein the picture property comprises at least one of picture size, picture format, and picture compression mode.
11. The electronic device of claim 7 , wherein the loss network is configured to define a feature reconstruction loss function and a style reconstruction loss function, and the processor is further configured to:
calculate a first high-order semantic loss between the raw image and the output image of the image conversion network with the raw image as an input, via the feature reconstruction loss; and
calculate a second high-order semantic loss between a style image and an output image of the image conversion network with the style image as an input, via the style reconstruction loss function.
12. The electronic device of claim 11 , wherein the processor is further configured to:
determine the mapping weight parameter based on the first high-order semantic loss and the second high-order semantic loss.
13. A non-transitory computer readable storage medium, configured to store an application program, wherein the application program is configured to:
scale a raw picture to obtain N scaled pictures having different resolutions, where N is a natural number equal to or greater than 2;
map the N scaled pictures respectively into N content pictures each having a same predetermined style feature by using a neural algorithm network;
scale the N content pictures to a same resolution; and
encode and combine the N content pictures having the same resolution into a dynamic video;
wherein the neural algorithm network comprises an image conversion network and a loss network, and the application program is further configured to:
calculate a reconstruction loss via the loss network, wherein the reconstruction loss is a loss between a raw image and an output image of the image conversion network with the raw image as an input;
determine a mapping weight parameter of the neural algorithm network based on the reconstruction loss; and
map the N scaled pictures respectively into the N content pictures each having the same predetermined style feature based on the mapping weight parameter.
14. The non-transitory computer readable storage medium of claim 13 , wherein N is a product of a number of frames contained in the dynamic video and a playing time length of the dynamic video.
15. The non-transitory computer readable storage medium of claim 13 , wherein the application program is further configured to:
add a noisy background picture into each of the N scaled pictures; and
map the N scaled pictures each added with the noisy background picture respectively into the N content pictures each having the same predetermined style feature.
16. The non-transitory computer readable storage medium of claim 13 , wherein the application program is further configured to:
scale the raw picture to obtain the N scaled pictures having different resolutions by modifying a picture property of the raw picture, wherein the picture property comprises at least one of picture size, picture format, and picture compression mode.
17. The non-transitory computer readable storage medium of claim 13 , wherein the loss network is configured to define a feature reconstruction loss function and a style reconstruction loss function, and the application program is further configured to:
calculate a first high-order semantic loss between the raw image and the output image of the image conversion network with the raw image as an input, via the feature reconstruction loss; and
calculate a second high-order semantic loss between a style image and an output image of the image conversion network with the style image as an input, via the style reconstruction loss function.