Granted Patent B2

US 12,567,129 · App. 18/256,158 · Granted Mar 3, 2026

Inventors: Wendong Chen (Shenzhen, CN); Shuai Chen (Shenzhen, CN); Meng Liu (Shenzhen, CN)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

G06T5/50G06T7/10H04N23/683G06T2207/20081G06T2207/20221

Abstract

Claims (55)

1 . A method applied to an electronic device and comprising:

obtaining a first image sequence;

processing the first image sequence based on a target migration style using k fused style migration models in order to obtain a second image sequence, wherein output images of the k fused style migration models are the second image sequence, wherein an output image of one fused style migration model is one frame of image or a plurality of consecutive frames of images in the second image sequence, wherein both the first image sequence and the second image sequence comprise n frames of images, wherein higher-layer semantic information of an i th frame of image in the first image sequence is the same as that of an i th frame of image in the second image sequence, wherein low-layer semantic information of the i th frame of image in the first image sequence and the i th frame of image in the second image sequence have different styles, wherein the target migration style indicates that a style of a 1 st frame of image to a style of an n th frame of image in the second image sequence change in M styles in a first style order, wherein the low-layer semantic information comprises a color and a texture of an image, wherein the one fused style migration model is based on weighting M single-style migration models, wherein when a style of the output image of the one fused style migration model is closer to a style of an output image of a j th single-style migration model, a weight of the j th single-style migration model is larger when the one fused style migration model is generated, wherein styles of respective output images of the M single-style migration models form the M styles, and wherein j is a positive integer less than or equal to M, wherein k is less than or equal to n, wherein n and M are integers greater than 1, and wherein i is a positive integer less than or equal to n; and

saving the second image sequence.

2 . The method of claim 1 , wherein the k fused style migration models and the M single-style migration models are neural network models and have a same neural network structure.

3 . The method of claim 2 , wherein the single-style migration model is trained, and wherein the method further comprises:

obtaining a training dataset, wherein the training dataset comprises one or more frames of style images and a plurality of frames of content images in a first video, and wherein a style of the one or more frames of style images is a style of an output image of a trained single-style migration model;

processing the plurality of frames of content images using a to-be-trained single-style migration model in order to obtain a plurality of frames of composite images;

determining a high-layer semantic information loss function based on high-layer semantic information of the plurality of frames of content images and high-layer semantic information of the plurality of frames of composite images;

determining a style loss function based on styles of the plurality of frames of content images and styles of the plurality of frames of composite images;

determining a time-domain constraint loss function based on a style of one frame of composite image in the plurality of frames of composite images and styles of a second plurality of frames of composite images adjacent to the one frame of composite image; and

training the to-be-trained single-style migration model by using a loss function in order to obtain the trained single-style migration model, wherein the loss function comprises the high-layer semantic information loss function, the style loss function, and the time-domain constraint loss function.

4 . The method of claim 1 , wherein obtaining the first image sequence comprises:

enabling a camera to collect a first video; and

extracting the n frames of images in the first image sequence from z frames of images, wherein the first video comprises the z frames of images.

5 . The method of claim 4 , wherein before obtaining the n frames of images in the first image sequence based on the first video, the method further comprises performing image stabilization processing on the first video.

6 . The method of claim 4 , wherein a frame extraction ratio of extraction is based on a playback duration of the first image sequence selected by a user, and wherein the frame extraction ratio is a ratio of the playback duration of the first image sequence to a collection duration of the first video.

7 . The method of claim 4 , wherein saving the second image sequence comprises sequentially saving the n frames of images in the second image sequence in series as a video.

8 . The method of claim 4 , wherein obtaining the first image sequence further comprises:

obtaining a first image; and

segmenting the first image in order to obtain the n frames of images in the first image sequence.

9 . The method of claim 8 , wherein saving the second image sequence comprises:

capturing one splicing area from each frame of image in the second image sequence in order to obtain n splicing areas, wherein the n splicing areas have no overlapping part;

splicing the n splicing areas to obtain a second image; and

storing the second image, wherein a resolution of the second image is the same as a resolution of the first image.

10 . The method of claim 8 , wherein all frames of images in the first image sequence have a same resolution, and wherein two adjacent frames of images in the first image sequence have an overlapping part.

11 . The method of claim 1 , wherein obtaining the first image sequence comprises:

obtaining, based on a first video selected by a user, the first video from a locally-stored video; and

extracting the n frames of images in the first image sequence from z frames of images, wherein the first video comprises the z frames of images.

12 . An electronic device, comprising:

a memory configured to store a plurality of single-style migration models and a computer program; and

one or more processors coupled to the memory and configured to execute the computer program to cause the electronic device to:

obtain a first image sequence;

process the first image sequence based on a target migration style using k fused style migration models in order to obtain a second image sequence, wherein output images of the k fused style migration models are the second image sequence, wherein an output image of one fused style migration model is one frame of image or a plurality of consecutive frames of images in the second image sequence, wherein both the first image sequence and the second image sequence comprise n frames of images, wherein higher-layer semantic information of an i th frame of image in the first image sequence is the same as that of an i th frame of image in the second image sequence, wherein low-layer semantic information of the i th frame of image in the first image sequence and the i th frame of image in the second image sequence have different styles, wherein the target migration style indicates that a style of a 1 st frame of image to a style of an n th frame of image in the second image sequence change in M styles in a first style order, wherein the low-layer semantic information comprises a color and a texture of an image, wherein the one fused style migration model is based on weighting M single-style migration models, wherein when a style of the output image of the one fused style migration model is closer to a style of an output image of a j th single-style migration model, a weight of the j th single-style migration model is larger when the one fused style migration model is generated, wherein styles of respective output images of the M single-style migration models form the M styles, and wherein j is a positive integer less than or equal to M, wherein k is less than or equal to n, wherein n and M are integers greater than 1, and wherein i is a positive integer less than or equal to n; and

save the second image sequence.

13 . The electronic device of claim 12 , wherein the k fused style migration models and the M single-style migration models are neural network models and have a same neural network structure.

14 . The electronic device of claim 13 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to:

obtain a training dataset, wherein the training dataset comprises one or more frames of style images and a plurality of frames of content images in a first video, and a style of the one or more frames of style images is a style of an output image of a trained single-style migration model;

process the plurality of frames of content images using a to-be-trained single-style migration model in order to obtain a plurality of frames of composite images;

determine a high-layer semantic information loss function based on high-layer semantic information of the plurality of frames of content images and high-layer semantic information of the plurality of frames of composite images;

determine a style loss function based on styles of the plurality of frames of content images and styles of the plurality of frames of composite images;

determine a time-domain constraint loss function based on a style of one frame of composite image in the plurality of frames of composite images and styles of a second plurality of frames of composite images adjacent to the one frame of composite image; and

train the to-be-trained single-style migration model by using a loss function in order to obtain the trained single-style migration model, wherein the loss function comprises the high-layer semantic information loss function, the style loss function, and the time-domain constraint loss function.

15 . The electronic device of claim 12 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to:

enable a camera to collect a first video; and

extract the n frames of images in the first image sequence from z frames of images, wherein the first video comprises the z frames of images.

16 . The electronic device of claim 15 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to perform image stabilization processing on the first video.

17 . The electronic device of claim 15 , wherein a frame extraction ratio of extraction is based on a playback duration of the first image sequence selected by a user, and wherein the frame extraction ratio is a ratio of the playback duration of the first image sequence to a collection duration of the first video.

18 . The electronic device of claim 15 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to save the second image sequence by sequentially saving the n frames of images in the second image sequence in series as a video.

19 . The electronic device of claim 15 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to obtain the first image sequence by:

obtaining a first image; and

segmenting the first image in order to obtain the n frames of images in the first image sequence.

20 . The electronic device of claim 12 , wherein the one or more processors are further configured to execute the computer program to cause the electronic device to obtain the first image sequence by:

obtaining, based on a first video selected by a user, the first video from a locally-stored video; and

extracting the n frames of images in the first image sequence from z frames of images, wherein the first video comprises the z frames of images.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (15)