Granted Patent B2

US 12,548,244 · App. 18/139,929 · Granted Feb 10, 2026

Inventors: Kai Tian (Beijing, CN); Wei Chen (Beijing, CN); Xuefeng Su (Beijing, CN)

Assignee: BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO., LTD.

G06T17/00G06V10/24G06V10/44G06V10/54G06V10/761H04N5/265

Abstract

Claims (65)

1 . A method for processing an action of a virtual object performed by a computer device, the method comprising:

receiving an action instruction, the action instruction comprising: an action identifier and time-dependent information of performing an action associated with the action identifier;

determining, among a plurality of pre-generated action video frame sequences, each action video frame sequence corresponding to an action performed by the virtual object and having a corresponding action identifier, an action video frame sequence corresponding to the action identifier;

determining, from the action video frame sequence, an action state image corresponding to a preset state image of the virtual object at a target time, the target time being determined according to the time-dependent information, further including:

comparing a visual feature corresponding to the preset state image with a visual feature corresponding to a candidate action state image in the action video frame sequence to obtain a match value between the preset state image and the candidate action state image; and

choosing the candidate action state image having a maximum match value with the preset state image as the action state image corresponding to the preset state image of the virtual object at the target time;

obtaining a pair of images including an aligned preset state image and an aligned action state image by performing pose information alignment on the preset state image and the action state image that further improves a matching degree between the virtual object in the preset state image and the virtual object in the action state image;

generating a connection video frame sequence according to the aligned preset state image and the aligned action state image, the connection video frame sequence representing a transition from a preset state corresponding to the preset state image to an action state corresponding to the action state image and connecting the preset state image with the action video frame sequence; and

splicing the connection video frame sequence with the action video frame sequence, to obtain an action video.

2 . The method according to claim 1 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features.

3 . The method according to claim 1 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features and texture features and/or deep features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features and the texture features and/or the deep features.

4 . The method according to claim 1 , wherein the pose information includes position information and posture information of the virtual object in the preset state image and the action state image.

5 . The method according to claim 1 , further comprising:

extracting a part preset state image from the preset state image, and determining, on the basis of three-dimensional reconstruction, a third visual feature corresponding to the part preset state image;

extracting a part action state image from the action state image, and determining, on the basis of three-dimensional reconstruction, a fourth visual feature corresponding to the part action state image;

generating a part connection video frame sequence according to the third visual feature and the fourth visual feature; and

adding the part connection video frame sequence to the connection video frame sequence.

6 . The method according to claim 1 , wherein the time-dependent information comprises: text information corresponding to the action identifier.

7 . A computer device, comprising a processor and a memory, the memory storing a program, the program, when executed by the processor, causing the computer device to perform a method for processing an action of a virtual object including:

receiving an action instruction, the action instruction comprising: an action identifier and time-dependent information of performing an action associated with the action identifier;

determining, among a plurality of pre-generated action video frame sequences, each action video frame sequence corresponding to an action performed by the virtual object and having a corresponding action identifier, an action video frame sequence corresponding to the action identifier;

determining, from the action video frame sequence, an action state image corresponding to a preset state image of the virtual object at a target time, the target time being determined according to the time-dependent information, further including:

comparing a visual feature corresponding to the preset state image with a visual feature corresponding to a candidate action state image in the action video frame sequence to obtain a match value between the preset state image and the candidate action state image; and

choosing the candidate action state image having a maximum match value with the preset state image as the action state image corresponding to the preset state image of the virtual object at the target time;

obtaining a pair of images including an aligned preset state image and an aligned action state image by performing pose information alignment on the preset state image and the action state image that further improves a matching degree between the virtual object in the preset state image and the virtual object in the action state image;

generating a connection video frame sequence according to the aligned preset state image and the aligned action state image, the connection video frame sequence representing a transition from a preset state corresponding to the preset state image to an action state corresponding to the action state image and connecting the preset state image with the action video frame sequence; and

splicing the connection video frame sequence with the action video frame sequence, to obtain an action video.

8 . The computer device according to claim 7 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features.

9 . The computer device according to claim 7 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features and texture features and/or deep features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features and the texture features and/or the deep features.

10 . The computer device according to claim 7 , wherein the pose information includes position information and posture information of the virtual object in the preset state image and the action state image.

11 . The computer device according to claim 7 , wherein the method further comprises:

extracting a part preset state image from the preset state image, and determining, on the basis of three-dimensional reconstruction, a third visual feature corresponding to the part preset state image;

extracting a part action state image from the action state image, and determining, on the basis of three-dimensional reconstruction, a fourth visual feature corresponding to the part action state image;

generating a part connection video frame sequence according to the third visual feature and the fourth visual feature; and

adding the part connection video frame sequence to the connection video frame sequence.

12 . The computer device according to claim 7 , wherein the time-dependent information comprises: text information corresponding to the action identifier.

13 . A non-transitory computer-readable storage medium, which stores a program, the program, when executed by one or more processors, causing the computer device to perform a method for processing an action of a virtual object including:

receiving an action instruction, the action instruction comprising: an action identifier and time-dependent information of performing an action associated with the action identifier;

determining, among a plurality of pre-generated action video frame sequences, each action video frame sequence corresponding to an action performed by the virtual object and having a corresponding action identifier, an action video frame sequence corresponding to the action identifier;

determining, from the action video frame sequence, an action state image corresponding to a preset state image of the virtual object at a target time, the target time being determined according to the time-dependent information, further including:

comparing a visual feature corresponding to the preset state image with a visual feature corresponding to a candidate action state image in the action video frame sequence to obtain a match value between the preset state image and the candidate action state image; and

choosing the candidate action state image having a maximum match value with the preset state image as the action state image corresponding to the preset state image of the virtual object at the target time;

obtaining a pair of images including an aligned preset state image and an aligned action state image by performing pose information alignment on the preset state image and the action state image that further improves a matching degree between the virtual object in the preset state image and the virtual object in the action state image;

generating a connection video frame sequence according to the aligned preset state image and the aligned action state image, the connection video frame sequence representing a transition from a preset state corresponding to the preset state image to an action state corresponding to the action state image and connecting the preset state image with the action video frame sequence; and

splicing the connection video frame sequence with the action video frame sequence, to obtain an action video.

14 . The non-transitory computer-readable storage medium according to claim 13 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features.

15 . The non-transitory computer-readable storage medium according to claim 13 , wherein the generating a connection video frame sequence according to the action state image comprises:

determining optical flow features and texture features and/or deep features separately corresponding to the action state image; and

generating the connection video frame sequence according to the optical flow features and the texture features and/or the deep features.

16 . The non-transitory computer-readable storage medium according to claim 13 , wherein the pose information includes position information and posture information of the virtual object in the preset state image and the action state image.

17 . The non-transitory computer-readable storage medium according to claim 13 , wherein the method further comprises:

extracting a part preset state image from the preset state image, and determining, on the basis of three-dimensional reconstruction, a third visual feature corresponding to the part preset state image;

extracting a part action state image from the action state image, and determining, on the basis of three-dimensional reconstruction, a fourth visual feature corresponding to the part action state image;

generating a part connection video frame sequence according to the third visual feature and the fourth visual feature; and

adding the part connection video frame sequence to the connection video frame sequence.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (41)