Method and apparatus for rendering interaction picture, device, storage medium, and program product

1 . A method for electronically rendering an interaction picture comprising a virtual character, the method comprising:

obtaining, by an image rendering device from an image capture device comprising a real camera, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters, the obtaining the interaction data of the at least two real characters including:

obtaining, by the image rendering device, original data that is recorded by the real camera of the at least two real characters performing interaction with each other in the target scene, wherein the real camera is mounted on a mechanical arm, and the mechanical arm is connected to the real camera through an end actuator of the mechanical arm;

determining, by the image rendering device, a first coordinate conversion relationship between the real camera and the mechanical arm, and a second coordinate conversion relationship between the mechanical arm and a virtual camera; and

mapping, by the image rendering device, the original data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain mapped data of the original data in the virtual camera, and determining the mapped data as the interaction data,

the motion-capture data being configured for determining a movement of the virtual character, the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character; and

performing, by the image rendering device, picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene, wherein the image rendering device includes the virtual camera and the virtual camera is configured for performing the picture rendering,

the second character being a character other than the first character in the at least two real characters.

2 . The method according to claim 1 , wherein the determining, by the image rendering device, a first coordinate conversion relationship between the real camera and the mechanical arm comprises:

obtaining a third coordinate conversion relationship between the real camera and the end actuator of the mechanical arm, and a fourth coordinate conversion relationship between the end actuator of the mechanical arm and a base coordinate system of the mechanical arm; and

determining the first coordinate conversion relationship between the real camera and the mechanical arm based on the third coordinate conversion relationship and the fourth coordinate conversion relationship.

3 . The method according to claim 1 , wherein the obtaining, by the image rendering device, original data that is recorded by a real camera of the at least two real characters performing interaction with each other in the target scene comprises:

obtaining first scene data of the target scene, wherein the first scene data is scene data that is recorded by the real camera during movement of the mechanical arm along a target trajectory and that does not comprise the at least two real characters;

obtaining second scene data of the target scene, wherein the second scene data is scene data that is recorded by the real camera during the movement of the mechanical arm along the target trajectory when the at least two real characters perform interaction with each other in the target scene and that comprises the at least two real characters; and

determining the first scene data and the second scene data as the original data.

4 . The method according to claim 3 , wherein the obtaining, by the image rendering device, first scene data of the target scene comprises:

receiving a movement instruction for the mechanical arm, wherein the movement instruction is configured for instructing the mechanical arm to move along the target trajectory;

controlling, in response to the movement instruction for the mechanical arm, the mechanical arm to move along the target trajectory; and

controlling the real camera to record the first scene data during the movement of the mechanical arm.

5 . The method according to claim 3 , wherein the mapping the original data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain mapped data of the original data in the virtual camera comprises:

mapping the first scene data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain first mapped data of the first scene data in the virtual camera;

mapping the second scene data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain second mapped data of the second scene data in the virtual camera; and

determining the first mapped data and the second mapped data as the mapped data of the original data in the virtual camera.

6 . The method according to claim 5 , wherein the performing picture rendering based on the interaction data and the motion-capture data, to obtain a picture in which the virtual character performs interaction with a second character in the target scene comprises:

performing picture rendering based on the first mapped data, to obtain a first rendered picture, and performing picture rendering based on the second mapped data, to obtain a second rendered picture;

creating a space mask corresponding to the first character, and performing picture rendering on the space mask, to obtain a third rendered picture;

generating, based on the motion-capture data, the virtual character corresponding to the first character at a position associated with the first character, and performing picture rendering on the virtual character, to obtain a fourth rendered picture comprising the virtual character; and

overlaying the first rendered picture, the second rendered picture, the third rendered picture, and the fourth rendered picture, to obtain the picture in which the virtual character performs interaction with the second character in the target scene.

7 . The method according to claim 4 , wherein the determining the first scene data and the second scene data as the original data comprises:

obtaining a receiving moment of the movement instruction and a target duration required for transmitting the second scene data from the real camera to the virtual camera; and

starting a timer at the receiving moment, playing the first scene data when the timer reaches the target duration, and determining the played first scene data and the recorded second scene data as the original data.

8 . The method according to claim 4 , wherein before the performing picture rendering based on the interaction data and the motion-capture data, the method further comprises:

mapping the first scene data and the second scene data to a timeline, and displaying, on the timeline, at least one first key frame corresponding to the first scene data and at least one second key frame corresponding to the second scene data;

screening, out of the at least one second key frame, a second target key frame corresponding to a first target key frame in the at least one first key frame; and

obtaining a time difference between the first target key frame and the second target key frame, and adjusting, when the time difference is greater than a first time threshold, a position of at least one of the first scene data and the second scene data on the timeline until the time difference is less than a second time threshold.

9 . The method according to claim 1 , wherein the mapping the original data to obtain mapped data of the original data in the virtual camera comprises:

obtaining a color mapping relationship between the real camera and the virtual camera; and

performing color-space conversion on the original data based on the color mapping relationship, to obtain the mapped data of the original data in the virtual camera; and

the performing picture rendering based on the interaction data and the motion-capture data, to obtain a picture in which the virtual character performs interaction with a second character in the target scene comprises:

performing the picture rendering based on the interaction data and the motion-capture data, to obtain an initial rendered picture in which the virtual character performs interaction with the second character in the target scene; and

performing color-space inverse conversion on the initial rendered picture based on the color mapping relationship, to obtain the picture in which the virtual character performs interaction with the second character in the target scene.

10 . The method according to claim 1 , wherein the obtaining motion-capture data of a first character in the at least two real characters comprises:

obtaining, when the first character in the at least two real characters wears a motion capture device, position change data that is of a marked point on the motion capture device and that is captured by a real camera, and using the position change data as motion data of the first character, wherein the marked point corresponds to a skeletal key point of the first character;

obtaining, when the first character is mounted with an expression capture device, expression data that is of the first character and that is captured by the expression capture device;

obtaining, when the first character is mounted with a voice collection device, voice data that is of the first character and that is collected by the voice collection device; and

determining at least one of the motion data, the expression data, and the voice data that are of the first character as the motion-capture data.

11 . The method according to claim 1 , wherein the performing picture rendering based on the interaction data and the motion-capture data, to obtain a picture in which the virtual character performs interaction with a second character in the target scene comprises:

performing the picture rendering based on the interaction data, to obtain a first interaction picture in which the at least two real characters perform interaction with each other in the target scene;

creating a space mask corresponding to the first character, and masking the first character in the first interaction picture based on the space mask, to obtain a second interaction picture in which the second character performs interaction with the space mask; and

generating the virtual character corresponding to the first character at a position of the space mask, and rendering the virtual character into the second interaction picture based on the motion-capture data, to obtain the picture in which the virtual character performs interaction with the second character in the target scene.

12 . An apparatus for rendering an interaction picture, the apparatus comprising:

a processor; and

memory storing computer readable instructions that, when executed by the processor, cause the apparatus to:

obtain, from an image capture device comprising a real camera, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters, the obtaining the interaction data of the at least two real characters includes:

obtaining original data that is recorded by the real camera of the at least two real characters performing interaction with each other in the target scene, wherein the real camera is mounted on a mechanical arm, and the mechanical arm is connected to the real camera through an end actuator of the mechanical arm;

determining a first coordinate conversion relationship between the real camera and the mechanical arm, and a second coordinate conversion relationship between the mechanical arm and a virtual camera; and

mapping the original data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain mapped data of the original data in the virtual camera, and determining the mapped data as the interaction data,

the motion-capture data being configured for determining a movement of a virtual character, the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character; and

perform, using the virtual camera, picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene,

the second character being a character other than the first character in the at least two real characters.

13 . The apparatus according to claim 12 , wherein the determining a first coordinate conversion relationship between the real camera and the mechanical arm comprises:

obtaining a third coordinate conversion relationship between the real camera and the end actuator of the mechanical arm, and a fourth coordinate conversion relationship between the end actuator of the mechanical arm and a base coordinate system of the mechanical arm; and

determining the first coordinate conversion relationship between the real camera and the mechanical arm based on the third coordinate conversion relationship and the fourth coordinate conversion relationship.

14 . The apparatus according to claim 12 , wherein the obtaining original data that is recorded by a real camera of the at least two real characters performing interaction with each other in the target scene comprises:

obtaining first scene data of the target scene, wherein the first scene data is scene data that is recorded by the real camera during movement of the mechanical arm along a target trajectory and that does not comprise the at least two real characters;

obtaining second scene data of the target scene, wherein the second scene data is scene data that is recorded by the real camera during the movement of the mechanical arm along the target trajectory when the at least two real characters perform interaction with each other in the target scene and that comprises the at least two real characters; and

determining the first scene data and the second scene data as the original data.

15 . The apparatus according to claim 14 , wherein the obtaining first scene data of the target scene comprises:

receiving a movement instruction for the mechanical arm, wherein the movement instruction is configured for instructing the mechanical arm to move along the target trajectory;

controlling, in response to the movement instruction for the mechanical arm, the mechanical arm to move along the target trajectory; and

controlling the real camera to record the first scene data during the movement of the mechanical arm.

16 . A non-transitory computer-readable medium storing computer-readable instructions that, when executed, cause an apparatus to:

obtain, from an image capture device comprising a real camera, interaction data of at least two real characters performing interaction with each other in a target scene and motion-capture data of a first character in the at least two real characters, the obtaining the interaction data of the at least two real characters includes:

obtaining original data that is recorded by the real camera of the at least two real characters performing interaction with each other in the target scene, wherein the real camera is mounted on a mechanical arm, and the mechanical arm is connected to the real camera through an end actuator of the mechanical arm;

determining a first coordinate conversion relationship between the real camera and the mechanical arm, and a second coordinate conversion relationship between the mechanical arm and a virtual camera; and

mapping the original data based on the first coordinate conversion relationship and the second coordinate conversion relationship, to obtain mapped data of the original data in the virtual camera, and determining the mapped data as the interaction data,

the motion-capture data being configured for determining a movement of a virtual character, the virtual character corresponding to the first character, to perform an interactive action consistent with one or more movements of the first character; and

perform, using the virtual camera, picture rendering based on the interaction data and the motion-capture data, to generate an image in which the virtual character performs interaction with a second character in the target scene,

the second character being a character other than the first character in the at least two real characters.

17 . The non-transitory computer-readable medium according to claim 16 , wherein the determining a first coordinate conversion relationship between the real camera and the mechanical arm comprises:

obtaining a third coordinate conversion relationship between the real camera and the end actuator of the mechanical arm, and a fourth coordinate conversion relationship between the end actuator of the mechanical arm and a base coordinate system of the mechanical arm; and

determining the first coordinate conversion relationship between the real camera and the mechanical arm based on the third coordinate conversion relationship and the fourth coordinate conversion relationship.