Image processing method, cloud server, VR terminal and storage medium

1 . An image processing method, applied to a cloud server in communication connection with a Virtual Reality (VR) terminal device, wherein the cloud server is equipped with a first Field of View (FOV) rendering camera and a second FOV rendering camera, the method comprising:

acquiring a first rendered frame and a second rendered frame, wherein the first rendered frame is from the first FOV rendering camera, and the second rendered frame is from the second FOV rendering camera;

acquiring difference data from the second rendered frame, wherein the difference data is from an area in the second rendered frame which is not repeated in the first rendered frame; and

sending the first rendered frame and the difference data to the VR terminal device such that the VR terminal device restores the second rendered frame according to the first rendered frame and the difference data and obtains a display image according to the first rendered frame and the second rendered frame;

wherein before acquiring the first rendered frame and the second rendered frame, the method further comprises:

acquiring device information and pose matrices sent by the VR terminal device, wherein the device information comprises pupillary distance information and FOV angle information, and the pose matrices comprise a first pose matrix and a second pose matrix;

determining a distance between the first FOV rendering camera and the second FOV rendering camera according to the pupillary distance information; and

configuring the first FOV rendering camera according to the first pose matrix and the FOV angle information, and configuring the second FOV rendering camera according to the second pose matrix and the FOV angle information.

2 . The method of claim 1 , wherein acquiring the difference data from the second rendered frame comprises:

determining a repetitive area between the first rendered frame and the second rendered frame according to the FOV angle information and the pupillary distance information;

determining an area in the second rendered frame other than the repetitive area as a difference area; and

acquiring the difference data from the second rendered frame according to the difference area.

3 . The method of claim 2 , wherein sending the first rendered frame and the difference data to the VR terminal device such that the VR terminal device restores the second rendered frame according to the first rendered frame and the difference data comprises:

adding identification information of the repetitive area to the first rendered frame; and

sending the first rendered frame carrying the identification information and the difference data to the VR terminal device, such that the VR terminal device determines the repetitive area according to the identification information, acquires first repetitive frame data from the first rendered frame according to the repetitive area, and restores the second rendered frame according to the first repetitive frame data and the difference data.

4 . The method of claim 1 , wherein after acquiring the first rendered frame and the second rendered frame, the method further comprises:

acquiring pose update information sent by the VR terminal device; and

updating the pose matrices according to the pose update information.

5 . The method of claim 1 , wherein sending the first rendered frame and the difference data to the VR terminal device comprises:

acquiring full frame data of the first rendered frame, and encoding the full frame data to obtain a first encoding result;

encoding the difference data to obtain a second encoding result; and

sending the first encoding result and the second encoding result to the VR terminal device in parallel, such that the VR terminal device decodes the first encoding result to obtain the first rendered frame and decodes the second encoding result to obtain the difference data.

6 . An image processing method, applied to a Virtual Reality (VR) terminal device in communication connection with a cloud server, wherein the cloud server is equipped with a first Field of View (FOV) rendering camera and a second FOV rendering camera, the method comprising:

acquiring a first rendered frame and difference data sent by the cloud server, wherein the first rendered frame is from the first FOV rendering camera, the difference data is from an area in the second rendered frame which is not repeated in the first rendered frame, and the second rendered frame is from the second FOV rendering camera;

restoring the second rendered frame according to the first rendered frame and the difference data; and

obtaining a display image according to the first rendered frame and the second rendered frame;

wherein before acquiring the first rendered frame and the difference data sent by the cloud server, the method further comprises:

acquiring device information and pose matrices, wherein the device information comprises pupillary distance information and FOV angle information, and the pose matrices comprise a first pose matrix and a second pose matrix; and

sending the device information and the pose matrices to the cloud server, such that the cloud server determines a distance between the first FOV rendering camera and the second FOV rendering camera according to the pupillary distance information, configures the first FOV rendering camera according to the first pose matrix and the FOV angle information, and configures the second FOV rendering camera according to the second pose matrix and the FOV angle information.

7 . The method of claim 6 , wherein the first rendered frame carries identification information of a repetitive area,

the repetitive area is an area in the first rendered frame which is repeated in the second rendered frame,

the repetitive area is determined by the cloud server according to the FOV angle information and the pupillary distance information, and

restoring the second rendered frame according to the first rendered frame and the difference data comprises:

determining the repetitive area according to the identification information, and acquiring first repetitive frame data from the first rendered frame according to the repetitive area; and

restoring the second rendered frame according to the first repetitive frame data and the difference data.

8 . The method of claim 7 , wherein the first repetitive frame data comprises first screen coordinates of each pixel in a screen space coordinate system and further comprises a pixel Red Green Blue (RGB) value corresponding to the first screen coordinates, and

restoring the second rendered frame according to the first repetitive frame data and the difference data comprises:

performing matrix transformation for the first screen coordinates to obtain second screen coordinates;

assigning the pixel RGB value to the corresponding second screen coordinates according to a mapping relationship between the first screen coordinates and the second screen coordinates to obtain second repetitive frame data; and

combining the second repetitive frame data with the difference data to obtain the second rendered frame.

9 . The method of claim 8 , wherein performing the matrix transformation for the first screen coordinates to obtain the second screen coordinates comprises:

converting the first screen coordinates of each pixel to first device coordinates in a device coordinate system by screen mapping;

acquiring a preset projection matrix, and converting the first device coordinates of each pixel to world coordinates according to the first pose matrix and the projection matrix;

converting the world coordinates of each pixel to second device coordinates according to the projection matrix and the second pose matrix; and

converting the second device coordinates of each pixel to the second screen coordinates by screen mapping.

10 . The method of claim 9 , wherein the first repetitive frame data further comprises depth information of each pixel, and

converting the first screen coordinates of each pixel to the first device coordinates in the device coordinate system by screen mapping comprises:

acquiring screen width information, screen height information, and a default weight which are pre-configured;

determining an abscissa of the first device coordinates according to an abscissa of the first screen coordinates and the screen width information;

determining an ordinate of the first device coordinates according to an ordinate of the first screen coordinates and the screen height information; and

determining the depth information as a depth coordinate of the first device coordinates, and determining the default weight as a weight of the first device coordinates.

11 . The method of claim 6 , wherein after acquiring the first rendered frame and the difference data sent by the cloud server, the method further comprises:

acquiring pose update information of the pose matrices in response to detection of a change of the pose matrices; and

sending the pose update information to the cloud server, such that the cloud server updates the pose matrices according to the pose update information.

12 . The method of claim 6 , wherein acquiring the first rendered frame and the difference data sent by the cloud server comprises:

acquiring a first encoding result and a second encoding result sent by the cloud server in parallel, wherein the first encoding result is obtained by the cloud server by encoding full frame data of the first rendered frame, and the second encoding result is obtained by the cloud server by encoding the difference data;

decoding the first encoding result to obtain the first rendered frame; and

decoding the second encoding result to obtain the difference data.

13 . A cloud server, comprising: a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, causes the processor to perform the image processing method of claim 1 .

14 . A Virtual Reality (VR) terminal device, comprising:

a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, causes the processor to perform the image processing method of claim 6 .

15 . A non-transitory computer-readable storage medium, storing a computer-executable instruction which, when executed by a processor, causes the processor to perform an image processing method, applied to a cloud server in communication connection with a Virtual Reality (VR) terminal device, wherein the cloud server is equipped with a first Field of View (FOV) rendering camera and a second FOV rendering camera, the method comprising:

acquiring a first rendered frame and a second rendered frame, wherein the first rendered frame is from the first FOV rendering camera, and the second rendered frame is from the second FOV rendering camera;

acquiring difference data from the second rendered frame, wherein the difference data is from an area in the second rendered frame which is not repeated in the first rendered frame; and

sending the first rendered frame and the difference data to the VR terminal device such that the VR terminal device restores the second rendered frame according to the first rendered frame and the difference data and obtains a display image according to the first rendered frame and the second rendered frame;

wherein before acquiring the first rendered frame and the second rendered frame, the method further comprises:

acquiring device information and pose matrices sent by the VR terminal device, wherein the device information comprises pupillary distance information and FOV angle information, and the pose matrices comprise a first pose matrix and a second pose matrix;

determining a distance between the first FOV rendering camera and the second FOV rendering camera according to the pupillary distance information; and

configuring the first FOV rendering camera according to the first pose matrix and the FOV angle information, and configuring the second FOV rendering camera according to the second pose matrix and the FOV angle information.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein acquiring the difference data from the second rendered frame comprises:

determining a repetitive area between the first rendered frame and the second rendered frame according to the FOV angle information and the pupillary distance information;

determining an area in the second rendered frame other than the repetitive area as a difference area; and

acquiring the difference data from the second rendered frame according to the difference area.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein sending the first rendered frame and the difference data to the VR terminal device such that the VR terminal device restores the second rendered frame according to the first rendered frame and the difference data comprises:

adding identification information of the repetitive area to the first rendered frame; and

sending the first rendered frame carrying the identification information and the difference data to the VR terminal device, such that the VR terminal device determines the repetitive area according to the identification information, acquires first repetitive frame data from the first rendered frame according to the repetitive area, and restores the second rendered frame according to the first repetitive frame data and the difference data.