Granted Patent B2

US 12,561,882 · App. 18/552,777 · Granted Feb 24, 2026

Inventors: Jinyuan Wu (Beijing, CN); Yi Guo (Beijing, CN)

Assignee: Beijing Zitiao Network Technology Co., Ltd.

G06T15/00G06T7/50G06V10/761

Abstract

Claims (85)

1 . An image rendering method, comprising:

obtaining a first depth and a second depth corresponding to each pixel point of pixel points in an image to be rendered, wherein the first depth corresponding to the pixel point is a depth of a real scene corresponding to the pixel point in a Normalized Device Coordinate system, and the second depth corresponding to the pixel point is a depth of a virtual scene corresponding to the pixel point in the Normalized Device Coordinate system;

obtaining a depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point, wherein the depth difference corresponding to the pixel point is a difference between a depth of a real scene corresponding to the pixel point and a depth of a virtual scene corresponding to the pixel point in a camera coordinate system;

determining an opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point, wherein the depth difference corresponding to the each pixel point is positively correlated with the opacity corresponding to the each pixel point; and

rendering a layer corresponding to the virtual scene over a layer corresponding to the real scene to generate the image to be rendered based on the opacity corresponding to the each pixel point,

wherein the obtaining the depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point comprises:

converting the first depth corresponding to the each pixel point to a third depth corresponding to the each pixel point, wherein the third depth corresponding to the pixel point is the depth of the real scene corresponding to the pixel point in the camera coordinate system;

converting the second depth corresponding to the each pixel point to a fourth depth corresponding to the each pixel point, wherein the fourth depth corresponding to the pixel point is the depth of the virtual scene corresponding to the pixel point in the camera coordinate system; and

obtaining the depth difference corresponding to the each pixel point by subtracting the third depth from the fourth depth corresponding to the each pixel point.

2 . The image rendering method according to claim 1 , wherein the converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point comprises:

converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the first depth corresponding to the each pixel point is inversely proportional to the third depth corresponding to the each pixel point.

3 . The image rendering method according to claim 2 , wherein the converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum comprises:

obtaining a first calculation value corresponding to the each pixel point, which is twice a product of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a second calculation value corresponding to the each pixel point, which is a difference between the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a third calculation value corresponding to the each pixel point, which is a product of the second calculation value and the first depth corresponding to the pixel point;

obtaining a fourth calculation value corresponding to the each pixel point, which is a sum of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a fifth calculation value corresponding to the each pixel point, which is a difference between the third calculation value and the fourth calculation value; and

obtaining a ratio of the first calculation value to the fifth calculation value corresponding to the each pixel point as the third depth corresponding to the each pixel point.

4 . The image rendering method according to claim 1 , wherein the converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point comprises:

converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the first depth corresponding to the each pixel point is negatively correlated with the third depth corresponding to the each pixel point.

5 . The image rendering method according to claim 4 , wherein the converting the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum comprises:

obtaining a first calculation value corresponding to the each pixel point, which is twice a product of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a second calculation value corresponding to the each pixel point, which is a difference between the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a third calculation value corresponding to the each pixel point, which is a product of the second calculation value and the first depth corresponding to the pixel point;

obtaining a fourth calculation value corresponding to the each pixel point, which is a sum of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a fifth calculation value corresponding to the each pixel point, which is a difference between the third calculation value and the fourth calculation value; and

obtaining a ratio of the first calculation value to the fifth calculation value corresponding to the each pixel point as the third depth corresponding to the each pixel point.

6 . The image rendering method according to claim 1 , wherein the converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point comprises:

converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the second depth corresponding to the each pixel point is inversely proportional to the fourth depth corresponding to the each pixel point.

7 . The image rendering method according to claim 6 , wherein the converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point based on the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum comprises:

obtaining a sixth calculation value corresponding to the each pixel point, which is twice a product of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a seventh calculation value corresponding to the each pixel point, which is a difference between the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining an eighth calculation value corresponding to the each pixel point, which is a product of the seventh calculation value and the second depth corresponding to the each pixel point;

obtaining a ninth calculation value corresponding to the each pixel point, which is a sum of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a tenth calculation value corresponding to the each pixel point, which is a difference between the eighth calculation value and the ninth calculation value; and

obtaining a ratio of the sixth calculation value to the tenth calculation value corresponding to the each pixel point as the fourth depth corresponding to the each pixel point.

8 . The image rendering method according to claim 1 , wherein the converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point comprises:

converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the second depth corresponding to the each pixel point is negatively correlated with the fourth depth corresponding to the each pixel point.

9 . The image rendering method according to claim 8 , wherein the converting the second depth corresponding to the each pixel point to the fourth depth corresponding to the each pixel point based on the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum comprises:

obtaining a sixth calculation value corresponding to the each pixel point, which is twice a product of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a seventh calculation value corresponding to the each pixel point, which is a difference between the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining an eighth calculation value corresponding to the each pixel point, which is a product of the seventh calculation value and the second depth corresponding to the each pixel point;

obtaining a ninth calculation value corresponding to the each pixel point, which is a sum of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtaining a tenth calculation value corresponding to the each pixel point, which is a difference between the eighth calculation value and the ninth calculation value; and

obtaining a ratio of the sixth calculation value to the tenth calculation value corresponding to the each pixel point as the fourth depth corresponding to the each pixel point.

10 . The image rendering method according to claim 1 , wherein the determining the opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point comprises:

determining that an opacity corresponding to a first pixel point is 0 in a case where a depth difference corresponding to the first pixel point is less than or equal to 0.

11 . The image rendering method according to claim 1 , wherein the determining the opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point comprises:

determining that an opacity corresponding to a second pixel point is a ratio of a depth difference corresponding to the second pixel point to a preset threshold in a case where a depth difference corresponding to the second pixel point is greater than 0 and less than the preset threshold.

12 . The image rendering method according to claim 1 , wherein the determining the opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point comprises:

determining that an opacity corresponding to a third pixel point is 100% in a case where a depth difference corresponding to the third pixel point is greater than or equal to a preset threshold.

13 . The image rendering method according to claim 1 , further comprising:

collecting images of a preset scene to obtain image data corresponding to the preset scene before the obtaining the first depth and the second depth corresponding to the each pixel point of the pixel points in the image to be rendered; and

performing three dimensional reconstruction of the preset scene according to the image data to obtain the real scene.

14 . An electronic device, comprising:

a memory for storing a computer program; and

a processor that, when executing the computer program, causes the electronic device to implement an image rendering method comprising:

obtaining a first depth and a second depth corresponding to each pixel point of pixel points in an image to be rendered, wherein the first depth corresponding to the pixel point is a depth of a real scene corresponding to the pixel point in a Normalized Device Coordinate system, and the second depth corresponding to the pixel point is a depth of a virtual scene corresponding to the pixel point in the Normalized Device Coordinate system;

obtaining a depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point, wherein the depth difference corresponding to the pixel point is a difference between a depth of a real scene corresponding to the pixel point and a depth of a virtual scene corresponding to the pixel point in a camera coordinate system;

determining an opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point, wherein the depth difference corresponding to the each pixel point is positively correlated with the opacity corresponding to the each pixel point; and

rendering a layer corresponding to the virtual scene over a layer corresponding to the real scene to generate the image to be rendered based on the opacity corresponding to the each pixel point,

the obtaining the depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point comprises

converting the first depth corresponding to the each pixel point to a third depth corresponding to the each pixel point, wherein the third depth corresponding to the pixel point is the depth of the real scene corresponding to the pixel point in the camera coordinate system;

converting the second depth corresponding to the each pixel point to a fourth depth corresponding to the each pixel point, wherein the fourth depth corresponding to the pixel point is the depth of the virtual scene corresponding to the pixel point in the camera coordinate system; and

obtaining the depth difference corresponding to the each pixel point by subtracting the third depth from the fourth depth corresponding to the each pixel point.

15 . The electronic device according to claim 14 , wherein the processor is configured to:

convert the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the first depth corresponding to the each pixel point is inversely proportional to the third depth corresponding to the each pixel point.

16 . The electronic device according to claim 15 , wherein the processor is configured to:

obtain a first calculation value corresponding to the each pixel point, which is twice a product of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtain a second calculation value corresponding to the each pixel point, which is a difference between the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtain a third calculation value corresponding to the each pixel point, which is a product of the second calculation value and the first depth corresponding to the pixel point;

obtain a fourth calculation value corresponding to the each pixel point, which is a sum of the distance from the virtual camera to the far plane of the view frustum and the distance from the virtual camera to the near plane of the view frustum;

obtain a fifth calculation value corresponding to the each pixel point, which is a difference between the third calculation value and the fourth calculation value; and

obtain a ratio of the first calculation value to the fifth calculation value corresponding to the each pixel point as the third depth corresponding to the each pixel point.

17 . The electronic device according to claim 14 , wherein the processor is configured to:

convert the first depth corresponding to the each pixel point to the third depth corresponding to the each pixel point based on a distance from a virtual camera to a far plane of a view frustum and a distance from the virtual camera to a near plane of the view frustum, wherein the first depth corresponding to the each pixel point is negatively correlated with the third depth corresponding to the each pixel point.

18 . A non-transitory computer-readable storage medium stored thereon a computer program that, when executed by a computing device, causes the computing device to implement an image rendering method comprising:

obtaining a first depth and a second depth corresponding to each pixel point of pixel points in an image to be rendered, wherein the first depth corresponding to the pixel point is a depth of a real scene corresponding to the pixel point in a Normalized Device Coordinate system, and the second depth corresponding to the pixel point is a depth of a virtual scene corresponding to the pixel point in the Normalized Device Coordinate system;

obtaining a depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point, wherein the depth difference corresponding to the pixel point is a difference between a depth of a real scene corresponding to the pixel point and a depth of a virtual scene corresponding to the pixel point in a camera coordinate system;

determining an opacity corresponding to the each pixel point according to the depth difference corresponding to the each pixel point, wherein the depth difference corresponding to the each pixel point is positively correlated with the opacity corresponding to the each pixel point; and

rendering a layer corresponding to the virtual scene over a layer corresponding to the real scene to generate the image to be rendered based on the opacity corresponding to the each pixel point;

the obtaining the depth difference corresponding to the each pixel point based on the first depth and the second depth corresponding to the each pixel point comprises:

converting the first depth corresponding to the each pixel point to a third depth corresponding to the each pixel point, wherein the third depth corresponding to the pixel point is the depth of the real scene corresponding to the pixel point in the camera coordinate system;

converting the second depth corresponding to the each pixel point to a fourth depth corresponding to the each pixel point, wherein the fourth depth corresponding to the pixel point is the depth of the virtual scene corresponding to the pixel point in the camera coordinate system; and

obtaining the depth difference corresponding to the each pixel point by subtracting the third depth from the fourth depth corresponding to the each pixel point.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (36)