Granted Patent B2

US 12,641,211 · App. 18/350,603 · Granted May 26, 2026

Inventors: Hui Huang (Shenzhen, CN); Zimu Yi (Shenzhen, CN)

Assignee: SHENZHEN UNIVERSITY

H04N13/279G06T15/06G06V10/761

Abstract

Claims (71)

1 . A method of image rendering quality prediction and path planning for large-scale scenes, comprising:

obtaining multiple surface points of a proxy model within a viewpoint range of a virtual viewpoint, wherein the proxy model is a virtual object obtained by acquiring images of a real object from multiple acquisition viewpoints;

for each of the surface points, determining a photometric error among multiple photometric information corresponding to the surface point, wherein the multiple photometric information are multiple first photometric information, and the photometric error is a first photometric error, the multiple first photometric information are photometric information of the surface point at multiple visible acquisition viewpoints respectively, and the visible acquisition viewpoints are those acquisition viewpoints where the surface point is visible among the multiple acquisition viewpoints;

determining a viewing angle similarity and a resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error;

determining multiple first rays that are emitted from the virtual viewpoint and do not intersect with the proxy model;

determining second rays respectively emitted from the multiple acquisition viewpoints, wherein the second rays are parallel to the first rays and do not intersect with the proxy model;

determining second photometric errors among multiple second photometric information of the multiple second rays, wherein the second photometric information are the photometric information of those points that are located on the second rays and intersect with a background of the proxy model;

obtaining a background rendering quality corresponding to the virtual viewpoint based on the multiple second photometric errors corresponding to the multiple first rays; and

determining a rendering quality information of the proxy model at the virtual viewpoint based on the background rendering quality, and the viewing angle similarities and the resolution similarities corresponding to the multiple surface points.

2 . The method according to claim 1 , wherein the determining the viewing angle similarity and the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error comprises:

determining angle information between the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point;

determining distance information of the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point;

determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the angle information; and

determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the distance information.

3 . The method according to claim 2 , wherein the determining the angle information between the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point comprises:

determining a virtual ray between the virtual viewpoint and the surface point;

determining an acquisition ray between the visible acquisition viewpoint and the surface point; and

determining a value of an angle between the virtual ray and the acquisition as the angle information between the virtual viewpoint and the visible acquisition viewpoint with respect to the surface point.

4 . The method according to claim 2 , wherein the determining the distance information of the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point comprises:

determining a virtual distance between the virtual viewpoint and the surface point;

determining an acquisition distance between the visible acquisition viewpoint and the surface point; and

obtaining the distance information of the virtual viewpoint and the visible acquisition viewpoint with respect to the surface point based on a ratio of a distance difference between the virtual distance and the acquisition distance to the acquisition distance.

5 . The method according to claim 2 , wherein the determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the angle information comprises:

determining a minimum angle information from the angle information corresponding to the multiple visible acquisition viewpoints; and

determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the photometric error and the minimum angle information, wherein when the minimum angle information remains unchanged, the photometric error is negatively correlated with the viewing angle similarity.

6 . The method according to claim 2 , wherein the determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the distance information comprises:

determining a minimum distance information from the distance information corresponding to the multiple visible acquisition viewpoints; and

determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the minimum distance information and the first photometric error, wherein when the minimum distance information remains unchanged, the first photometric error is negatively correlated with the resolution similarity.

7 . The method according to claim 1 , before obtaining the multiple surface points of the proxy model within the viewpoint range of the virtual viewpoint, the method further comprising:

determining a start point and an end point of an acquisition path of a virtual camera and a rendering quality threshold; and

selecting the virtual viewpoint based on multiple path points between the start point and the end point;

wherein after determining the rendering quality of the proxy model at the virtual viewpoint based on the viewing angle similarities and the resolution similarities corresponding to the multiple surface points, the method further comprises:

reselecting, if the rendering quality is less than the rendering quality threshold, a new virtual viewpoint and repeat the step of obtaining the multiple surface points on the proxy model within the viewpoint range of the virtual viewpoint and subsequent steps until a distance between a selected virtual viewpoint and the end point satisfies an ending condition.

8 . A system of image rendering quality prediction and path planning for large-scale scenes, comprising a prediction end,

wherein the prediction end is configured to: obtain multiple surface points of a proxy model within a viewpoint range of a virtual viewpoint, the proxy model being a virtual object obtained by acquiring images of a real object from multiple acquisition viewpoints; for each of the surface points, determine a photometric error among multiple photometric information corresponding to the surface point, wherein the multiple photometric information are multiple first photometric information, and the photometric error is a first photometric error, the multiple first photometric information being the photometric information of the surface point at multiple visible acquisition viewpoints respectively, and the visible acquisition viewpoints being those acquisition viewpoints where the surface is visible among the multiple acquisition viewpoints;

wherein the prediction end is further configured to determine a viewing angle similarity and a resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error;

the prediction end is further configured to: determine multiple first rays that are emitted from the virtual viewpoint and do not intersect with the proxy model; determine second rays respectively emitted from the multiple acquisition viewpoints, wherein the second rays are parallel to the first rays and do not intersect with the proxy model; determine second photometric errors among multiple second photometric information of the multiple second rays, wherein the second photometric information are the photometric information of those points that are located on the second rays and intersect with a background of the proxy model; and obtain a background rendering quality corresponding to the virtual viewpoint based on the multiple second photometric errors corresponding to the multiple first rays; and

wherein the prediction end is further configured to determine a rendering quality information of the proxy model at the virtual viewpoint based on the background rendering quality, and the viewing angle similarities and the resolution similarities corresponding to the multiple surface points.

9 . A computer device, comprising a memory and a processor, the memory storing a computer program, wherein when the computer program is executed by the processor, a method of image rendering quality prediction and path planning for large-scale scenes is implemented, wherein the method comprises:

obtaining multiple surface points of a proxy model within a viewpoint range of a virtual viewpoint, wherein the proxy model is a virtual object obtained by acquiring images of a real object from multiple acquisition viewpoints;

for each of the surface points, determining a photometric error among multiple photometric information corresponding to the surface point, wherein the multiple photometric information are multiple first photometric information, and the photometric error is a first photometric error, the multiple first photometric information are photometric information of the surface point at multiple visible acquisition viewpoints respectively, and the visible acquisition viewpoints are those acquisition viewpoints where the surface point is visible among the multiple acquisition viewpoints;

determining a viewing angle similarity and a resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the photometric error;

determining multiple first rays that are emitted from the virtual viewpoint and do not intersect with the proxy model;

determining second rays respectively emitted from the multiple acquisition viewpoints, wherein the second rays are parallel to the first rays and do not intersect with the proxy model;

determining second photometric errors among multiple second photometric information of the multiple second rays, wherein the second photometric information are the photometric information of those points that are located on the second rays and intersect with a background of the proxy model;

obtaining a background rendering quality corresponding to the virtual viewpoint based on the multiple second photometric errors corresponding to the multiple first rays; and

determining a rendering quality information of the proxy model at the virtual viewpoint based on the background rendering quality, and the viewing angle similarities and the resolution similarities corresponding to the multiple surface points.

10 . The computer device according to claim 9 , wherein the determining the viewing angle similarity and the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error comprises:

determining angle information between the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point;

determining distance information of the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point;

determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the angle information; and

determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the distance information.

11 . The computer device according to claim 10 , wherein the determining the angle information between the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point comprises:

determining a virtual ray between the virtual viewpoint and the surface point;

determining an acquisition ray between the visible acquisition viewpoint and the surface point; and

determining a value of an angle between the virtual ray and the acquisition as the angle information between the virtual viewpoint and the visible acquisition viewpoint with respect to the surface point.

12 . The computer device according to claim 10 , wherein the determining the distance information of the virtual viewpoint and each visible acquisition viewpoint with respect to the surface point comprises:

determining a virtual distance between the virtual viewpoint and the surface point;

determining an acquisition distance between the visible acquisition viewpoint and the surface point; and

obtaining the distance information of the virtual viewpoint and the visible acquisition viewpoint with respect to the surface point based on a ratio of a distance difference between the virtual distance and the acquisition distance to the acquisition distance.

13 . The computer device according to claim 10 , wherein the determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the angle information comprises:

determining a minimum angle information from the angle information corresponding to the multiple visible acquisition viewpoints; and

determining the viewing angle similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the photometric error and the minimum angle information, wherein when the minimum angle information remains unchanged, the photometric error is negatively correlated with the viewing angle similarity.

14 . The computer device according to claim 10 , wherein the determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the first photometric error and the distance information comprises:

determining a minimum distance information from the distance information corresponding to the multiple visible acquisition viewpoints; and

determining the resolution similarity between the virtual viewpoint and the visible acquisition viewpoints with respect to the surface point based on the minimum distance information and the first photometric error, wherein when the minimum distance information remains unchanged, the first photometric error is negatively correlated with the resolution similarity.

15 . The computer device according to claim 9 , before obtaining the multiple surface points of the proxy model within the viewpoint range of the virtual viewpoint, the method further comprising:

determining a start point and an end point of an acquisition path of a virtual camera and a rendering quality threshold; and

selecting the virtual viewpoint based on multiple path points between the start point and the end point;

wherein after determining the rendering quality of the proxy model at the virtual viewpoint based on the viewing angle similarities and the resolution similarities corresponding to the multiple surface points, the method further comprises:

reselecting, if the rendering quality is less than the rendering quality threshold, a new virtual viewpoint and repeat the step of obtaining the multiple surface points on the proxy model within the viewpoint range of the virtual viewpoint and subsequent steps until a distance between a selected virtual viewpoint and the end point satisfies an ending condition.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (193)