Systems and methods for filling holes in a virtual reality model

1. A system for constructing a three-dimensional virtual reality model of a property, comprising:

a storage device configured to store point cloud data and texture information captured of an interior of the property; and

at least one processor configured to:

receive a user instruction for touring the property;

construct a structure polygon mesh of the three-dimensional virtual reality model based on the point cloud data, the structure polygon mesh including a first set of polygons;

construct a textured polygon mesh of the three-dimensional virtual reality model by mapping the texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;

select a first hole polygon from the first set of polygons that is missing from the second set of polygons;

identify a hole in the textured polygon mesh by searching from the first hole polygon for a plurality of connected hole polygons using a tree structure search;

determine an area of the hole is below a predetermined threshold;

identify boundary polygons in the second set of polygons that are contiguous to the hole;

estimate texture information of the hole polygons based on texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons;

repair the textured polygon mesh using the estimated texture information of the hole polygons; and

provide a virtual reality tour of the property in the repaired three-dimensional virtual reality model.

2. The system of claim 1 , wherein the first set of polygons, the second set of polygons, and the hole polygons are triangles.

3. The system of claim 1 , wherein to identify the hole in the textured polygon mesh, the at least one processor is configured to identify a third set of polygons from the first set of polygons that do not have corresponding textured polygons in the second set of polygons.

4. The system of claim 3 , the hole polygons in the hole correspond to the third set of polygons.

5. The system of claim 1 , wherein to estimate texture information for the hole polygons, the at least one processor is further configured to:

determine pixel values for vertices of each hole polygon; and

determine pixel values inside each hole polygon based on the pixel values of its vertices.

6. The system of claim 5 , wherein to determine the pixel value for a vertex of a hole polygon, the at least one processor is further configured to:

determine distances between the vertex and vertices of boundary polygons; and

calculate a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by the distances between the vertex of the hole polygon and vertices of the respective boundary vertices.

7. The system of claim 5 , wherein to determine pixel values inside each hole polygon, the at least one processor is further configured to perform data fusion on the pixel values of the vertices of the hole polygon based on a Gaussian process.

8. The system of claim 1 , wherein to determine the area of the hole, the at least one processor is further configured to:

determine areas of the hole polygons in the hole; and

determine the area of the hole as a sum of the areas of the hole polygons.

9. The system of claim 1 , wherein the structure polygon mesh is constructed from the point cloud data using Poisson surface reconstruction.

10. A computer-implemented method for constructing a three-dimensional virtual reality model of a property, comprising:

receiving a user instruction for touring the property;

constructing, by at least one processor, a structure polygon mesh of the three-dimensional virtual reality model based on point cloud data captured of an interior of the property, the structure polygon mesh including a first set of polygons;

constructing, by the at least one processor, a textured polygon mesh of the three-dimensional virtual reality model by mapping texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;

selecting a first hole polygon from the first set of polygons that is missing from the second set of polygons;

identifying, by the at least one processor, a hole in the textured polygon mesh by searching from the first hole polygon for a plurality of connected hole polygons using a tree structure search;

determining, by the at least one processor, an area of the hole is below a predetermined threshold;

identifying, by the at least one processor, boundary polygons in the second set of polygons that are contiguous to the hole;

estimating, by the at least one processor, texture information of the hole polygons based on the texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons;

repairing, by the at least one processor, the textured polygon mesh using the estimated texture information of the hole polygons; and

providing a virtual reality tour of the property in the repaired three-dimensional virtual reality model.

11. The computer-implemented method of claim 10 , wherein the first set of polygons, the second set of polygons, and the hole polygons are triangles.

12. The computer-implemented method of claim 10 , wherein identifying the hole in the textured polygon mesh further includes identifying a third set of polygons from the first set of polygons that do not have corresponding textured polygons in the second set of polygons.

13. The computer-implemented method of claim 12 , the hole polygons in the hole correspond to the third set of polygons.

14. The computer-implemented method of claim 10 , wherein estimating texture information for the hole polygons further include:

determining pixel values for vertices of each hole polygon; and

determining pixel values inside each hole polygon based on the pixel values of its vertices.

15. The computer-implemented method of claim 14 , wherein determining the pixel value for a vertex of a hole polygon further includes:

determining distances between the vertex and vertices of boundary polygons; and

calculating a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by the distances between the vertex of the hole polygon and vertices of the respective boundary vertices.

16. The computer-implemented method of claim 14 , wherein determining pixel values inside each hole polygon further includes performing data fusion on the pixel values of the vertices of the hole polygon based on a Gaussian process.

17. The computer-implemented method of claim 10 , wherein determining the area of the hole further includes:

determining areas of the hole polygons in the hole; and

determine the area of the hole as a sum of the areas of the hole polygons.

18. The computer-implemented method of claim 10 , wherein the structure polygon mesh is constructed from the point cloud data using Poisson surface reconstruction.

19. A non-transitory computer-readable medium having stored thereon computer instructions, when executed by at least one processor, perform a method for constructing a three-dimensional virtual reality model of a property, comprising:

receiving a user instruction for touring the property;

constructing a structure polygon mesh of the three-dimensional virtual reality model based on point cloud data captured of an interior of the property, the structure polygon mesh including a first set of polygons;

constructing a textured polygon mesh of the three-dimensional virtual reality model by mapping the texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;

selecting a first hole polygon from the first set of polygons that is missing from the second set of polygons;

identifying a hole in the textured polygon mesh by searching from the first hole polygon for a plurality of connected hole polygons using a tree structure search;

determining an area of the hole is below a predetermined threshold;

identifying boundary polygons in the second set of polygons that are contiguous to the hole;

estimating texture information of the hole polygons based on texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons;

repairing the textured polygon mesh using the estimated texture information of the hole polygons; and

providing a virtual reality tour of the property in the repaired three-dimensional virtual reality model.

20. The non-transitory computer-readable medium of claim 19 , wherein estimating texture information of the hole polygons further includes:

determining pixel values for vertices of each hole polygon as a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by distances between the vertices of the hole polygon and vertices of the respective boundary vertices; and

determining pixel values inside each hole polygon based on the pixel values of its vertices based on a Gaussian process.