Generating and validating a virtual 3D representation of a real-world structure

1. A method for generating a 3D model of a real-world structure, the method comprising:

for each image of a plurality of images of the real-world structure, identifying one or more landmarks in the image;

providing metadata for the one or more identified landmarks of each image, the metadata for each landmark of the one or more identified landmarks specifying a shape of the landmark and a label for the landmark;

correlating one or more of the identified landmarks in each image of the plurality of images with one or more identified landmarks in other images of the plurality of images;

selecting, from a plurality of structure categories, one or more structure categories applicable to the real-world structure, wherein each structure category comprises structure data that defines an overall peripheral shape of structures belonging to the structure category, wherein selecting one or more structure categories applicable to the real-world structure is based, at least in part, on one or more of the identified landmarks; and

generating the 3D model of the real-world structure based, at least in part, on:

the identified landmarks,

the correlations between the identified landmarks in different ones of the plurality of images; and

geometrical constraints for the identified landmarks.

2. The method of claim 1 , wherein generating the 3D model of the real-world structure comprises:

generating, based on the overall peripheral shape, a portion of the 3D model that corresponds to a portion of the real-world structure that is not visible in the plurality of images.

3. The method of claim 1 , further comprising validating the 3D model by:

comparing the 3D model and a reference representation of the real-world structure, wherein the reference representation comprises one or more of: a digital image, an outline, a wireframe representation, and a second 3D model;

calculating a reprojection error based on the comparison; and

determining, based on the reprojection error, whether a particular structure category of the one or more selected structure categories is a correct structure category for the real-world structure.

4. The method of claim 1 , wherein the one or more landmarks comprises one or more lines, wherein the one or more lines comprise one or more of:

lines forming a gable;

lines around windows and doors;

lines marking a side of the real-world structure;

an apex of a roof;

a lower apex of a roof;

fascia lines; and

eve lines.

5. The method of claim 1 , wherein identifying the one or more landmarks in the image comprises:

identifying a view of the real-world structure depicted by the image;

identifying one or more expected landmarks for the view; and

identifying one or more of the expected landmarks within the image, wherein the one or more of the expected landmarks are the one or more landmarks.

6. The method of claim 1 , wherein identifying the one or more landmarks in the image comprises:

identifying one or more potentially significant areas in the image, wherein the potentially significant areas comprise one or more of:

areas in which multiple lines intersect; and

areas depicting structure features.

7. The method of claim 1 , wherein correlating the one or more of the identified landmarks comprises determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more of the identified landmarks in others of the plurality of images, and

further comprising, in response to determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more landmarks of the identified landmarks in other images of the plurality of images:

generating a notification to re-capture the image with no correlating landmarks.

8. A system for forming a 3D model of a real-world structure, the system comprising:

one or more processors;

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

for each image of a plurality of images of the real-world structure, identifying one or more landmarks in the image;

providing metadata for the one or more identified landmarks of each image, the metadata for each landmark of the one or more identified landmarks specifying a shape of the landmark and a label for the landmark;

correlating one or more of the identified landmarks in each image of the plurality of images with one or more identified landmarks in other images of the plurality of images;

selecting, from a plurality of structure categories, one or more structure categories applicable to the real-world structure, wherein each structure category comprises structure data that defines an overall peripheral shape of structures belonging to the structure category, wherein selecting one or more structure categories applicable to the real-world structure is based, at least in part, on one or more of the identified landmarks; and

generating the 3D model of the real-world structure based, at least in part, on:

the identified landmarks,

the correlations between the identified landmarks in different ones of the plurality of images; and

geometrical constraints for the identified landmarks.

9. The system of claim 8 , wherein generating the 3D model of the real-world structure comprises:

generating, based on the overall peripheral shape, a portion of the 3D model that corresponds to a portion of the real-world structure that is not visible in the plurality of images.

10. The system of claim 8 , wherein the operations further comprise validating the 3D model by:

comparing the 3D model and a reference representation of the real-world structure, wherein the reference representation comprises one or more of: a digital image, an outline, a wireframe representation, and a second 3D model;

calculating a reprojection error based on the comparison; and

determining, based on the reprojection error, whether a particular structure category of the one or more selected structure categories is a correct structure category for the real-world structure.

11. The system of claim 8 , wherein the one or more landmarks comprises one or more lines, wherein the one or more lines comprise one or more of:

lines forming a gable;

lines around windows and doors;

lines marking a side of the real-world structure;

an apex;

a lower apex of a roof;

fascia lines; and

eve lines.

12. The system of claim 8 , wherein identifying the one or more landmarks in the image comprises:

identifying a view of the real-world structure depicted by the image;

identifying one or more expected landmarks for the view; and

identifying one or more of the expected landmarks within the image, wherein the one or more of the expected landmarks are the one or more landmarks.

13. The system of claim 8 , wherein identifying the one or more landmarks in the image comprises:

identifying one or more potentially significant areas in the image, wherein the potentially significant areas comprise one or more of:

areas in which multiple lines intersect; and

areas depicting structure features.

14. The system of claim 8 , wherein correlating one or more of the identified landmarks comprises determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more of the identified landmarks in others of the plurality of images, and

further comprising, in response to determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more landmarks of the identified landmarks in other images of the plurality of images: generating a notification to re-capture the image with no correlating landmarks.

15. Non-transitory computer storage media storing instructions that when executed by a system comprising one or more processors, cause the one or more processors to perform operations comprising:

for each image of a plurality of images of the real-world structure, identifying one or more landmarks in the image;

providing metadata for the one or more identified landmarks of each image, the metadata for each landmark of the one or more identified landmarks specifying a shape of the landmark and a label for the landmark;

correlating one or more of the identified landmarks in each image of the plurality of images with one or more identified landmarks in other images of the plurality of images;

selecting, from a plurality of structure categories, one or more structure categories applicable to the real-world structure, wherein each structure category comprises structure data that defines an overall peripheral shape of structures belonging to the structure category, wherein selecting one or more structure categories applicable to the real-world structure is based, at least in part, on one or more of the identified landmarks; and

generating the 3D model of the real-world structure based, at least in part, on:

the identified landmarks,

the correlations between the identified landmarks in different ones of the plurality of images; and

geometrical constraints for the identified landmarks.

16. The non-transitory computer storage media of claim 15 , wherein generating the 3D model of the real-world structure comprises:

generating, based on the overall peripheral shape, a portion of the 3D model that corresponds to a portion of the real-world structure that is not visible in the plurality of images.

17. The non-transitory computer storage media of claim 15 , wherein the operations further comprise validating the 3D model by:

comparing the 3D model and a reference representation of the real-world structure, wherein the reference representation comprises one or more of: a digital image, an outline, a wireframe representation, and a second 3D model;

calculating a reprojection error based on the comparison; and

determining, based on the reprojection error, whether a particular structure category of the one or more selected structure categories is a correct structure category for the real-world structure.

18. The non-transitory computer storage media of claim 15 , wherein the one or more landmarks comprises one or more lines, wherein the one or more lines comprise one or more of:

lines forming a gable;

lines around windows and doors;

lines marking a side of the real-world structure;

an apex;

a lower apex of a roof;

fascia lines; and

eve lines.

19. The non-transitory computer storage media of claim 15 , wherein identifying the one or more landmarks in the image comprises:

identifying a view of the real-world structure depicted by the image;

identifying one or more expected landmarks for the view; and

identifying one or more of the expected landmarks within the image, wherein the one or more of the expected landmarks are the one or more landmarks.

20. The non-transitory computer storage media of claim 15 , wherein identifying the one or more landmarks in the image comprises:

identifying one or more potentially significant areas in the image, wherein the potentially significant areas comprise one or more of:

areas in which multiple lines intersect; and

areas depicting structure features.

21. The non-transitory computer storage media of claim 15 , wherein correlating one or more of the identified landmarks comprises determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more of the identified landmarks in others of the plurality of images, and

further comprising, in response to determining that one or more of the identified landmarks in an image of the plurality of images do not correlate to one or more landmarks of the identified landmarks in other images of the plurality of images: generating a notification to re-capture the image with no correlating landmarks.