Automated classification based on photo-realistic image/model mappings
Techniques are provided for increasing the accuracy of automated classifications produced by a machine learning engine. Specifically, the classification produced by a machine learning engine for one photo-realistic image is adjusted based on the classifications produced by the machine learning engine for other photo-realistic images that correspond to the same portion of a 3D model that has been generated based on the photo-realistic images. Techniques are also provided for using the classifications of the photo-realistic images that were used to create a 3D model to automatically classify portions of the 3D model. The classifications assigned to the various portions of the 3D model in this manner may also be used as a factor for automatically segmenting the 3D model.
1. A method comprising:
determining a target portion of a 3D-model, the 3D-model being previously generated based on a collection of digital images;
identifying a plurality of source-regions in the collection of digital images, each of the plurality of source-regions corresponding to the target portion of the 3D-model;
assigning a source-region classification to each of the plurality of source-regions in the collection of digital images; and
assigning a particular classification to a target region of a target digital image based, at least in part, on the assigned source-region classifications for each of the plurality of source-regions in the collection of digital images.
2. The method of claim 1 , further comprising: determining the target region in the collection of digital images.
3. The method of claim 2 , wherein the target portion corresponding to a portion of the 3D-model that the target region maps to.
4. The method of claim 1 , wherein the identifying the plurality of source-regions in the collection of digital images is based on spatial metadata of the collection of digital images.
5. The method of claim 4 , wherein the spatial metadata includes capture location orientation data and depth data of the collection of digital images.
6. The method of claim 1 , wherein the 3D-model is a 3D-model of a building and the target portion of the 3D-model represents a particular room in the building.
7. The method of claim 6 , wherein the particular classification assigned to the target region is a room-type classification.
8. The method of claim 1 , wherein assigning the particular classification to the target region comprises:
aggregating the classifications assigned to the plurality of source-regions to generate an aggregate classification;
assigning the aggregate classification to the target portion of the 3D-model; and
determining the particular classification based, at least in part, on the aggregate classification.
9. The method of claim 1 , wherein at least one source-region of the plurality of source-regions is a set of pixels within a digital image from the collection of digital images.
10. The method of claim 1 , further comprising, providing, to one or more computing devices, a confidence score associated with the target region classification.
11. The method of claim 10 , wherein the confidence score is calculated based on an aggregate confidence scores of the plurality of source-regions corresponding to the target portion of the 3D-model.
12. The method of claim 11 , wherein the aggregate of confidence scores is a weighted average of the plurality of source-regions corresponding to the target portion of the 3D-model.
13. One or more non-transitory computer-readable media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform:
determining a target portion of a 3D-model, the 3D-model being previously generated based on a collection of digital images;
identifying a plurality of source-regions in the collection of digital images, each of the plurality of source-regions corresponding to the target portion of the 3D-model;
assigning a source-region classification to each of the plurality of source-regions in the collection of digital images; and
assigning a particular classification to a target region of a target digital image based, at least in part, on the assigned source-region classifications for each of the plurality of source-regions in the collection of digital images.
14. The one or more non-transitory computer-readable media of claim 13 , wherein the instructions, when executed by one or more computing devices, further cause, determining the target region in the collection of digital images.
15. The one or more non-transitory computer-readable media of claim 13 , wherein the target portion corresponding to a portion of the 3D-model that the target region maps to.
16. The one or more non-transitory computer-readable media of claim 13 , wherein the identifying the plurality of source-regions in the collection of digital images is based on spatial metadata of the collection of digital images.
17. The one or more non-transitory computer-readable media of claim 16 , wherein the spatial metadata includes capture location orientation data and depth data of the collection of digital images.
18. The one or more non-transitory computer-readable media of claim 13 , wherein the 3D-model is a 3D-model of a building and the target portion of the 3D-model represents a particular room in the building.
19. The one or more non-transitory computer-readable media of claim 18 , wherein the particular classification assigned to the target region is a room-type classification.
20. The one or more non-transitory computer-readable media of claim 13 , wherein assigning the particular classification to the target region comprises:
aggregating the classifications assigned to the plurality of source-regions to generate an aggregate classification;
assigning the aggregate classification to the target portion of the 3D-model; and
determining the particular classification based, at least in part, on the aggregate classification.
21. The one or more non-transitory computer-readable media of claim 13 , wherein at least one source-region of the plurality of source-regions is a set of pixels within a digital image from the collection of digital images.
22. The one or more non-transitory computer-readable media of claim 13 , wherein the instructions, when executed by one or more computing devices, further cause, providing, to one or more computing devices, a confidence score associated with the target region classification.
23. The one or more non-transitory computer-readable media of claim 22 , wherein the confidence score is calculated based on an aggregate confidence scores of the plurality of source-regions corresponding to the target portion of the 3D-model.
24. The one or more non-transitory computer-readable media of claim 23 , wherein the aggregate of confidence scores is a weighted average of the plurality of source-regions corresponding to the target portion of the 3D-model.