Automated classification based on photo-realistic image/model mappings
Techniques are provided for increasing the accuracy of automated classifications produced by a machine learning engine. Specifically, the classification produced by a machine learning engine for one photo-realistic image is adjusted based on the classifications produced by the machine learning engine for other photo-realistic images that correspond to the same portion of a 3D model that has been generated based on the photo-realistic images. Techniques are also provided for using the classifications of the photo-realistic images that were used to create a 3D model to automatically classify portions of the 3D model. The classifications assigned to the various portions of the 3D model in this manner may also be used as a factor for automatically segmenting the 3D model.
1. A method comprising:
determining a target portion of a 3D-model, the 3D-model being previously generated based on a collection of digital images;
identifying at least two images in the collection of digital images, each of the at least two images including at least a portion of the target portion of the 3D-model;
assigning a portion classification to the portion of the target portion included in each of the at least two images in the collection of digital; and
assigning a particular classification to a target region based, at least in part, on the assigned portion classifications.
2. The method of claim 1 , further comprising: determining the target region in the collection of digital images.
3. The method of claim 1 , wherein the target portion corresponding to a portion of the 3D-model that the target region maps to.
4. The method of claim 1 , wherein identifying the at least two images of collection of digital images based on spatial metadata of the collection of digital images.
5. The method of claim 4 , wherein the spatial metadata includes capture location orientation data and depth data of the collection of digital images.
6. The method of claim 1 , wherein the 3D-model is a 3D-model of a building and the target portion of the 3D-model represents a particular room in the building.
7. The method of claim 6 , wherein the particular classification assigned to the target region is a room-type classification.
8. The method of claim 1 , wherein assigning the particular classification to the target region comprises:
aggregating the classifications assigned to the portion of the target portion included in each of the at least two images to generate an aggregate classification;
assigning the aggregate classification to the target portion of the 3D-model; and
determining the particular classification based, at least in part, on the aggregate classification.
9. The method of claim 1 , further comprising, providing, to one or more computing devices, a confidence score associated with the portion classification to the portion of the target portion included in each of the at least two images in the collection of digital images.
10. The method of claim 9 , wherein the confidence score is calculated based on an aggregate confidence scores of the at least two images corresponding to the target portion of the 3D-model.
11. The method of claim 10 , wherein the aggregate confidence scores is a weighted average of the confidence score associated with the portion classification corresponding to the target portion of the 3D-model.
12. One or more non-transitory computer-readable media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform:
determining a target portion of a 3D-model, the 3D-model being previously generated based on a collection of digital images;
identifying at least two images in the collection of digital images, each of the at least two images including at least a portion of the target portion of the 3D-model;
assigning a portion classification to the portion of the target portion included in each of the at least two images in the collection of digital images; and
assigning a particular classification to a target region based, at least in part, on the assigned portion classifications.
13. The one or more non-transitory computer-readable media of claim 12 , wherein the instructions, when executed by one or more computing devices, further cause, determining the target region in the collection of digital images.
14. The one or more non-transitory computer-readable media of claim 12 , wherein the target portion corresponding to a portion of the 3D-model that the target region maps to.
15. The one or more non-transitory computer-readable media of claim 12 , wherein identifying the at least two images of collection of digital images based on spatial metadata of the collection of digital images.
16. The one or more non-transitory computer-readable media of claim 15 , wherein the spatial metadata includes capture location orientation data and depth data of the collection of digital images.
17. The one or more non-transitory computer-readable media of claim 12 , wherein the 3D-model is a 3D-model of a building and the target portion of the 3D-model represents a particular room in the building.
18. The one or more non-transitory computer-readable media of claim 17 , wherein the particular classification assigned to the target region is a room-type classification.
19. The one or more non-transitory computer-readable media of claim 12 , wherein assigning the particular classification to the target region comprises:
aggregating the classifications assigned to the portion of the target portion included in each of the at least two images to generate an aggregate classification;
assigning the aggregate classification to the target portion of the 3D-model; and
determining the particular classification based, at least in part, on the aggregate classification.
20. The one or more non-transitory computer-readable media of claim 13 , wherein the instructions, when executed by one or more computing devices, further cause, providing, to one or more computing devices, a confidence score associated with the portion classification to the portion of the target portion included in each of the at least two images in the collection of digital images.
21. The one or more non-transitory computer-readable media of claim 20 , wherein the confidence score is calculated based on an aggregate confidence scores of the at least two images corresponding to the target portion of the 3D-model.
22. The one or more non-transitory computer-readable media of claim 21 , wherein the aggregate confidence scores are a weighted average of the confidence score associated with the portion classification corresponding to the target portion of the 3D-model.