Method for property feature segmentation
The method for determining property feature segmentation includes: receiving a region image for a region; determining parcel data for the region; determining a final segmentation output based on the region image and parcel data using a trained segmentation module; optionally generating training data; and training a segmentation module using the training data S 500.
1. A method, comprising:
receiving a region image that depicts a property feature;
determining an instance-aware mask for the property feature and a semantic segmentation mask for the property feature, based on the region image;
computing a distance transform from the instance-aware mask;
determining a flooded mask by assigning instance identifiers to pixels of the instance-aware mask based on the distance transform; and
generating a pixel-accurate mask by combining the flooded mask and the semantic segmentation mask.
2. The method of claim 1 , wherein the region image is an aerial image.
3. The method of claim 1 , the region image comprises depth information.
4. The method of claim 1 , wherein the property feature comprises at least one of: a roof, driveway, paved surface, vegetation, or waterfront.
5. The method of claim 1 , further comprising:
retrieving parcel data for a region depicted in the region image; and
using the parcel data, in addition to the region image, to determine the instance-aware mask and the semantic segmentation mask.
6. The method of claim 5 , wherein the parcel data is retrieved from a third-party datastore.
7. The method of claim 5 , wherein the parcel data represents parcel boundaries for parcels depicted in the region image.
8. The method of claim 1 , wherein the instance-aware mask represents an under segmented version of the property feature, and wherein property features belonging to different parcels are separated by a predetermined distance of pixels of the instance-aware mask.
9. The method of claim 1 , wherein the instance-aware mask and the semantic segmentation mask are determined by a segmentation module.
10. The method of claim 9 , wherein the segmentation module is trained using training data generated by:
receiving multiple property feature segment polygon sets for a train image, wherein the train image depicts a plurality of property features in a region;
retrieving parcel data for the region;
generating an instance polygon set by combining contiguous property feature segment polygons, from a property feature segment polygon set, that are within a parcel defined by the parcel data; and
generating a label for the train image by combining the instance polygon sets;
wherein the train image and the label are used to train the segmentation module.
11. The method of claim 1 , wherein the distance transform determines, for pixels in the instance-aware mask, the distance to a nearest property feature instance represented by the instance-aware mask, and wherein the instance identifiers are assigned to the pixels in the instance-aware mask, using the watershed transform.
12. A method for generating training data for property feature segmentation, comprising:
receiving a region image of a property feature;
receiving multiple property feature segment polygon sets for the region image;
retrieving parcel data for a region depicted in the region image;
generating an instance polygon set by combining contiguous property feature segment polygons of a property feature segment polygon set that are within a common parcel, where in the common parcel is determined based on the parcel data; and
generating a label for the region image by combining the instance polygon sets;
wherein the region image and the label are used to train a segmentation module.
13. The method of claim 12 , wherein the label comprises a foreground heatmap and an edge heatmap, wherein the segmentation module comprises a first output channel that is configured to output a first output heatmap, wherein the segmentation module receives the region image as input and is trained based on a comparison between the first output heatmap and the foreground heatmap.
14. The method of claim 12 , wherein the label comprises a foreground heatmap and an edge heatmap, the method further comprising:
calculating an instance-aware heatmap by multiplying the foreground heatmap by a transformation of the edge heatmap; and
using the instance-aware heatmap to train the segmentation module.
15. The method of claim 14 , wherein the segmentation module comprises an additional second output head that outputs a second output heatmap, wherein a comparison between the second output heatmap and the instance-aware heatmap is used to train the segmentation module.
16. The method of claim 12 , wherein the segmentation module is trained using a weighted loss function, wherein pixels inside of a predetermined area of the region image are weighted more than pixels outside of the predetermined area.
17. The method of claim 16 , wherein the pixels inside of the predetermined area are weighted using a weight mask, wherein the weight mask comprises a transformation of the label, and wherein the transformation is multiplied by a weight factor.
18. The method of claim 12 , wherein the label comprises a foreground heatmap and an edge heatmap, wherein the foreground heatmap comprises rendered interiors of polygons, and wherein the edge heatmap comprises rendered exterior edges of polygons.
19. The method of claim 12 , wherein the multiple property feature segment polygon sets are determined manually and the multiple instance polygon sets are determined automatically.
20. The method of claim 12 , wherein the region image is an aerial image.
21. A system, comprising a set of processors connected to memory, that is configured to perform the method of any of claims 1 - 20 .