Machine learning techniques for extracting floorplan elements from architectural drawings
One embodiment of the present invention sets forth a technique for extracting data from an architectural drawing. The technique includes performing one or more operations via one or more machine learning models to extract a first image of a floorplan area from the architectural drawing. The technique also includes generating a boundary segmentation based on the first image of the floorplan area, wherein the boundary segmentation includes one or more boundary types for one or more portions of the floorplan area.
1. A computer-implemented method for extracting data from an architectural drawing, the method comprising:
performing one or more operations via one or more machine learning models to identify that at least a portion of the architectural drawing includes a first image of a floorplan area by determining that text extracted from the architectural drawing corresponds to at least one predetermined keyword associated with floorplans;
in response, extracting the first image of the floorplan area from the architectural drawing; and
generating a boundary segmentation based on the first image of the floorplan area, wherein the boundary segmentation includes one or more boundary types for one or more portions of the floorplan area.
2. The computer-implemented method of claim 1 , further comprising generating a room segmentation of the floorplan area based on a vectorized representation of the boundary segmentation by partitioning the floorplan area into a plurality of regions representing a plurality of rooms.
3. The computer-implemented method of claim 2 , wherein generating the room segmentation of the floorplan area further comprises:
adjusting a plurality of endpoints for a plurality of line segments included in the vectorized representation of the boundary segmentation to produce a plurality of adjusted line segments; and
determining the plurality of regions based on a plurality of intersection points associated with the plurality of adjusted line segments.
4. The computer-implemented method of claim 1 , wherein performing the one or more operations via the one or more machine learning models comprises executing a classifier that determines a drawing type associated with the architectural drawing based on a second image of the architectural drawing.
5. The computer-implemented method of claim 4 , wherein executing the classifier comprises:
applying the classifier to the image of the floorplan area to generate a set of scores; and
applying a threshold to the set of scores to determine the drawing type.
6. The computer-implemented method of claim 1 , wherein performing the one or more operations via the one or more machine learning models comprises executing a detection model that generates a bounding box for the floorplan area based on a second image of the architectural drawing.
7. The computer-implemented method of claim 1 , further comprising:
generating a training dataset based on one or more outputs generated by the one or more machine learning models; and
updating a plurality of parameters of the one or more machine learning models based on the training dataset.
8. The computer-implemented method of claim 7 , wherein generating the training dataset comprises combining the first image of the floorplan area with one or more images extracted from one or more additional architectural drawings to generate a synthetic floorplan.
9. The computer-implemented method of claim 1 , wherein generating the boundary segmentation comprises:
applying an encoder to the first image to generate a set of features; and
applying a decoder to the set of features to generate the boundary segmentation.
10. The computer-implemented method of claim 1 , wherein the one or more boundary types comprise at least one of a wall, a window, an interior door, an exterior door, a railing, or a stairway.
11. One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
performing one or more operations via one or more machine learning models to identify that at least a portion of an architectural drawing includes a first image of a floorplan area by determining that text extracted from the architectural drawing corresponds to at least one predetermined keyword associated with floorplans;
in response, extracting the first image of the floorplan area from the architectural drawing; and
generating a boundary segmentation based on the first image of the floorplan area, wherein the boundary segmentation includes one or more boundary types for one or more portions of the floorplan area.
12. The one or more non-transitory computer readable media of claim 11 , wherein the instructions further cause the one or more processors to perform the step of generating a room segmentation of the floorplan area based on a vectorized representation of the boundary segmentation by partitioning of the floorplan area into a plurality of regions representing a plurality of rooms.
13. The one or more non-transitory computer readable media of claim 12 , wherein generating the room segmentation of the floorplan area further comprises:
adjusting a plurality of endpoints for a plurality of line segments included in the vectorized representation of the boundary segmentation to produce a plurality of adjusted line segments; and
determining the plurality of regions based on a plurality of intersection points associated with the plurality of adjusted line segments.
14. The one or more non-transitory computer readable media of claim 12 , wherein the instructions further cause the one or more processors to perform the step of retrieving at least one of the first image, the boundary segmentation, or the room segmentation in response to a search associated with the architectural drawing.
15. The one or more non-transitory computer readable media of claim 11 , wherein the instructions further cause the one or more processors to perform the steps of:
generating a training dataset based on one or more outputs generated by the one or more machine learning models; and
updating parameters of the one or more machine learning models based on the training dataset.
16. The one or more non-transitory computer readable media of claim 15 , wherein generating the training dataset comprises combining the first image of the floorplan area with one or more images extracted from one or more additional architectural drawings to generate a synthetic floorplan.
17. The one or more non-transitory computer readable media of claim 11 , wherein performing the one or more operations via the one or more machine learning models to comprises:
applying a classifier to a second image of the architectural drawing to determine a drawing type associated with the architectural drawing; and
when the drawing type corresponds to a floorplan, applying a detection model to the second image of the architectural drawing to generate a bounding box for the floorplan area.
18. The one or more non-transitory computer readable media of claim 17 , wherein performing the one or more operations via the one or more machine learning models further comprises applying one or more keyword filters to a set of documents to identify the architectural drawing.
19. The one or more non-transitory computer readable media of claim 11 , wherein the one or more boundary types comprise at least one of a wall, a window, an interior door, an exterior door, a railing, or a stairway.
20. A system, comprising:
one or more memories that store instructions, and
one or more processors that are coupled to the one or more memories and,
when executing the instructions, are configured to:
perform one or more operations via one or more machine learning models to identify that at least a portion of an architectural drawing includes a first image of a floorplan area by determining that text extracted from the architectural drawing corresponds to at least one predetermined keyword associated with floorplans;
in response, extract the first image of the floorplan area from the architectural drawing;
generate a boundary segmentation based on the first image of the floorplan area, wherein the boundary segmentation includes one or more boundary types for one or more portions of the floorplan area; and
generate a room segmentation of the floorplan area based on a vectorized representation of the boundary segmentation, wherein the room segmentation comprises a partitioning of the floorplan area into a plurality of regions representing a plurality of rooms.