System and methods for refining room segments to improve aesthetic quality for end-user applications
The disclosure presents an efficient method for improving the aesthetic quality of noisy room segments corresponding to a scanned environment, such as those used by mobile robots to navigate an environment to perform a task. Room polygons are extracted from the noisy room segments and visual scores of the extracted room polygons are assessed. Based on the visual score, low scoring room polygons are removed or merged. A polygon mesh is formed from the room segments, which is then simplified and aligned using efficient methods. Finally, room segments and boundaries between rooms are recovered to generate refined room segments from the polygon mesh. Compared to the noisy room segments, the refined room segments are better suited for visual presentation to users, such as in mobile applications for operating such mobile robots.
1 . A method for refining room segmentation of a scanned environment, the method comprising:
receiving, with a processor, a plurality of input room segments corresponding to the scanned environment;
extracting, with the processor, a plurality of extracted room polygons from the plurality of input room segments;
determining, with the processor, a respective score for each respective extracted room polygon in the plurality of extracted room polygons;
generating, with the processor, a plurality of refined room polygons by, for each respective extracted room polygon in the plurality of extracted room polygons, (i) including the respective extracted room polygon in the plurality of refined room polygons in response to the respective score being greater than a threshold score, and (ii) excluding the respective extracted room polygon from the plurality of refined room polygons in response to the respective score being less than the threshold score and the respective extracted room polygon being isolated from each other extracted room polygon in the plurality of extracted room polygons;
determining, with the processor, a polygon mesh based on the plurality of refined room polygons; and
generating, with the processor, a plurality of refined room segments based on the polygon mesh.
2 . The method according to claim 1 , the extracting the plurality of extracted room polygons further comprising:
extracting the plurality of extracted room polygons using a contour detection technique.
3 . The method according to claim 1 , the refining the plurality of extracted room polygons further comprising at least one of:
removing at least one extracted room polygon from the plurality of extracted room polygons; and
merging at least two extracted room polygons from the plurality of extracted room polygons.
4 . The method according to claim 1 , the refining the plurality of extracted room polygons further comprising, for each respective extracted room polygon in the plurality of extracted room polygons:
determining whether the respective extracted room polygon is isolated from each other extracted room polygon in the plurality of extracted room polygons, in response to the respective score being less than the threshold score.
5 . The method according to claim 1 , the refining the plurality of extracted room polygons further comprising, for each respective extracted room polygon in the plurality of extracted room polygons:
determining a merged room polygon by merging the respective extracted room polygon with an adjacent extracted room polygon of the plurality of extracted room polygons in response to (i) the respective score being less than the threshold score and (ii) the respective extracted room polygon intersecting with at least one other extracted room polygon in the plurality of extracted room polygons; and
including the merged room polygon in the plurality of refined room polygons.
6 . The method according to claim 1 , the determining the respective score for the respective extracted room polygon further comprising:
determining a rotated bounding box that encompasses the respective extracted room polygon with a smallest area; and
determining the respective score based on (i) an area of the respective extracted room polygon and (ii) at least one of dimensions of the rotated bounding box and an area of the rotated bounding box.
7 . The method according to claim 1 , the determining the polygon mesh further comprising:
determining vertices of the polygon mesh by merging adjacent polygon vertices of the plurality of refined room polygons.
8 . The method according to claim 1 , the determining the polygon mesh further comprising:
simplifying the polygon mesh by removing vertices from the polygon mesh.
9 . The method according to claim 8 , the simplifying the polygon mesh further comprising, for each respective polygon in the polygon mesh:
selecting at least one vertex of the respective polygon to be removed using a polyline simplification technique.
10 . The method according to claim 9 , the simplifying the polygon mesh further comprising, for each respective polygon in the polygon mesh:
identifying any shared vertices of the respective polygon that are shared by another polygon of the polygon mesh; and
applying the polyline simplification technique in such a manner that the shared vertices are not candidates for removal from the polygon mesh.
11 . The method according to claim 1 , the determining the polygon mesh further comprising:
aligning at least one edge of the polygon mesh with a horizontal axis or a vertical axis.
12 . The method according to claim 11 , the aligning the at least one edge of the polygon mesh further comprising:
determining that the at least one edge of the polygon mesh has an angle with respect to the horizontal or the vertical axis that is less than a threshold angle; and
repositioning at least one vertex of the at least one edge so as to align the at least one edge with the horizontal axis or the vertical axis.
13 . The method according to claim 1 , the generating the plurality of refined room segments further comprising:
determining, for each respective polygon in the polygon mesh, an input room segment from the plurality of input room segments to which the respective polygon corresponds;
determining which edges of the polygon mesh correspond to room boundaries based on the determined input room segments to which each polygon in the polygon mesh corresponds; and
generating the plurality of refined room segments using the edges of each polygon in the polygon mesh that correspond to room boundaries and segmented according to the determined input room segment to which each respective polygon corresponds.
14 . The method according to claim 13 , the determining the input room segment to which each respective polygon corresponds further comprising:
determining, for each sample position of a plurality of sample positions within the respective polygon, a corresponding room segment identifier at the sample position within the plurality of input room segments; and
determining a room segment identifier for the respective polygon as a most common corresponding room segment identifier among the plurality of sample positions within the plurality of input room segments.
15 . The method according to claim 13 , the determining which edges of the polygon mesh correspond to room boundaries further comprising:
dividing each edge of the polygon mesh into fixed length edge segments;
determining, for each respective fixed length edge segment of each edge of the polygon mesh, whether rooms are on both sides of the respective fixed length edge segment; and
determining, for each respective fixed length edge segment of each edge of the polygon mesh, that the respective fixed length edge segment corresponds to a room boundary in response to determining that rooms are not on both sides of the respective fixed length edge segment.
16 . The method according to claim 1 further comprising:
displaying, on a display, the plurality of refined room segments to a user;
receiving from the user, via a user interface, a selection of at least one room segment of the plurality of refined room segments; and
operating a mobile robot to perform a task in the scanned environment depending on the selection.
17 . A method for refining room segmentation of a scanned environment, the method comprising:
receiving, with a processor, a plurality of input room segments corresponding to the scanned environment;
extracting, with the processor, a plurality of extracted room polygons from the plurality of input room segments;
determining, with the processor, a respective score for each respective extracted room polygon in the plurality of extracted room polygons;
generating, with the processor, a plurality of refined room polygons by, for each respective extracted room polygon in the plurality of extracted room polygons, (i) including the respective extracted room polygon in the plurality of refined room polygons in response to the respective score being greater than a threshold score, (ii) determining a merged room polygon by merging the respective extracted room polygon with an adjacent extracted room polygon of the plurality of extracted room polygons in response to the respective score being less than the threshold score and the respective extracted room polygon intersecting with at least one other extracted room polygon in the plurality of extracted room polygons, the merged room polygon being included in the plurality of refined room polygons;
determining, with the processor, a polygon mesh based on the plurality of refined room polygons; and
generating, with the processor, a plurality of refined room segments based on the polygon mesh.