Granted Patent B2

US 12,299,784 · App. 17/855,513 · Granted May 13, 2025

Inventor: Justin Meiners (Murray, UT)

Assignee: PassiveLogic, Inc.

G06T11/203G06T7/73G06T2207/10028

Abstract

Claims (67)

1. A machine-implemented method for extracting a floor plan from an imperfect floor plan scan composed of known line segments, comprising:

using a processor to generate a more-optimized floor plan;

displaying, by a user device with 3-D depth scanner and a 2-D camera, the more-optimized floor plan;

receiving feedback indicative of accuracy of the more-optimized floor plan by use of the 3-D depth scanner and the 2-D camera;

in response to the feedback, using the processor to adjust the more-optimized floor plan based on the feedback;

wherein receiving feedback indicative of accuracy of the floor plan comprises receiving a wall marking from a reading from the 3-D depth scanner and the 2-D camera, the wall marking comprising multiple positions and multiple orientations of a location in 2-D and 3-D, the wall marking indicated on the user device by a user;

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment, and wherein the directed line segment is added to the known line segments; and

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment further comprises using both depth scanner measurements and camera measurements to determine an average of the multiple positions and the multiple orientations.

2. The machine-implemented method of claim 1 , wherein applying a heuristic to the floor plan comprises:

for each line segment within the known line segments, extending the line segment by a known amount creating an associated with the line segment with a same orientation, and creating a second line segment of a second size perpendicular to and touching the line segment, creating extended line segments; and

adding the extended line segment and the second line segment to a set of extended line segments.

3. The machine-implemented method of claim 2 , wherein adding the chosen edge to the floor plan graph comprises adding the possible edge path between the two line segments to the floor plan graph.

4. The machine-implemented method of claim 3 , wherein finding a possible end path comprises:

determining a first angle between two nearest ends of the two line segments;

determining a second angle between a far end of a first of the two line segments and a far end of a second of the two line segments; and

when the first angle is below a first threshold amount and the second angle is below a second threshold amount joining the first of the two line segments and the second of the two line segments.

5. The machine-implemented method of claim 4 , wherein when the first angle is above the first threshold amount or the second line segment is above the second threshold amount then zig-zagging the two line segments.

6. The machine-Implemented method of claim 5 , wherein zigzagging the two line segments comprises creating a new line segment perpendicular to and between the first of the two line segments and extending a near end of the two line segments to the new line segment.

7. The machine-implemented method of claim 6 , wherein the cost further comprises adding a weighting value to the cost.

8. The method of claim 2 , further comprising for each two extended line segments within the set of extended line segments, adding possible intersections whose line segment orientations are similar to a set of possible intersections, and finding a possible edge path between two line segments associated with the extended line segments.

9. The method of claim 8 , further comprising creating a cost graph comprising a vertex set and an edge set, wherein the vertex set comprises vertices representing a midpoint of each line segment, and wherein the edge set comprises edges, each edge comprising an intersection within the set of possible intersections, a starting vertex of the intersection, an ending vertex of the intersection and a cost, wherein the cost includes length from the starting vertex to the ending vertex and aesthetic costs.

10. The method of claim 9 , further comprising iteratively:

in the cost graph, choose an edge with a lowest cost, giving a chosen edge;

add the chosen edge to a floor plan graph; and

delete each edge with the starting vertex equal to the chosen edge starting vertex;

until the floor plan graph forms a loop.

11. A system for extracting a floor plan from an imperfect floor plan scan composed of line segments, the system comprising: a memory; a processor in operable communication with the memory, and a user device with a 3-D depth scanner and a 2-D camera, the processor configured to for extracting a floor plan from an imperfect floor plan scan composed of known line segments, comprising:

generating a more-optimized floor plan by applying a heuristic to the imperfect floor plan scan;

displaying, by the user device, the more-optimized floor plan; receiving feedback from a user using the user device indicative of accuracy of the floor plan; and

in response to the feedback, adjusting the more-optimized floor plan based on the feedback;

wherein receiving feedback indicative of accuracy of the floor plan comprises receiving a wall marking from a reading from the 3-D depth scanner and the 2-D camera, the wall marking comprising multiple positions and multiple orientations of a location in 2-D and 3-D, the wall marking indicated on the user device by a user;

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment, and wherein the directed line segment is added to the known line segments; and

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment further comprises using both depth scanner measurements and camera measurements to determine an average of the multiple positions and the multiple orientations.

12. The system of claim 11 , wherein a line segment comprises an orientation, and wherein the known line segments comprises line segments.

13. The system of claim 12 , wherein applying a heuristic to the imperfect floor plan scan comprises:

for each line segment within the known line segments, extending the line segment by a known amount creating an extended line segment associated with the line segment with a same orientation, and creating a second line segment of a second size perpendicular to and touching the line segment;

adding the extended line segment and the second line segment to a set of extended line segments;

for each two extended line segments within the set of extended line segments, adding possible intersections whose line segment orientations associated with the extended line segments are similar to a set of possible intersections, and finding a possible edge path between two line segments associated with the extended line segments;

create a cost graph comprising a vertex set and an edge set, wherein the vertex set comprises vertices representing a midpoint of each line segment, and wherein the edge set comprises edges, each edge comprising an intersection within the set of possible intersections, a starting vertex of the intersection, an ending vertex of the intersection and a cost, wherein the cost includes length from the starting vertex to the ending vertex and aesthetic costs; and

iteratively:

in the cost graph, choose an edge with a lowest cost, giving a chosen edge;

add the chosen edge to a floor plan graph; and

delete each edge with the starting vertex equal to a starting edge of the chosen edge;

until the floor plan graph forms a loop.

14. The system of claim 13 , wherein receiving feedback from a user using the user device indicative of accuracy of the floor plan comprises the display showing a portion of a structure corresponding to the imperfect floor plan scan, and receiving a purposeful mark made on the display.

15. The system of claim 14 , wherein the processor processes the purposeful mark to create a directed line segment, and adds the directed line segment to the known line segments.

16. The system of claim 15 , further comprising deleting a vertex when the vertex leads to a dead end.

17. A non-transitory machine-readable storage medium configured with data and instructions which upon execution by at least one processor cause one or more devices to perform a method of extracting a floor plan from an imperfect floor plan scan composed of known line segments, the method comprising:

using a processor to generate a more-optimized floor plan by applying a heuristic to the imperfect floor plan scan;

displaying, by a user device with a 3-D depth scanner and a 2-D camera, the more-optimized floor plan;

receiving feedback indicative of accuracy of the floor plan by use of the user device; and

in response to the feedback, using the processor to adjust the more-optimized floor plan based on the feedback;

wherein receiving feedback indicative of accuracy of the floor plan comprises receiving a wall marking from a reading from the 3-D depth scanner and the 2-D camera, the wall marking comprising multiple positions and multiple orientations of a location in 2-D and 3-D, the wall marking indicated on the user device by a user;

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment, and wherein the directed line segment is added to the known line segments; and

wherein the processor processes the multiple positions and multiple orientations of a location in 2-D and 3-D to determine a directed line segment further comprises using both depth scanner measurements and camera measurements to determine an average of the multiple positions and the multiple orientations.

18. The non-transitory machine-readable storage medium of claim 17 , wherein applying a heuristic to the imperfect floor plan scan comprises:

for each line segment within the known line segments, extending the line segment by a known amount creating an extended line segment associated with the line segment with a same orientation, and creating a second line segment of a second size perpendicular to and touching the line segment;

adding the extended line segment and the second line segment to a set of extended line segments;

for each two extended line segments within the set of extended line segments, adding possible intersections, whose line segment orientations associated with the extended line segments are similar, to a set of possible intersections, and finding a possible edge path between two line segments associated with the extended line segments;

create a cost graph comprising a vertex set and an edge set, wherein the vertex set comprises vertices representing a midpoint of each line segment, and wherein the edge set comprises edges, each edge comprising an intersection within the set of possible intersections, a starting vertex of the intersection, an ending vertex of the intersection and a cost, wherein the cost includes length from the starting vertex to the ending vertex and aesthetic costs; and

iteratively:

in the cost graph, choose an edge with a lowest cost, giving a chosen edge;

add the chosen edge to a floor plan graph; and

delete each edge with the starting vertex equal to a starting vertex of the chosen edge;

until the floor plan graph forms a loop.

19. The non-transitory machine-readable storage medium of claim 18 , wherein the user device further comprises a display and wherein receiving feedback from a user using the LiDAR-enabled user device indicative of accuracy of the floor plan comprises the display showing a portion of a structure corresponding to the imperfect floor plan scan, and receiving a purposeful mark made on the display.

20. The non-transitory machine-readable storage medium of claim 19 , wherein the more-optimized floor plan is used to modify equipment location at a location represented by the imperfect floor plan scan.

Assignments (2)

Continuity (1)

References Cited (22)