Apparatus and method for creating floor plans
View Patent ↗A system which generates a floor plan of a building includes: a camera which obtains a series of images as it is moved, each image represented by a first data set representing color and a second 3D data set representing depth. A processor generates the floor plan from the image data, defined by polylines that represent structures of the building and polygons that represent an area which has been observed by the camera. Suitably, the processor: performs a local matching sub-process which is operative to align two adjacent images with one another; performs a global matching sub-process in which key images and registered to one another; finds a 2D subset of points from the image data associated with each image, corresponding to a plane defined therethrough; determines the polylines based on said subset of points; and defines the polygons based on the polylines and a determined pose of the camera.
1. A system for automatically generating a two-dimensional (2D) floor plan of a building being studied, said system comprising:
a camera that is employed to obtain a series of images over time as the camera is moved about a floor of the building being studied, each obtained image being represented by image data including a first set of data representing one or more color values of points observed by the camera and a second set of three-dimensional (3D) data representative of distances from the camera to the points observed by the camera; and
a processor provisioned to generate the 2D floor plan from the image data, said 2D floor plan being defined by one or more polylines that represent one or more structures of the building and one or more polygons that represent an area of the floor which has been observed by the camera;
wherein to generate the 2D floor plan, said processor:
performs a local matching sub-process which is operative to align two adjacent images with one another, said adjacent images being obtained by the camera one after another in time;
performs a global matching sub-process in which one or more key images are identified from the series of image and two key images, obtained by the camera apart in time from one another but sharing similar image content, are registered to one another;
finds, for each image of the series, a 2D subset of points from the image data associated with each image, said 2D subset of points corresponding to a plane defined through the image data;
determines the polylines based on said 2D subset of points; and
defines the polygons based on the determined polylines and a determined pose of the camera.
2. The system of claim 1 , wherein said camera is a video camera and the images in the series of images correspond to video frames captured by the video camera.
3. The system of claim 1 , further comprising:
a handheld device in operative communication with said processor such that the generated 2D floor plan is selectively output on a screen of the handheld device.
4. The system of 1 , wherein the 2D floor plan is incrementally grown in real or near-real time as the camera is moved about a floor of the building being studied.
5. The system of claim 1 , further comprising a user interface through which a user can selectively make annotations to the generated 2D floor plan.
6. The system of claim 1 , further comprising:
a humanly wearable device which carries the processor and on which the camera is mounted.
7. The system of claim 1 , wherein the processor further performs a preprocessing sub-process prior to the local matching, said preprocessing sub-process being operative to: apply a camera calibration result to un-distort the image data, align the first and second data sets to one another and down-sample the image data.
8. The system of claim 1 , wherein the local matching sub-process includes: identifying a set of one or more consistent features in the two adjacent images and matching said features between the two adjacent images.
9. The system of claim 1 , wherein the processor determines said polylines by: extracting line segments from the 2D subset of points; rectifying the extracted line segments to a set of determined orientations; merging line segments groups by their rectified orientation; identifying valid and invalid intersections of merged lines segments; and defining the polylines based on the merged lines segments and identified intersections.
10. A method for automatically generating a two-dimensional (2D) floor plan of a building being studied, said method comprising:
obtaining a series of images captured over time by a camera as the camera is moved about a floor of the building being studied, each obtained image being represented by image data including a first set of data representing one or more color values of points observed by the camera and a second set of three-dimensional (3D) data representative of distances from the camera to the points observed by the camera; and
generating the 2D floor plan from the image data, said 2D floor plan being defined by one or more polylines that represent one or more structures of the building and one or more polygons that represent an area of the floor which has been observed by the camera;
wherein said generating includes:
performing a local matching sub-process which is operative to align two adjacent images with one another, said adjacent images being obtained by the camera one after another in time;
performing a global matching sub-process in which one or more key images are identified from the series of image and two key images, obtained by the camera apart in time from one another but sharing similar image content, are registered to one another;
finding, for each image of the series, a 2D subset of points from the image data associated with each image, said 2D subset of points corresponding to a plane defined through the image data;
determining the polylines based on said 2D subset of points; and
defining the polygons based on the determined polylines and a determined pose of the camera.
11. The method of claim 10 , wherein said camera is a video camera and the images in the series of images correspond to video frames captured by the video camera.
12. The method of claim 10 , further comprising:
outputting the 2D floor plan on a screen.
13. The method of 10 , wherein the 2D floor plan is incrementally grown in real or near-real time as the camera is moved about a floor of the building being studied.
14. The method of claim 10 , further comprising:
providing a user interface through which annotations to the generated 2D floor plan can be received from a user.
15. The method of claim 10 , further comprising performing a preprocessing sub-process prior to the local matching, said preprocessing sub-process being operative to: apply a camera calibration result to un-distort the image data, align the first and second data sets to one another and down-sample the image data.
16. The method of claim 10 , wherein the local matching sub-process includes: identifying a set of one or more consistent features in the two adjacent images and matching said features between the two adjacent images.
17. The method of claim 10 , wherein determining said polylines includes: extracting line segments from the 2D subset of points; rectifying the extracted line segments to a set of determined orientations; merging line segments groups by their rectified orientation; identifying valid and invalid intersections of merged lines segments; and defining the polylines based on the merged lines segments and identified intersections.