IP Library Granted Patent US 10,891,769
Granted Patent B2
US 10,891,769 · App. 16/275,525 · Granted Jan 12, 2021

System and method of scanning two dimensional floorplans using multiple scanners concurrently

Inventors: João Santos (Kornwestheim, DE); Oliver Zweigle (Stuttgart, DE); Ahmad Ramadneh (Kornwestheim, DE); Aleksej Frank (Stuttgart, DE)
Assignee: FARO TECHNOLOGIES, INC
G06T11/60G06T3/0075G06T7/33G06T7/521G06T7/70H04N5/247H04N13/243H04N13/254H04W4/33G06T3/20G06T3/60G06T2207/10012G06T2207/10024G06T2207/10028G06T2207/10048H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,891,769
App. No.
16/275,525
Granted
Jan 12, 2021
Kind
B2
Abstract

One or more embodiments are described for generating a two dimensional map of an environment using a set of submaps that include point clouds of the environment that are captured using multiple scanner systems that move independently from one position to another in the environment. Each 2D scanner system steers a beam of light within a first plane to illuminate object points in the environment, and a controller determines a distance value to at least one of the object points. The 2D submaps of the environment are generated based on an activation signal from an operator and based at least in part on the distance value, each submap generated from a respective point in the environment and by a respective 2D scanner system. A central processor generates the 2D image of the environment using the 2D submaps.

Claims (59)

1. A system of generating a two-dimensional (2D) image of an environment, the system comprising:

a plurality of 2D scanner systems, each 2D scanner system comprises a light source, an image sensor, and a controller, and wherein, in each 2D scanner system, said light source steers a beam of light within a first plane to illuminate object points in the environment, said image sensor is arranged to receive light reflected from the object points, and said controller is operable to determine a distance value to at least one of the object points;

one or more processors operably coupled to the 2D scanner systems, the one or more processors being responsive to non-transitory executable instructions for generating a plurality of 2D submaps of the environment in response to an activation signal from an operator and based at least in part on the distance value, each submap generated from a respective point in the environment and by a respective 2D scanner system; and

a central processor operably coupled to the one or more processors, the central processor being responsive to non-transitory executable instructions for generating the 2D image of the environment using the 2D submaps from the 2D scanner system, wherein generating the 2D image of the environment using the 2D submaps comprises aligning a first submap to the 2D image of the environment, the alignment comprising:

performing at least one of a shift or a rotation of the first submap relative to the 2D image;

identifying overlapping natural features in the 2D image and the first submap; and

automatically shifting and rotating the first submap to align with the 2D image when the overlapping natural features are aligned by the operator within a predetermined threshold.

2. The system of claim 1 , wherein the 2D submaps are for distinct portions of the environment.

3. The system of claim 1 , further comprising a headgear for visualizing the 2D image of the environment in an augmented reality view.

4. The system of claim 3 , wherein the augmented reality view comprises an occupancy grid being mapped to the 2D image of the environment.

5. The system of claim 3 , wherein augmented reality view comprises an annotation being shown at coordinates corresponding to a predetermined location in the environment, the predetermined location having a point of interest for an operator of the system.

6. The system of claim 1 , wherein each of 2D scanner systems comprises:

the image sensor as part of a first equipment of an agent; and

the controller as part of a second equipment of the agent.

7. A method for generating a two-dimensional (2D) image of an environment, the method comprising:

integrating a plurality of measurement devices into a gear of plurality of field agents respectively;

moving each of the measurement devices to a plurality of registration positions in the environment, each of the measurement devices comprising a 2D scanner that is sized and weighted to be carried by a single operator, each of the 2D scanners being configured to sweep a beam of light in a horizontal plane the plurality of registration positions including a first registration position of a first measurement device corresponding to a first field agent and a second registration position of a second measurement device corresponding to a second field agent;

obtaining by the 2D scanners a plurality of 2D submaps, the first field agent obtaining a first submap from the first registration position and the second field agent obtaining a second submap from the second registration position, each of the plurality of submaps being a set of 2D coordinates of points in the environment, each of the plurality of submaps being collected by the 2D scanners at a different position relative to the first registration position; and

generating, by a central processor, a 2D image of the environment based at least in part on the plurality of 2D submaps from the plurality of 2D scanners, wherein generating the 2D image of the environment using the 2D submaps comprises aligning a first submap to the 2D image of the environment, the alignment comprising:

performing at least one of a shift or a rotation of the first submap relative to the 2D image;

identifying overlapping natural features in the 2D image and the first submap; and

automatically shifting and rotating the first submap to align with the 2D image when the overlapping natural features are aligned by the operator within a predetermined threshold.

8. The method of claim 7 , wherein the 2D submaps are for distinct portions of the environment.

9. The method of claim 7 , further comprising:

transmitting, by the central processor, the 2D image to the measurement devices for visualizing the 2D image by the corresponding field agents.

10. The method of claim 9 , further comprising:

emitting a visible laser light beam from a laser projector onto a point on an object, the laser projector being coupled to the measurement device;

measuring a distance from the measurement device to the point on the object using a three-dimensional camera coupled to the measurement device; and

annotating the 2D image to include a position of the point on the object based at least in part on the distance.

11. The method of claim 10 , wherein the 2D image is annotated by the first measurement device, and the method further comprising:

annotating the 2D image to include the position of the point on the object in the second measurement device.

12. The method of claim 9 , further comprising:

coupling for communication a portable computing device to the measurement device, the portable computing device having an image sensor;

acquiring a first image with the image sensor;

transmitting the first image to the measurement device; and

annotating the 2D image to include the first image, wherein a position of the first image in the 2D image is based at least in part on a device position of the measurement device when the first image is acquired by the image sensor.

13. A system of generating a two-dimensional (2D) image of an environment, the system comprising:

a display device;

a memory device; and

one or more processors coupled with the display device and the memory device, the one or more processors configured to:

receive a plurality of submaps of the environment from a plurality of 2D scanners, each submap generated by a respective 2D scanner being moved in separate portions of the environment, each of the plurality of submaps being a set of 2D coordinates of points in the environment, each of the plurality of submaps being collected by the 2D scanner at a different position relative to a starting position;

generate the 2D image of the environment based at least in part on the plurality of submaps, wherein generating the 2D image of the environment using the 2D submaps comprises aligning a first submap to the 2D image of the environment, the alignment comprising:

performing at least one of a shift or a rotation of the first submap relative to the 2D image;

identifying overlapping natural features in the 2D image and the first submap; and

automatically shifting and rotating the first submap to align with the 2D image when the overlapping natural features are aligned by an operator within a predetermined threshold; and

transmit the 2D image to the plurality of 2D scanners.

14. The system of claim 13 , wherein the one or more processors are further configured to:

receive a plurality of occupancy grids from the 2D scanners;

augment the occupancy grids with the 2D image; and

transmit the augmented view of the 2D image with the occupancy grid to the 2D scanners.

15. The system of claim 13 , wherein the one or more processors are further configured to:

receive an annotation for an object in the environment from a first 2D scanner, the annotation comprising a distance from the 2D scanner to a point on an object in the environment, wherein 2D scanner measures the distance using a three-dimensional camera coupled to 2D scanner by emitting a visible laser light beam from a laser projector onto the point on the object;

annotate the 2D image of the environment to include the annotation at a position of the point on the object based at least in part on the distance; and

transmit the annotated 2D image to the plurality of 2D scanners.

16. The system of claim 13 , wherein the one or more processors are further configured to:

receive an annotation for an object in the environment from a first 2D scanner, the annotation comprising a distance from the 2D scanner to a point on an object in the environment, wherein 2D scanner measures the distance using a three-dimensional camera coupled to 2D scanner by emitting a visible laser light beam from a laser projector onto the point on the object;

annotate the 2D image of the environment to include the annotation at a position of the point on the object based at least in part on the distance; and

transmit the annotated 2D image to the plurality of 2D scanners.

17. The system of claim 13 , wherein the one or more processors are at a remote location from the plurality of 2D scanners, each 2D scanner comprising one or more sensors integrated in equipment gear of an agent.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2019
From: SANTOS, JOAO; ZWEIGLE, OLIVER; RAMADNEH, AHMAD; FRANK, ALEKSEJ
To: FARO TECHNOLOGIES, INC.
Reel/Frame 048668/0168 →
Continuity (1)
Related Publication 20200265621A1 · Aug 20, 2020
Cited By (1)
US 12,436,288