Augmented reality of a building

1 . A method of responding to a hazardous condition using an augmented reality system, the method comprising:

receiving, at a self-propelled monitoring device, instructions to monitor an assigned area associated with a monitored structure;

the self-propelled monitoring device traveling to the assigned area in response to the instructions to monitor the assigned area;

capturing, by the self-propelled monitoring device, image data comprising one of a real-time or a near real-time video of the assigned area;

generating a three-dimensional representation of the assigned area from the captured image data while the self-propelled monitoring device is present in the assigned area;

receiving by the self-propelled monitoring device, sensor data directly from one or more building sensors associated with the assigned area, the sensor data comprising both occupancy data identifying persons within the assigned area and condition data indicative of a hazardous condition;

determining that a hazardous condition exists at least at a first location in the assigned area;

determining, based on sensor data from multiple locations within the assigned area, whether the hazardous condition is spreading from the first location;

using the three-dimensional representation of the assigned area, the received data from the one or more building sensors, and the determination of the hazardous condition spreading, generate spatially-resolved suggestions for an action plan that correspond to locations within the three-dimensional representation; and

displaying on a user interface, an augmented reality view comprising the three-dimensional representation of the assigned area with the generated suggestions for an action plan overlaid at corresponding locations within the representation.

2 . The method of claim 1 , wherein the self-propelled monitoring device comprises a drone.

3 . The method of claim 1 , further comprising displaying sensor data on the three-dimensional representation.

4 . The method of claim 1 , wherein the image data comprises at least one of real-time video and one or more still images.

5 . The method of claim 1 , further comprising receiving information specific to the assigned area, the information comprising a floor plan, and information regarding whether hazardous materials are stored in the assigned area.

6 . The method of claim 1 , further comprising transmitting the augmented view to a monitoring computer for display.

7 . An augmented reality system, the system comprising:

a first computer configured to receive information regarding a monitored structure, the information comprising a building layout;

one or more building sensors; and

a self-propelled monitoring device communicatively coupled to the first computer, wherein the self-propelled monitoring device is configured to:

receive, from the first computer, instructions to monitor an assigned area associated with the monitored structure;

in response to the instructions, travel to the assigned area;

capture image data of the assigned area and transmit the captured image data to the first computer;

wherein the first computer comprises software instructions that, when executed, cause the first computer to:

create a three-dimensional representation of the assigned area from one of real-time or near real-time image data captured by the self-propelled monitoring device while the device is located in the assigned area and the building layout;

receive sensor data directly from one or more of the building sensors, the sensor data comprising occupancy data identifying persons within the assigned area and condition data indicative of a hazardous condition;

determine that a hazardous condition exists at least at a first location in the assigned area;

determine whether the hazardous condition is spreading from the first location based on sensor data from multiple sensor locations within the assigned area;

generate suggestions for an action plan that are spatially associated with corresponding locations in the three-dimensional representation using the three-dimensional representation, the received sensor data, and the determination of the hazardous condition spreading; and

display, on a user interface, an augmented reality view comprising the three-dimensional representation with the generated suggestions overlaid thereon.

8 . The system of claim 7 , wherein the first computer is communicatively coupled to a server, and wherein the first computer is configured to receive layout information for the monitored structure from the server.

9 . The system of claim 7 , wherein the user interface also displays sensor data on the three-dimensional representation of the assigned area.

10 . The system of claim 7 , wherein the one or more building sensors are configured to broadcast the sensor data as a signal at regular intervals, and wherein the signal comprises identification information for each of the one or more building sensors and data indicative a sensed condition.

11 . The system of claim 7 , wherein the self-propelled monitoring device comprises a drone.

12 . The system of claim 7 , wherein the image data comprises at least one of a real-time video and one or more still images.

13 . The system of claim 7 , wherein the information regarding the monitored structure further comprises a floor plan and information regarding whether hazardous materials are stored in the assigned area.

14 . The system of claim 7 , wherein the first computer transmits the three-dimensional representation of the assigned area and the generated suggestions for an action plan to a monitoring computer that is located remotely from the first computer.

15 . A method of monitoring a structure with an augmented reality system, the method comprising:

receiving at a self-propelled monitoring device, instructions to monitor an assigned area associated with a monitored structure;

traveling, by the self-propelled monitoring device, to the assigned area;

receiving, by the self-propelled monitoring device, image data, the image data comprising one of a real-time or a near real-time video of the assigned area;

generating a three-dimensional representation of the assigned from the captured image data while the self-propelled monitoring device is present in the assigned area;

receiving, by the self-propelled monitoring device, sensor data directly from one or more building sensors associated with the assigned area, the sensor data comprising both occupancy data identifying persons within the assigned area and condition data indicative of a hazardous condition;

receiving a floor plan and information regarding whether hazardous materials are stored in the assigned area;

determining that a hazardous condition exists at least at a first location in the assigned area;

determining, based on sensor data from multiple locations within the assigned area, whether the hazardous condition is spreading from the first location;

generate spatially-resolved suggestions for an action plan to address the hazardous condition using the three-dimensional representation of the assigned area, the received data from the one or more building sensors, the floor plan, information regarding whether hazardous materials are stored in the assigned area, and the determination of the hazardous condition is spreading; and

displaying an augmented reality view comprising the three-dimensional representation of the assigned area, the display comprising sensor data and the generated suggestions for an action plan overlaid at corresponding locations within the representation on a user interface; and

transmitting the augmented view to a monitoring computer for display.