Personal protective equipment for navigation and map generation within a visually obscured environment
A system includes a personal protective equipment (PPE) configured to be worn by an agent. The PPE includes a sensor assembly comprising a radar device configured to generate radar data, a microphone configured to capture speech input from the agent, and an inertial measurement device configured to generate inertial data. The system includes a computing device configured to: process sensor data from the sensor assembly, the sensor data including at least the radar data and the inertial data, to generate pose data of the agent based on the processed sensor data, the pose data including a location and an orientation of the agent as a function of time, to process the speech input to identify an item of interest in an environment in which the PPE is deployed, and to form mapping information for the environment with the item of interest being marked, based on the processed speech input.
1. A system comprising:
a first personal protective equipment (PPE) configured to be worn by an agent, wherein the first PPE includes a sensor assembly comprising a radar device configured to generate radar data, a microphone configured to capture speech input from the agent, and an inertial measurement device configured to generate inertial data; and
at least one computing device comprising a memory and one or more processors coupled to the memory, wherein the at least one computing device is configured to:
process sensor data from the sensor assembly, wherein the sensor data includes at least the radar data and the inertial data;
generate pose data of the agent based on the processed sensor data, wherein the pose data includes a location and an orientation of the agent as a function of time;
process a first speech input to identify an item of interest in a hazardous environment in which the first PPE is deployed;
form mapping information for the hazardous environment with the item of interest being marked, based on the processed first speech input;
combine information obtained from the first speech input with the pose data of the agent within a space to discern information about surroundings of the agent, thereby representing a portion of the hazardous environment;
receive, from a second PPE of the system, a second speech input identifying the item of interest;
determine a location difference between the item of interest as identified in the first speech input and as identified in the second speech input; and
update the mapping information based on the determined location difference.
2. The system of claim 1 ,
wherein the PPE comprises a breathing apparatus that includes a wearable pack and headgear comprising a heads-up display (HUD),
wherein the microphone is integrated into the headgear, and
wherein the sensor assembly is integrated within at least one of a frame of the wearable pack, the headgear, or a handheld sensor.
3. The system of claim 1 , wherein the at least one computing device is further configured to output, for display via a heads-up display (HUD), a visual indication confirming receipt of the speech input.
4. The system of claim 1 , wherein to form the mapping information, the at least one computing device is configured to build a map of the hazardous environment.
5. The system of claim 1 , wherein to form the mapping information, the at least one computing device is configured to update an existing map of the hazardous environment.
6. The system of claim 1 , wherein the at least one computing device is further configured to determine a location of the item of interest based on one or both of distance information or directionality information provided in the speech input.
7. The system of claim 2 , wherein the at least one computing device is further configured to determine a location of the item of interest based on image data captured by an image capture hardware integrated into the HUD.
8. The system of claim 7 , wherein the image data represents a gesture performed by the agent.
9. The system of claim 1 , wherein the item of interest identified in the speech input comprises at least one of an egress point or a status of a hose.
10. A system comprising:
a first personal protective equipment (PPE) configured to be worn by an agent, wherein the first PPE includes a sensor assembly comprising a radar device configured to generate radar data, a microphone configured to capture speech input from the agent, and an inertial measurement device configured to generate inertial data; and
at least one computing device comprising a memory and one or more processors coupled to the memory, wherein the at least one computing device is configured to:
process sensor data from the sensor assembly, wherein the sensor data includes at least the radar data and the inertial data;
generate pose data of the agent based on the processed sensor data, wherein the pose data includes a location and an orientation of the agent as a function of time;
process a first speech input to identify a condition of interest in a hazardous environment in which the first PPE is deployed;
form mapping information for the hazardous environment with the condition of interest being marked, based on the processed first speech input;
combine information obtained from the first speech input with the pose data of the agent within a space to discern information about surroundings of the agent, thereby representing a portion of the hazardous environment;
receive, from a second PPE of the system, a second speech input identifying the condition of interest;
determine a location difference between the condition of interest as identified in the first speech input and as identified in the second speech input; and
update the mapping information based on the determined location difference.
11. The system of claim 10 ,
wherein the PPE comprises a breathing apparatus that includes a wearable pack and headgear comprising a heads-up display (HUD),
wherein the microphone is integrated into the headgear, and
wherein the sensor assembly is integrated within at least one of a frame of the wearable pack, the headgear, or a handheld sensor.
12. The system of claim 10 , wherein the at least one computing device is further configured to output, for display via a heads-up display (HUD), a visual indication confirming receipt of the speech input.
13. The system of claim 10 , wherein to form the mapping information, the at least one computing device is configured to build a map of the hazardous environment.