HUMAN-ROBOT COLLABORATION FOR 3D FUNCTIONAL MAPPING

1 . At least one machine readable medium including instructions, which when executed by processing circuitry, cause the processing circuitry to perform operations comprising:

receiving identification of a direction or location based on a user gaze identified via an extended reality device;

causing environmental data of an environment to be captured using a sensor of a robotic device, the environmental data corresponding to the direction or location based on receiving the identification;

detecting, within the environmental data, at least one physical feature of the environment;

determining, from a user input, an annotation to apply to the at least one physical feature; and

labeling the at least one physical feature with the annotation.

2 . The at least one machine readable medium of claim 1 , wherein the extended reality device is a head mounted display.

3 . The at least one machine readable medium of claim 1 , wherein the sensor is a video or image capture device and the environmental data includes a video or images.

4 . The at least one machine readable medium of claim 1 , wherein the at least one physical feature of the environment is a moveable object.

5 . The at least one machine readable medium of claim 1 , wherein the user input includes a spoken utterance, and wherein determining the annotation from the spoken utterance includes using natural language processing.

6 . The at least one machine readable medium of claim 1 , wherein labeling the at least one physical feature with the annotation includes displaying an indication of the annotation in the extended reality device overlaid on or near the at least one physical feature, while permitting at least a portion of the environment to be visible through the extended reality device.

7 . The at least one machine readable medium of claim 1 , wherein the operations further comprise:

determining voxel connectivity for captured portions of the environment; and

causing the robotic device to steer toward a region lacking point-sampling consistency in the voxel connectivity.

8 . The at least one machine readable medium of claim 7 , wherein causing the robotic device to steer includes using LIDAR.

9 . The at least one machine readable medium of claim 1 , wherein the sensor is registered to the extended reality device in seven dimensions including time.

10 . The at least one machine readable medium of claim 1 , wherein labeling the at least one physical feature with the annotation includes using a machine learning trained model to determine that the annotation applies to the at least one physical feature.

11 . The at least one machine readable medium of claim 10 , wherein the operations further comprise causing an indication to be displayed in the extended reality device, based on an output of the machine learning trained model, the indication suggesting that the annotation applies to the at least one physical feature, and receiving a user confirmation of the applicability of the annotation to the at least one physical feature.

12 . The at least one machine readable medium of claim 10 , wherein the machine learning trained model is selected from a plurality of models based on a geometry of the at least one physical feature or the annotation.

13 . A system comprising:

an extended reality device including a display, the extended reality device to output an indication of a direction or location based on a detected orientation or user gaze; and

a robotic device including:

a sensor to capture environmental data of an environment corresponding to the direction or location;

processing circuitry; and

memory, including instructions, which when executed by the processing circuitry, cause the processing circuitry to perform operations to:

receive an identification of the direction or location from the extended reality device;

detect, within the environmental data, at least one physical feature of the environment;

determine, from a user input, an annotation to apply to the at least one physical feature; and

label the at least one physical feature with the annotation.

14 . The system of claim 13 , wherein the extended reality device includes a head mounted display.

15 . The system of claim 13 , wherein the sensor includes a video or image capture device and the environmental data includes a video or images.

16 . The system of claim 13 , wherein the at least one physical feature of the environment is a moveable object.

17 . The system of claim 13 , wherein the user input includes a spoken utterance captured by a microphone of the extended reality device, and wherein determining the annotation from the spoken utterance includes using natural language processing.

18 . The system of claim 13 , wherein labeling the at least one physical feature with the annotation includes displaying an indication of the annotation, in the display of the extended reality device, overlaid on or near the at least one physical feature, while permitting at least a portion of the environment to be visible through the display.

19 . The system of claim 13 , wherein the operations further comprise:

determining voxel connectivity for captured portions of the environment; and

causing the robotic device to steer toward a region lacking point-sampling consistency in the voxel connectivity.

20 . An apparatus comprising:

means for receiving identification of a direction or location based on a user gaze identified via an extended reality device;

means for capturing environmental data of an environment, the environmental data corresponding to the direction or location based on receiving the identification;

means for detecting, within the environmental data, at least one physical feature of the environment;

means for determining, from a user input, an annotation to apply to the at least one physical feature; and

means for labeling the at least one physical feature with the annotation.