User interface and related flow for controlling a robotic arm

1. A method, comprising:

displaying, via a user interface (UI), a first page on a first pane, wherein the first page provides a first control input for registering a working frame of a target object with a reference frame of a robot;

receiving, via the UI, a first user selection of the first control input for registering the working frame with the reference frame;

displaying, via the UI, a second page on the first pane, wherein the second page provides a second control input for generating a path for the robot to traverse over a surface of the target object, wherein the path comprises virtual points;

receiving, via the UI, a second user selection of the second control input for generating the path;

detecting a modification of a virtual point, wherein the modification comprises adding the virtual point to the virtual points, moving the virtual point, or deleting the virtual point from the virtual points;

displaying, via the UI, the modification on the second pane; and

receiving an indication to regenerate the path based on the modification.

2. The method of claim 1 , wherein the method further comprises displaying the path on a second pane, wherein the path further comprises a respective directional marker between pairs of virtual points, and wherein the path is based on the first control input and the second control input.

3. The method of claim 1 , wherein the method further comprises:

displaying a third page on the first pane, wherein the third page provides a third control input for identifying objects in a point cloud representation of the target object; and

receiving a third user selection of the third control input for identifying objects in the point cloud representation of the target object.

4. The method of claim 1 , wherein the method further comprises:

displaying the second page on the first pane, wherein the second page provides a fourth control input for generating a trajectory comprising robot joint parameters used to traverse the surface of the target object along the path; and

receiving a fourth user selection of the fourth control input for generating the trajectory.

5. The method of claim 1 , wherein the method further comprises:

displaying a fourth page on the first pane, wherein the fourth page provides a fifth control input for determining a class of a real-world target using a machine learning model; and

receiving a fifth user selection of the fifth control input for determining the class of the real-world target using the machine learning model.

6. The method of claim 1 , wherein the method further comprises displaying a real-time scanning of the target object by the robot on a third pane, wherein the robot follows a trajectory based on the path.

7. The method of claim 1 , wherein the UI is in operable communication with an application executing on a computing device, and wherein the application is in operable communication with the robot.

8. The method of claim 1 , wherein the method further comprises displaying, on a third pane, a first visual marker indicating that the robot has identified a real-world target in real time.

9. The method of claim 1 , wherein the method further comprises displaying a first field for receiving a first input parameter for guiding generation of the path.

10. A system comprising:

a user interface (UI);

one or more processors; and

a computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

displaying, via the UI, a first page on a first pane, wherein the first page provides a first control input for registering a working frame of a target object with a reference frame of a robot;

receiving, via the UI, a first user selection of the first control input for registering the working frame with the reference frame;

displaying, via the UI, a second page on the first pane, wherein the second page provides a second control input for generating a path for the robot to traverse over a surface of the target object, wherein the path comprises virtual points;

receiving, via the UI, a second user selection of the second control input for generating the path;

detecting a modification of a virtual point, wherein the modification comprises adding the virtual point to the virtual points, moving the virtual point, or deleting the virtual point from the virtual points;

displaying, via the UI, the modification on the second pane; and

receiving an indication to regenerate the path based on the modification.

11. The system of claim 10 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising displaying the path on a second pane, wherein the path further comprises a respective directional marker between pairs of virtual points, and wherein the path is based on the first control input and the second control input.

12. The system of claim 10 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

displaying a third page on the first pane, wherein the third page provides a third control input for identifying objects in a point cloud representation of the target object; and

receiving a third user selection of the third control input for identifying objects in the point cloud representation of the target object.

13. The system of claim 10 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

displaying the second page on the first pane, wherein the second page provides a fourth control input for generating a trajectory comprising robot joint parameters used to traverse the surface of the target object along the path; and

receiving a fourth user selection of the fourth control input for generating the trajectory.

14. The system of claim 10 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

displaying a fourth page on the first pane, wherein the fourth page provides a fifth control input for determining a class of a real-world target using a machine learning model; and

receiving a fifth user selection of the fifth control input for determining the class of the real-world target using the machine learning model.

15. A computer-readable medium including one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

displaying, via a UI, a first page on a first pane, wherein the first page provides a first control input for registering a working frame of a target object with a reference frame of a robot;

receiving, via the UI, a first user selection of the first control input for registering the working frame with the reference frame;

displaying, via the UI, a second page on the first pane, wherein the second page provides a second control input for generating a path for the robot to traverse over a surface of the target object, wherein the path comprises virtual points;

receiving, via the UI, a second user selection of the second control input for generating the path;

detecting a modification of a virtual point, wherein the modification comprises adding the virtual point to the virtual points, moving the virtual point, or deleting the virtual point from the virtual points;

displaying, via the UI, the modification on the second pane; and

receiving an indication to regenerate the path based on the modification.

16. The computer-readable medium of claim 15 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising displaying the path on a second pane, wherein the path further comprises a respective directional marker between pairs of virtual points, and wherein the path is based on the first control input and the second control input.

17. The computer-readable medium of claim 15 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

displaying a third page on the first pane, wherein the third page provides a third control input for identifying objects in a point cloud representation of the target object; and

receiving a third user selection of the third control input for identifying objects in the point cloud representation of the target object.

18. The computer-readable medium of claim 15 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

displaying the second page on the first pane, wherein the second page provides a fourth control input for generating a trajectory comprising robot joint parameters used to traverse the surface of the target object along the path; and

receiving a fourth user selection of the fourth control input for generating the trajectory.