Part modeling for path generation to guide robotic end effector
Aspects of the disclosure are directed towards path generation. A method includes a device registering working frame of a target object with a reference frame of a robot. The device can generate a path over a representation of a surface of the target object. The device can generate a trajectory over the surface of the target object based on the registration, the path, and a normal. The device can classify a target type for the real-world target using a machine learning model based on scanned data of the surface of the target object. The device can generate a robot job file, wherein the robot job file comprises the trajectory and an autonomous operation instruction. The device can transmit the robot job file to a robot controller.
1. A method, comprising:
registering, by a device, a working frame of a target object with a reference frame of a robot;
generating, by the device, a path over a representation of a surface of the target object, wherein the path comprises a virtual point and a normal at the virtual point, and wherein the virtual point corresponds to a real-world target;
generating, by the device, a trajectory over the surface of the target object based on the registration, the path, and the normal, wherein the trajectory comprises a set of robot joint parameters used to traverse the surface of the target object, and wherein the trajectory traverses the real-world target;
classifying, by the device, a target type for the real-world target using a machine learning model based on scanned data of the surface of the target object;
generating, by the device, a robot job file, wherein the robot job file comprises the trajectory and an autonomous operation instruction, and
transmitting, by the device, the robot job file to a robot controller.
2. The method of claim 1 , wherein the method further comprises:
clustering or segmenting data points of a point cloud representation of the target object to identify an object;
determining whether a cluster respectively represents a real-world target based on identifying a feature associated with the real-world target; and
calculating the normal for the cluster representing the real-world target based on the determination.
3. The method of claim 2 , wherein the method further comprises:
retrieving collision data comprising data of a first collision between the robot and the target object;
simulating a first candidate trajectory and a second candidate trajectory;
determining that the first candidate trajectory results in a second collision based on the collision data; and
selecting the second candidate trajectory as the trajectory based on the determination.
4. The method of claim 1 , wherein generating the trajectory is based on inverse kinematics.
5. The method of claim 1 , wherein classifying the target type using a machine learning model comprises:
receiving scanned data of the surface of the target object;
inputting the scanned data into the machine learning model, wherein the machine learning model is trained to identify a target object class; and
executing the machine learning model to classify the real-world target.
6. The method of claim 1 , wherein the method further comprises:
receiving an indication that a position of the virtual point in relation to the target object has been modified; and
generating a second path based on the modification of the position of the virtual point.
7. The method of claim 1 , wherein generating the path comprises using an optimization technique to identify a minimum length for traversing the surface of the target object.
8. A device comprising:
one or more processors;
a computer readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
registering a working frame of a target object with a reference frame of a robot;
generating a path over a representation of a surface of the target object, wherein the path comprises a virtual point and a normal at the virtual point, and wherein the virtual point corresponds to a real-world target;
generating a trajectory over the surface of the target object based on the registration, the path, and the normal, wherein the trajectory comprises a set of robot joint parameters used to traverse the surface of the target object, and wherein the trajectory traverses the real-world target;
classifying a target type for the real-world target using a machine learning model based on scanned data of the surface of the target object;
generating a robot job file, wherein the robot job file comprises the trajectory and an autonomous operation instruction, and
transmitting the robot job file to a robot controller.
9. The device of claim 8 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
clustering or segmenting data points of a point cloud representation of the target object to identify an object;
determining whether a cluster respectively represents a real-world target based on identifying a feature associated with the real-world target; and
calculating the normal for the cluster representing the real-world target based on the determination.
10. The device of claim 9 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
retrieving collision data comprising data of a first collision between the robot and the target object;
simulating a first candidate trajectory and a second candidate trajectory;
determining that the first candidate trajectory results in a second collision based on the collision data; and
selecting the second candidate trajectory as the trajectory based on the determination.
11. The device of claim 8 , wherein generating the trajectory is based on inverse kinematics.
12. The device of claim 8 , wherein classifying the target type using a machine learning model comprises:
receiving scanned data of the surface of the target object;
inputting the scanned data into the machine learning model, wherein the machine learning model is trained to identify a target object class; and
executing the machine learning model to classify the real-world target.
13. The device of claim 8 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
receiving an indication that a position of the virtual point in relation to the target object has been modified; and
generating a second path based on the modification of the position of the virtual point.
14. The device of claim 8 , wherein generating the path comprises using an optimization technique to identify a minimum length for traversing the surface of the target object.
15. A computer-readable medium including one or more sequences of instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising:
registering a working frame of a target object with a reference frame of a robot;
generating a path over a representation of a surface of the target object, wherein the path comprises a virtual point and a normal at the virtual point, and wherein the virtual point corresponds to a real-world target;
generating a trajectory over the surface of the target object based on the registration, the path, and the normal, wherein the trajectory comprises a set of robot joint parameters used to traverse the surface of the target object, and wherein the trajectory traverses the real-world target;
classifying a target type for the real-world target using a machine learning model based on scanned data of the surface of the target object;
generating a robot job file, wherein the robot job file comprises the trajectory and an autonomous operation instruction, and
transmitting the robot job file to a robot controller.
16. The computer-readable medium of claim 15 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
clustering or segmenting data points of a point cloud representation of the target object to identify an object;
determining whether a cluster respectively represents a real-world target based on identifying a feature associated with the real-world target; and
calculating the normal for the cluster representing the real-world target based on the determination.
17. The computer-readable medium of claim 16 , wherein the instructions that, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:
retrieving collision data comprising data of a first collision between the robot and the target object;
simulating a first candidate trajectory and a second candidate trajectory;
determining that the first candidate trajectory results in a second collision based on the collision data; and
selecting the second candidate trajectory as the trajectory based on the determination.
18. The computer-readable medium of claim 15 , wherein generating the trajectory is based on inverse kinematics.
19. The computer-readable medium of claim 15 , wherein classifying the target type using a machine learning model comprises:
receiving scanned data of the surface of the target object;
inputting the scanned data into the machine learning model, wherein the machine learning model is trained to identify a target object class; and
executing the machine learning model to classify the real-world target.
20. The computer-readable medium of claim 15 , wherein the operations further comprise:
receiving an indication that a position of the virtual point in relation to the target object has been modified; and
generating a second path based on the modification of the position of the virtual point.