Master-slave teleoperation robot system based on force mixed reality
The invention pertains to a master-slave teleoperation robot system utilizing force mixed reality, relevant to teleoperation robot technology. The system consists of a master robot linked to a mixed reality device, which receives point cloud information to create a virtual environment. The slave robot features a depth camera that captures point cloud data from the real environment, transmitting it to the master robot in real time. This point cloud information generates a virtual fixture, which produces virtual force applied to the slave robot. The virtual force interacts with the actual environmental forces acting on the slave robot, guiding the operator's behavior at the master side. The system addresses incorrect occlusion between the real dynamic robot and the virtual scene by blending virtual and real elements. Through mixed reality with force guidance and feedback, it minimizes the risk of potentially harmful collisions during operations.
1 . A master-slave teleoperation robot system based on force mixed reality, comprising a master robot and a slave robot, wherein the master robot is connected with a mixed reality device, which is used to receive point cloud information of a real environment and produce a virtual environment based on the point cloud information in the mixed reality device; an end of the slave robot is equipped with a depth camera, the depth camera is used to collect the point cloud information and send the point cloud information to the mixed reality device in real time, the point cloud information is also used to produce a virtual fixture, the virtual fixture is used to produce a virtual force which is applied to the slave robot, the virtual force comprises a guiding force and a prohibiting force, the virtual force interacts with an actual environment interaction force at a slave side to guide and norm an operator's behavior at a master side; wherein the mixed reality device projects a digital twin of the slave robot to a position coinciding with the master robot, and motions of the slave robot and the master robot are synchronized such that joint angles of the digital twin and the master robot are synchronized, thereby eliminating incorrect occlusion relationships between the master robot and the virtual environment projected in the mixed reality device.
2 . The master-slave teleoperation robot system based on force mixed reality according to claim 1 , wherein the virtual fixture comprises a forbidden-region virtual fixture (FRVF) and a guidance virtual fixture (GVF), through an artificial potential field, the point cloud information of a prohibited area of the real environment is collected to produce the FRVF; the GVF is a human-robot interaction motion path that the operator wants to achieve.
3 . The master-slave teleoperation robot system based on force mixed reality according to claim 1 , wherein the point cloud information is a three-dimensional information of the real environment collected by the depth camera, after the point cloud information is transformed to a robot base coordinate system, the point cloud information is sent to the master robot in real time, the master robot and the slave robot are in an isomorphic master-slave configuration, therefore, in the mixed reality device, the point cloud information is directly projected to a corresponding position in a base coordinate of the master robot.
4 . The master-slave teleoperation robot system based on force mixed reality according to claim 3 , wherein a coordinate system of the master-slave teleoperation robot system comprises a base coordinate system of the master robot, a base coordinate system of the slave robot, a coordinate system of the depth camera mounted on the slave robot and a coordinate system of an Optical See-Through Head-Mounted Display (OST-HMD).