Robotic control via a virtual world simulation
A system has a virtual-world (VW) controller and a physical-world (PW) controller. The pairing of a PW element with a VW element establishes them as corresponding physical and virtual twins. The VW controller and/or the PW controller receives measurements from one or more sensors characterizing aspects of the physical world, the VW controller generates the virtual twin, and the VW controller and/or the PW controller generates commands for one or more actuators affecting aspects of the physical world. To coordinate the corresponding virtual and physical twins, (i) the VW controller controls the virtual twin based on the physical twin or (ii) the PW controller controls the physical twin based on the virtual twin. Depending on the operating mode, one of the VW and PW controllers is a master controller, and the other is a slave controller, where the virtual and physical twins are both controlled based on one of VW or PW forces.
1 . A computer-implemented system comprising:
a computer-implemented physical-world (PW) master controller that controls motion of a PW master twin in a PW master world; and
a computer-implemented virtual-world (VW) slave controller that communicates with the PW master controller and controls motion of a VW slave twin in a VW slave world, wherein:
VW slave-world force data corresponding to a force originated in the VW slave world and applied to the VW slave twin in the VW slave world is provided to the PW master controller;
the PW master controller controls the motion of the PW master twin based on the VW slave-world force data;
PW master-world motion data corresponding to the motion of the PW master twin in the PW master world based on the VW slave-world force data is provided to the VW slave controller; and
the VW slave controller uses the PW master-world motion data based on the VW slave-world force data to control the motion of the VW slave twin in the VW slave world to match the motion of the PW master twin in the PW master world.
2 . The system of claim 1 , wherein:
the VW slave controller receives other PW master-world motion data corresponding to other motion of the PW master twin that is not based on a VW force originated in the VW slave world and applied to the VW slave twin in the VW slave world; and
the VW slave controller uses the other PW master-world motion data to control the motion of the VW slave twin to match the motion of the PW master twin.
3 . The system of claim 1 , wherein the PW controller controls motion of the physical twin such that collisions involving the physical twin are avoided in the physical world.
4 . The system of claim 1 , further comprising one or more sensors configured to generate sensor measurements corresponding to the physical world.
5 . The system of claim 4 , further comprising one or more actuators configured to move the physical twin.
6 . The system of claim 1 , further comprising one or more actuators configured to move the physical twin.
7 . The system of claim 1 , wherein the physical twin is a robot.
8 . The system of claim 1 , wherein the physical twin is not configured to be worn by a person.
9 . A computer-implemented method for controlling (i) motion of a physical-world (PW) master twin in a PW master world and (ii) motion of a virtual-world (VW) slave twin in a VW slave world, the method comprising one or more computers:
providing VW slave-world force data originated in the VW slave world and corresponding to a force applied to the VW slave twin in the VW slave world;
controlling the motion of the PW master twin based on the VW slave-world force data;
providing PW master-world motion data corresponding to the motion of the PW master twin in the PW master world based on the VW slave-world force data; and
using the PW master-world motion data based on the VW slave-world force data to control the motion of the VW slave twin in the VW slave world to match the motion of the PW master twin in the PW master world.
10 . The method of claim 9 , further comprising:
providing other PW master-world motion data corresponding to other motion of the PW master twin that is not based on a VW force originated in the VW slave world and applied to the VW slave twin in the VW slave world; and
using the other PW master-world motion data to control the motion of the VW slave twin to match the motion of the PW master twin.
11 . The method of claim 9 , wherein motion of the physical twin is controlled such that collisions involving the physical twin are avoided in the physical world.
12 . The method of claim 9 , wherein the physical twin is a robot.
13 . The method of claim 9 , wherein the physical twin is not worn by a person.