METHOD AND SYSTEM FOR ROBOT CONTROL USING VISUAL FEEDBACK

1 . A method for system control, wherein the system comprises an end-effector connected to a robot, comprising:

determining a generative model comprising a set of variables and a set of parameters, wherein the set of variables comprises observed variables and hidden variables, and wherein the set of parameters comprises camera parameters and structure parameters;

training the generative model, comprising:

sampling a set of images, wherein each image depicts the robot, the end-effector, and a feature rigidly connected to the end effector;

determining feature pixel locations of the feature detected in the set of images;

initializing the camera parameters;

determining an action for the robot, wherein the action is a variable of the set of variables;

calculating optimized camera parameters, optimized structure parameters, and an end-effector pose based on the initialized camera parameters, the action, and the feature pixel locations, wherein the end-effector pose is a hidden variable of the set of variables;

and

controlling the robot using the trained generative model.