Robotic camera control via motion capture
A motion capture setup records the movements of an operator, and a control engine then translates those movements into control signals for controlling a robot. The control engine may directly translate the operator movements into analogous movements to be performed by the robot, or the control engine may compute robot dynamics that cause a portion of the robot to mimic a corresponding portion of the operator.
1. A computer-implemented method for controlling a robot to generate multi-media content, the method comprising:
generating, via a first set of one or more sensors, motion capture data based on one or more movements of an operator;
processing the motion capture data to generate control signals for controlling the robot, wherein a second set of one or more sensors is coupled to the robot;
transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator; and
generating content associated with the one or more movements of the operator based on a plurality of image frames of the robot that are captured by the second set of one or more sensors.
2. The computer-implemented method of claim 1 , wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via the first set of one more sensors.
3. The computer-implemented method of claim 1 , wherein processing the motion capture data to generate the control signals comprises:
modeling a first set of dynamics of the operator that corresponds to the one or more movements;
determining a second set of dynamics of the robot that corresponds to the one or more movements; and
generating the control signals based on the second set of dynamics.
4. The computer-implemented method of claim 1 , wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements.
5. The computer-implemented method of claim 4 , wherein the robot comprises a robotic arm that includes the set of joints.
6. The computer-implemented method of claim 1 , wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements.
7. The computer-implemented method of claim 1 , further comprising capturing sensor data while the robot mimics the one or more movements.
8. The computer-implemented method of claim 1 , wherein the one or more movements comprise a target trajectory for the second set of one or more sensors coupled to the robot.
9. One or more non-transitory computer-readable media that, when executed by one or more processors, causes the one or more processors to control a robot in order to generate multi-media content, by performing the steps of:
generating, via a first set of one or more sensors, motion capture data based on one or more movements of an operator;
processing the motion capture data to generate control signals for controlling the robot, wherein a second set of one or more sensors is coupled to the robot;
transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator; and
generating content associated with the one or more movements of the operator based on a plurality of image frames of the robot that are captured by the second set of one or more sensors.
10. The one or more non-transitory computer-readable media of claim 9 , wherein generating the motion capture data comprises tracking at least one of head, body, and limb movements associated with the operator via the first set of one more sensors.
11. The one or more non-transitory computer-readable media of claim 9 , wherein processing the motion capture data to generate the control signals comprises:
modeling a first set of dynamics of the operator that corresponds to the one or more movements;
determining a second set of dynamics of the robot that corresponds to the one or more movements; and
generating the control signals based on the second set of dynamics.
12. The one or more non-transitory computer-readable media of claim 9 , wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements.
13. The one or more non-transitory computer-readable media of claim 9 , wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a subset of the one or more movements.
14. The one or more non-transitory computer-readable media of claim 9 , further comprising capturing sensor data while the robot mimics the one or more movements.
15. The one or more non-transitory computer-readable media of claim 9 , wherein the one or more movements comprise a target trajectory for the second set of one or more sensors coupled to the robot.
16. A system for controlling a robot to generate multi-media content, comprising:
a memory storing a control application; and
a processor coupled to the memory and configured to:
generate, via a first set of one or more sensors, motion capture data based on one or more movements of an operator,
process the motion capture data to generate control signals for controlling the robot, wherein a second set of one or more sensors is coupled to the robot,
transmit the control signals to the robot to cause the robot to mimic the one or more movements of the operator, and
generate content associated with the one or more movements of the operator based on a plurality of image frames of the robot that are captured by the second set of one or more sensors.
17. The system of claim 16 , wherein the processor is configured to execute the control application to:
generate the motion capture data;
process the motion capture data to generate the control signals; and
transmit the control signals to the robot.