Punch-in / punch-out smoothing for takeover of automated movement with haptic controller
View Patent ↗Example implementations relate to punch-in transitions to smoothly transition operation of parts of a robotic system from control tracks to real-time input, such as to smoothly transition a control track rotating a knob at a handheld controller to real-time input received at the handheld controller. Similarly, example implementations relate to punch-out transitions to smoothly transition operation of parts of a robotic system from real-time input to control tracks, such as to smoothly transition from the real-time input received at a handheld controller to a control track rotating a knob at a handheld controller. In particular, an example system may include a handheld haptic controller, a control system, and a robotic component.
1. A robotic system, comprising:
a handheld controller, comprising:
a rotatable knob coupled to a base;
at least one motor that is operable to apply a torque-generating force to the rotatable knob;
one or more touch sensors arranged to sense touch input on a curved surface of the rotatable knob; and
a control system configured to:
output a robot control signal based on playback of at least one pre-defined robot control track on a robotic system component;
generate a handheld controller control signal that controls torque applied to the rotatable knob based on the at least one pre-defined robot control track;
cause the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal;
detect a first touch input received from the one or more touch sensors; and
in response to detecting the first touch input, transition from: (a) causing the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal and outputting robot control signals based on playback of at least one pre-defined robot control track, to (b) receiving real-time input from the handheld controller and outputting a second control signal to operate robotic system component based on the real-time input from the handheld controller,
wherein the transition comprises application of a time-varying weighting function to the pre-defined robot control track and a rotational input from the real-time input, wherein the time-varying weighting function decreases weight applied to the pre-defined robot control track and increases weight applied to the rotational input over time.
2. The robotic system of claim 1 , wherein, in response to receiving the touch input, the control system is further configured to:
transition operation of the robotic system component from the robot control signals to real-time input received from the handheld controller.
3. The robotic system of claim 2 , wherein the touch input indicates a touch on the curved surface of the handheld controller.
4. The robotic system of claim 2 , wherein the one or more touch sensors comprise at least one proximity sensor, and wherein the touch input indicates a proximity to the at least one proximity sensors.
5. The robotic system of claim 1 , wherein the control system is further configured to:
detect second touch input on the handheld controller; and
in response to detecting the second touch input, transition operation of the handheld controller from real-time input received from the handheld controller to the handheld controller control signal.
6. The robotic system of claim 5 , wherein, in response to detecting the second touch input, the control system is further configured to:
transition operation of the robotic system component from real-time input received from the handheld controller to the robot control signals.
7. The robotic system of claim 5 , wherein the second touch input indicates a disengagement of the curved surface of the handheld controller.
8. The robotic system of claim 5 , wherein the one or more touch sensors comprise at least one proximity sensor, and wherein the second touch input indicates a disengagement of the at least one proximity sensor.
9. A method for a robotic system, comprising:
outputting a robot control signal based on playback of at least one pre-defined robot control track on a robotic system component;
generating a handheld controller control signal for a handheld controller comprising a rotatable knob coupled to a base, at least one motor operable to apply torque-generating force to the rotatable knob, and one or more touch sensors arranged to sense touch data on a curved surface of the rotatable knob, wherein the handheld controller control signal controls torque applied to the rotatable knob based on the at least one pre-defined robot control track;
causing the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal;
detecting first touch input based on touch data from the one or more touch sensors; and
in response to detecting the first touch input, transitioning from: (a) causing the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal and output robot control signals based on playback of at least one pre-defined robot control track, to (b) receiving real-time input from the handheld controller and outputting a second control signal to operate robotic system component based on the real-time input from the handheld controller
wherein transitioning comprises, during a transition period, applying a time-varying weighting function to the pre-defined robot control track and a rotational input from the real-time input, wherein the time-varying weighting function decreases weight applied to the pre-defined robot control track and increases weight applied to the rotational input over the transition period.
10. The method of claim 9 , further comprising:
in response to detecting the first touch input, transitioning operation of at least one robotic component of the robotic system from the robot control signals to real-time input received from the handheld controller.
11. The method of claim 9 , wherein the first touch input indicates a user touch on the curved surface.
12. The method of claim 9 , further comprising:
detecting second touch input on the handheld controller;
in response to detecting the second touch input, transitioning operation of the at least one robotic component of the robotic system from real-time input received from the handheld controller to the robot control signals.
13. The method of claim 12 , further comprising:
in response to detecting the second touch input, transitioning operation of the handheld controller from the handheld controller control signal to real-time input received from the handheld controller, wherein the second touch input indicates a disengagement from the curved surface.
14. The method of claim 9 , wherein transitioning operation of the handheld controller further comprises:
storing transition options on a control system, wherein the transition options correspond to pre-programmed takeover transitions or pre-programmed release transitions, and wherein the transition options are based at least in part on corresponding predicted takeover proximity data or predicted release proximity data;
detecting, by the control system, proximity data received from the handheld controller;
comparing, by the control system, proximity data received from the handheld controller to predicted proximity data; and
in response to receipt of proximity data that matches the predicted proximity data, transitioning operation of the handheld controller, based at least in part on the one or more transition options, from the handheld controller control signal to real-time input received from the handheld controller.
15. A robotic system, comprising:
a handheld controller, comprising:
a rotatable knob coupled to a base;
at least one motor that is operable to apply a torque-generating force to the rotatable knob;
one or more touch sensors arranged to sense touch data on a curved surface of the rotatable knob; and
a control system configured to:
output a robot control signal based on playback of at least one pre-defined robot control track on a robotic system component;
store one or more transition options, wherein the one or more transition options correspond to pre-programmed takeover transitions or pre-programmed release transitions, or both, and wherein the one or more transition options are based at least in part on corresponding expected takeover proximity data or expected release proximity data;
generate a handheld controller control signal that controls torque applied to the rotatable knob based on the at least one pre-defined robot control track;
cause the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal;
detect proximity data received from the one or more proximity sensors;
compare received proximity data to expected takeover proximity data; and
in response to receipt of proximity data that matches the expected takeover proximity data, transition, based at least in part on the one or more transition options, from (a) causing the handheld controller to apply torque-generating force to the rotatable knob based on the handheld controller control signal to (b) receiving real-time input from the handheld controller and outputting a second control signal to operate robotic system component based on the real-time input from the handheld controller,
wherein one of the transition options comprise specifies a transition period, and application of a time-varying weighting function to the pre-defined robot control track and a rotational input from the real-time input, wherein the time-varying weighting function decreases weight applied to the pre-defined robot control track and increases weight applied to the rotational input over the transition period.
16. The robotic system of claim 15 , wherein the control system is further configured to:
output robot control signals to a robotic system component; and
in response to receipt of proximity data that matches the expected takeover proximity data, transition operation of the robotic system component, based at least in part on the one or more transition options, from the one or more robot control signals to real-time input received from the handheld controller.
17. The robotic system of claim 16 , wherein the control system is further configured to:
detect additional proximity data;
compare additional proximity data to expected release proximity data; and
in response to a determination that the additional proximity data matches the expected release proximity data, transition operation of the handheld controller from the real-time input received from the handheld controller to the handheld controller control signal.
18. The robotic system of claim 17 , wherein, when additional proximity data matches expected release proximity data, the control system is further configured to transition operation of the robotic system component from the real-time input received from the handheld controller to the one or more robot control signals.
19. The robotic system of claim 18 , wherein the robotic system component comprises an actuator corresponding to a particular joint of the robotic system.
20. The robotic system of claim 19 , wherein the operation of the robotic component comprises movement of the particular joint based on the rotation of the knob.