Method and system for autonomous body interaction
A system, method, and apparatus for autonomous body interaction system is provided. A system and method for a robot interacting with a human or other soft or deformable body is provided for the purpose of active application of pressure to specific points or regions of the body. The system and method conducts at least one of: localizing the position of the body, detecting the configuration of the body, identifying the surface region(s) of the body, predicting the underlying anatomy of the body, assessing the state of the body, preparing a plan for manipulation of the body, and executing a predetermined and/or dynamically updated plan on the body.
1. A system, comprising:
a support structure;
a robotic manipulator including an end effector;
a controller operably coupled to the robotic manipulator and configured to control an interaction between the end effector and a deformable body when the deformable body is positioned on the support structure; and
a plurality of sensors positioned to detect, during use, multi-dimensional data associated with the deformable body when the deformable body is positioned on the support structure,
the controller configured to:
determine, based on the multi-dimensional data, at least one of a position, an orientation, or a shape of the deformable body when the deformable body is positioned on the support structure;
generate a body model for the deformable body based on the at least one of the position, the orientation, or the shape of the deformable body;
generate an interaction plan based on the body model, the interaction plan comprising a time sequence of interaction goals, wherein planned contact of the end effector with the deformable body is defined by the time sequence of interaction goals, and wherein the time sequence of interaction goals defines a trajectory associated with at least a portion of a stroke; and
cause the robotic manipulator to execute the interaction plan comprising the time sequence of interaction goals, wherein individual interaction goals are sent to the controller based at least in part on a time indicated with each interaction goal.
2. The system of claim 1 , wherein the controller is further configured to:
perform tracking of a movement of the deformable body during execution of the interaction plan; and
modify the interaction plan based on the tracking, to at least one of maintain contact with the deformable body, or position the end effector relative to a predefined part of the deformable body.
3. The system of claim 1 , wherein the interaction plan specifies at least one overlay, the at least one overlay including at least one of a variation of a position goal or a variation of a force goal.
4. The system of claim 1 , further comprising assigning labels to the multi-dimensional data using a deep learning model.
5. The system of claim 1 , wherein the deformable body is a human body and the planned contact between the end effector and the deformable body includes contact sufficient to at least one of displace, compress, or mobilize muscle tissue within the human body.
6. The system of claim 1 , wherein the body model is a current body model, the generating the current body model including modifying a historical body model based on the multi-dimensional data.
7. The system of claim 1 , wherein the interaction plan is a current interaction plan, and the generating the current interaction plan includes adjusting a historical interaction plan based on the multi-dimensional data.
8. The system of claim 1 , wherein the multi-dimensional data is a first multi-dimensional data, the controller further configured to:
determine second multi-dimensional data based on signals received from the plurality of sensors during the execution of the interaction plan; and
modify the interaction plan during the execution of the interaction plan based on the second multi-dimensional data.
9. The system of claim 8 , wherein modifying the interaction plan includes at least one of modifying a pressure of the interaction plan or modifying a frequency of touch of the interaction plan.
10. The system of claim 1 , wherein the deformable body is a human body and the support structure includes stirrups configured to accommodate feet of the human body when the deformable body is positioned on the support structure.
11. The system of claim 1 , wherein the deformable body is a human body and the support structure includes handles configured to support hands of the human body when the deformable body is positioned on the support structure.
12. A method, comprising:
receiving, at a processor and from a plurality of sensors, sensor data associated with a deformable body in a prone position;
determining, via the processor and based on the sensor data, at least one of a position, an orientation, or a shape of the deformable body in the prone position;
generating, via the processor, a body model for the deformable body based on the at least one of the position, the orientation, or the shape of the deformable body;
generating, via the processor, an interaction plan based on the body model, the interaction plan comprising a time sequence of interaction goals, wherein planned contact of an end effector of a robotic manipulator with the deformable body is defined by the time sequence of interaction goals, and wherein the time sequence of interaction goals defines a trajectory associated with at least a portion of a stroke; and
causing the robotic manipulator to execute the interaction plan comprising the time sequence of interaction goals, wherein individual interaction goals are sent to a controller operably coupled to the robotic manipulator based at least in part on a time indicated with each interaction goal.
13. The method of claim 12 , further comprising:
receiving, at the processor and during the execution of the interaction plan, a representation of a user-selected adjustment, the user-selected adjustment including one of: a command to skip a segment of the interaction plan, a command to repeat a segment of the interaction plan, a command to extend a duration of a segment of the interaction plan, or a command to shorten the duration of the segment of the interaction plan.
14. The method of claim 12 , wherein the interaction plan is a current interaction plan, and the generating the current interaction plan includes adjusting a past interaction plan based on the sensor data.
15. The method of claim 12 , further comprising estimating a subcutaneous anatomy of the deformable body, wherein the generating the interaction plan is further based on the estimated subcutaneous anatomy of the deformable body.
16. The method of claim 12 , wherein the plurality of sensors includes at least one camera.
17. The method of claim 12 , wherein the sensor data includes at least one of thermographic imagery, visible light imagery, infrared imagery, or three-dimensional range data.
18. The method of claim 12 , wherein the end effector is configured to mimic at least one of a form or a function of a human hand.
19. The method of claim 12 , wherein the end effector is configured to mimic at least one of a form or a function of a human finger.
20. The method of claim 12 , wherein the end effector is configured to mimic at least one of a form or a function of a human forearm.
21. The method of claim 12 , wherein the interaction plan includes a plurality of predefined stroke segments.
22. The method of claim 12 , wherein the causing the robotic manipulator to execute the interaction plan is in response to an input received via a user interface.
23. The method of claim 12 , further comprising identifying a projected contact point of the end effector relative to the deformable body, based on a torque moment of an external force measurement.