Controlling a robot during interaction with a human
Apparatuses, systems, and techniques to control a real-world and/or virtual device (e.g., a robot). In at least one embodiment, the device is controlled based, at least in part on, for example, one or more neural networks. Parameter values for the neural network(s) may be obtained by training the neural network(s) to control movement of a first agent with respect to at least one first target while avoiding collision with at least one stationary first holder of the at least one first target, and updating the parameter values by training the neural network(s) to control movement of a second agent with respect to at least one second target while avoiding collision with at least one non-stationary second holder of the at least one second target.
1 . A method comprising:
using, by a computer system, at least one demonstration to guide training at least one neural network to control movement of a first agent to complete at least one first task with respect to at least one first target and to avoid collision with at least one stationary first holder of the at least one first target; and
updating parameter values of the at least one neural network, by the computer system, by training the at least one neural network to control movement of a second agent to complete at least one second task with respect to at least one second target and to avoid collision with at least one non-stationary second holder of the at least one second target.
2 . The method of claim 1 , further comprising:
obtaining initial parameter values,
wherein the at least one demonstration is to be generated by at least one motion planner, and using the at least one demonstration to guide the training comprises repeatedly updating the initial parameter values based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained from the at least one demonstration, and the second set of transitions having been obtained from the at least one neural network using the initial parameter values.
3 . The method of claim 2 , wherein the at least one motion planner uses a set of pre-planned grasps from which any grasps that collide with the at least one stationary first holder has been removed.
4 . The method of claim 1 , wherein the parameter values before being updated are pre-trained parameter values, and updating the parameter values comprises:
repeatedly determining updated parameter values based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained from the at least one neural network using the pre-trained parameter values, and the second set of transitions having been obtained from the at least one neural network using the updated parameter values.
5 . The method of claim 1 , further comprising:
using, by the computing system, at least one other neural network to determine when to grasp the at least one second target held by the at least one non-stationary second holder.
6 . The method of claim 1 , wherein the at least one neural network receives as input data encoding the at least one second target and the at least one non-stationary second holder.
7 . The method of claim 1 , further comprising:
obtaining at least one point cloud encoding the at least one second target and the at least one non-stationary second holder from image data; and
providing the at least one point cloud to the at least one neural network as input.
8 . The method of claim 1 , wherein movement of the at least one non-stationary second holder of the at least one second target is simulated at least in part using motion capture data.
9 . The method of claim 1 , wherein the first and second agents each comprises at least one of an autonomous device, a semi-autonomous device, or a virtual device.
10 . The method of claim 1 , wherein the at least one stationary first holder and the at least one non-stationary second holder each comprises at least one of:
at least a portion of a living human being, or
at least a portion of a virtual character.
11 . A system comprising:
one or more processors to cause a first agent to move relative to at least one first target to be held by at least one moving first holder while avoiding colliding with the at least one moving first holder using one or more neural networks, the one or more neural networks to be trained, at least in part, by:
obtaining network parameters by using at least one demonstration to guide training the one or more neural networks to cause a second agent to move relative to at least one second target held by at least one non-moving second holder while avoiding colliding with the at least one non-moving second holder, and
modifying the network parameters by training the one or more neural networks to cause the second agent to move relative to at least one third target held by at least one moving third holder while avoiding colliding with the at least one moving third holder; and
one or more memories to store the one or more neural networks.
12 . The system of claim 11 , further comprising:
the first agent, which is to comprise at least one of an autonomous device or a semi-autonomous device.
13 . The system of claim 12 , wherein the at least one moving first holder comprises a living human being.
14 . The system of claim 12 , wherein the second agent is to comprise a virtual device,
the at least one non-moving second holder comprises at least a portion of a first virtual character, and
the at least one moving third holder comprises at least a portion of a second virtual character.
15 . The system of claim 11 , further comprising:
at least one image capture device positioned to capture image data representing the at least one first target and the at least one moving first holder, the one or more neural networks to be trained, at least in part, by obtaining at least one point cloud encoding the image data, and providing the at least one point cloud to the one or more neural networks as input.
16 . The system of claim 11 , wherein obtaining the network parameters by using the at least one demonstration to guide the training of the one or more neural networks comprises:
obtaining initial network parameters; and
repeatedly updating the initial network parameters based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained from the at least one demonstration, and the second set of transitions having been obtained from the one or more neural networks using the initial network parameters.
17 . The system of claim 16 , wherein the at least one motion planner uses a set of pre-planned grasps from which any grasps that collide with the at least one non-moving second holder have been removed.
18 . The system of claim 11 , wherein the network parameters before being modified are pre-trained network parameters, and modifying the network parameters comprises:
repeatedly determining updated network parameters based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained from the one or more neural networks using the pre-trained network parameters, and the second set of transitions having been obtained from the one or more neural networks using the updated network parameters.
19 . The system of claim 11 , wherein the one or more neural networks are to be trained, at least in part, by using at least one other neural network to determine when the second agent is to grasp the at least one second target held by the at least one non-moving second holder and to determine when the second agent is to grasp the at least one third target held by the at least one moving third holder.
20 . The system of claim 11 , wherein reinforcement learning is used to train the one or more neural networks.
21 . The system of claim 20 , wherein the reinforcement learning comprises at least one actor-critic process.
22 . A processor comprising:
one or more arithmetic logic units (ALUs) to maneuver a first agent to avoid collision with a first participant comprising a real-world living participant or at least a portion of a virtual character using one or more neural networks trained, at least in part, by:
performing a first portion of training one or more neural networks, the first portion of the training comprising using at least one first expert demonstration of maneuvering a second agent with respect to a first target held by a stationary second participant to guide training the one or more neural networks; and
performing a second portion of the training the one or more neural networks, the second portion of the training comprising using one or more actions output by the one or more neural networks operating with parameter values determined during the first portion of the training and at least one second expert demonstration of maneuvering a third agent with respect to a second target held by a non-stationary third participant to train the one or more neural networks.
23 . The processor of claim 22 , wherein the at least one first expert demonstration is provided by at least one motion planner.
24 . The processor of claim 23 , wherein the first agent comprises a gripper and maneuvering the first agent comprises grasping at least one object held by the first participant.
25 . The processor of claim 22 , wherein the first portion of the training comprises:
obtaining initial parameter values; and
repeatedly updating the initial parameter values based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained using the at least one first expert demonstration, and the second set of transitions having been obtained from the one or more neural networks using the initial parameter values.
26 . The processor of claim 22 , wherein the at least one first expert demonstration is obtained using at least one motion planner that uses a set of pre-planned grasps from which any grasps that collide with the stationary second participant have been removed.
27 . The processor of claim 22 , wherein the second portion of the training comprises:
repeatedly determining updated parameter values based at least in part on a first set of transitions and a second set of transitions, the first set of transitions having been obtained from the one or more neural networks operating with the parameter values, and the second set of transitions having been obtained from the one or more neural networks using the updated parameter values.
28 . The processor of claim 22 , wherein the one or more ALUs are to use at least one other neural network to at least one of:
determine when the second agent is to grasp the first target, or
determine when the third agent is to grasp the second target.
29 . The processor of claim 22 , wherein the one or more neural networks are to be trained, at least in part, by providing at least one point cloud to the one or more neural networks as input, the at least one point cloud encoding at least one of the stationary second participant, or the non-stationary third participant.
30 . The method of claim 1 , further comprising:
obtaining a policy using the at least one neural network after the at least one neural network has been trained to avoid collision with the at least one stationary first holder, wherein updating the parameter values of the at least one neural network comprise using the policy to guide refinement of the parameter values as the at least one neural network is trained to control movement of the second agent to avoid collision with the at least one non-stationary second holder of the at least one second target.