IP Library Granted Patent US 9,792,546
Granted Patent B2
US 9,792,546 · App. 13/918,298 · Granted Oct 17, 2017

Hierarchical robotic controller apparatus and methods

Inventors: Jean-Baptiste Passot (La Jolla, CA); Oleg Sinyavskiy (San Diego, CA); Filip Ponulak (San Diego, CA); Patryk Laurent (San Diego, CA); Borja Ibarz Gabardos (San Diego, CA); Eugene Izhikevich (San Diego, CA); Vadim Polonichko (San Diego, CA)
Assignee: Brain Corporation
G06N3/049G06N3/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,792,546
App. No.
13/918,298
Granted
Oct 17, 2017
Kind
B2
Abstract

A robot may be trained by a user guiding the robot along target trajectory using a control signal. A robot may comprise an adaptive controller. The controller may be configured to generate control commands based on the user guidance, sensory input and a performance measure. A user may interface to the robot via an adaptively configured remote controller. The remote controller may comprise a mobile device, configured by the user in accordance with phenotype and/or operational configuration of the robot. The remote controller may detect changes in the robot phenotype and/or operational configuration. The remote controller may comprise multiple control elements configured to activate respective portions of the robot platform. Based on training, the remote controller may configure composite controls configured based two or more of control elements. Activation of a composite control may enable the robot to perform a task.

Claims (63)

1. A method for controlling a robot to execute a task, the method comprising:

during a given trial of one or more training trials, based on a first indication received from a user, executing a plurality of actions, individual ones of the plurality of actions being configured based on sensory input and a given user input of a plurality of user inputs, where each one of the one or more training trials has a trial duration associated therewith, the trial duration at least being configured to separate two or more activations of at least one actuator of the robot;

determining a performance measure associated with the executing of the plurality of actions, the performance measure being based at least in part on a difference between target actions and the plurality of actions;

when the performance measure does not meet or exceed a target level, during a subsequent trial of the one or more training trials, executing another plurality of actions based on a second indication received from the user, and determining another performance measure associated with the executing of the other plurality of actions; and

when the performance measure meets or exceeds the target level, based on a third indication received from a user, associating a control component with the task, the control component being configured to be activated by a presence of a user discernible representation;

wherein:

the execution of the plurality of actions is configured to effectuate execution of the task by the robot; and

activation by the user of the control component using the user discernible representation is configured to cause the robot to execute the task in a sequence corresponding to the plurality of actions that were executed based on the first indication received from the user.

2. The method of claim 1 , wherein the provision of the user discernible representation comprises:

disposing an icon on a display; and

configuring a user interface device to receive input based on the user activation configured in accordance with the icon.

3. The method of claim 1 , wherein the user discernible representation comprises a voice command, an audio signal, or one or more gestures.

4. A non-transitory computer readable medium having instructions embodied thereon, the instructions configured to, when executed by a physical processor, cause the physical processor to:

based on a detection of a sequence of discrete control actions comprising two or more activations of a robotic apparatus by a user, generate a composite control element, the composite control element being configured to execute the sequence of discrete control actions in an order of execution provided by one or more parameters associated with at least one of the sequence of discrete control actions; and

when the execution of the sequence of discrete control actions is within an expected performance value assign the generated composite control element to a tag, the tag associated with a user interface control element;

wherein:

an invocation of the tag is configured to execute the sequence of discrete control actions in accordance with the determined order of execution;

the robotic apparatus comprises a controller configured to generate a control signal, individual ones of the sequence of discrete control actions being configured based on the control signal, the generation of the control signal being effectuated by a learning process;

the learning process comprises execution of multiple training trials, individual ones of the multiple training trials being characterized by a trial duration;

individual ones of the sequence of discrete control actions correspond to an execution of a given training trial of the multiple training trials; and

the two or more activations of the robotic apparatus comprise activations of at least one actuator at two or more instances of time, the two or more instances of time being separated by a time period, the time period corresponding to the trial duration.

5. The non-transitory computer readable medium of claim 4 , wherein:

the sequence is configured to cause the robotic apparatus to execute a target task;

individual ones of the two or more activations are configured to execute responsive to the user issuing two or more control commands via a remote control interface; and

the composite control element is configured to execute the target task responsive to a single activation of the composite control element by the user.

6. The non-transitory computer readable medium of claim 5 , wherein individual ones of the two or more control commands comprise multiple instances of a given control operation effectuated based on multiple activations of a first control element associated with the remote control interface.

7. The non-transitory computer readable medium of claim 5 , wherein:

individual ones of the two or more control commands comprise one or more instances of two or more control operation effectuated based on activations of a first control element and a second control element associated with the remote control interface; and

the single activation of the composite control element is configured to obviate the activations of the first control element and the second control element by the user.

8. The non-transitory computer readable medium of claim 5 , wherein:

the user comprises a human; and

detection is configured based on a request by the user.

9. The non-transitory computer readable medium of claim 4 , wherein:

the learning process comprises adjusting a learning parameter based on a performance measure, the performance measure being configured based on individual ones of the sequence of discrete control actions and a target action; and

the detection is effectuated based on an indication provided by the learning process absent user request.

10. The non-transitory computer readable medium of claim 4 , wherein:

the robotic apparatus comprises two or more individually controllable actuators; and

the two or more activations of the robotic apparatus comprise activations of individual ones of the two or more actuators.

11. A remote control apparatus of a robot, the remote control apparatus comprising:

a physical processor configured to operate a learning process;

a sensor coupled to the physical processor;

a user interface configured to present one or more human perceptible control elements; and

a remote communications interface configured to communicate to the robot a plurality of control commands configured by the learning process based on an association between a sensor input and individual ones of a plurality of user inputs provided via one or more of the one or more human perceptible control elements;

wherein:

the communication to the robot of the plurality of control commands is configured to cause the robot to execute a plurality of actions;

the learning process is configured based on a performance measure between a target action and individual ones of the plurality of actions;

the association between the sensor input and the individual ones of the plurality of user inputs is configured to cause generation of one or more of control primitives, individual ones of the one or more of control primitives being configured to cause execution of a respective action of the plurality of actions;

the learning process is configured to generate a composite control;

the composite control is configured to actuate the plurality of actions that result in an execution of the target action responsive to a single activation of the composite control by a user, the single activation by the user being configured to cause the execution of the plurality of actions in an order determined according to the plurality of user inputs;

the single activation of the composite control is configured based on a detection, by the remote control apparatus, of a presence of the one or more human perceptible control elements;

the generation of the composite control comprises a presentation of the one or more human perceptible control elements, the one or more human perceptible control elements being configured to cause activation of the composite control by the user; and

the activation of the composite control is configured based on a detection of one or more of: an audio signal, a touch signal, and an electrical signal by the user interface, where the user interface comprises a camera, and the electrical signal is configured based on a captured representation of the user by the camera.

12. The apparatus of claim 11 , wherein:

the learning process is configured to generate the composite control based on a detection, by the remote control apparatus, of the individual ones of the plurality of user inputs provided via the one or more of the one or more human perceptible control elements; and

an individual action of the plurality of actions corresponds to execution of a task.

13. The apparatus of claim 11 , wherein:

the learning process is configured to generate the composite control based on a request by the user.

14. The apparatus of claim 11 , wherein the learning process is configured based on a supervised learning process, the supervised learning process being configured based on a sensory context and a combination of a control signal and the individual ones of the plurality of user inputs.

15. The apparatus of claim 11 , wherein:

the provision of individual ones of the plurality of user inputs is configured based on an audible tag or a touch indication of the user interface.

16. The apparatus of claim 11 , wherein:

the user interface comprises a touch sensitive interface; and

the touch signal is configured based on a pattern provided by the user via the touch interface.

Assignments (2)
SECURITY INTEREST Recorded Oct 8, 2021
From: BRAIN CORPORATION
To: HERCULES CAPITAL, INC.
Reel/Frame 057851/0574 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 13, 2014
From: PASSOT, JEAN-BAPTISTE; SINYAVSKIY, OLEG; PONULAK, FILIP; LAURENT, PATRYK; GABARDOS, BORJA IBARZ; IZHIKEVICH, EUGENE; POLONICHKO, VADIM
To: BRAIN CORPORATION
Reel/Frame 033094/0703 →
Continuity (1)
Related Publication 20140371912A1 · Dec 18, 2014