IP Library Granted Patent US 11,279,036
Granted Patent B2
US 11,279,036 · App. 16/148,343 · Granted Mar 22, 2022

Methods and systems for implementing customized motions based on individual profiles for identified users

Inventors: Allison Thackston (San Jose, CA); Samuel Zapolsky (San Francisco, CA); Katarina Bouma (Palo Alto, CA); Laura Stelzner (Mountain View, CA); Ron Goldman (Los Altos, CA)
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
B25J9/1689B25J13/006B25J9/1679B25J19/023
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,279,036
App. No.
16/148,343
Granted
Mar 22, 2022
Kind
B2
Abstract

A teleoperation system includes a robot comprising an actuator configured to move at least a portion of the robot, and a remote computing device comprising: one or more processors, one or more sensors communicatively coupled to the one or more processors, a non-transitory memory component communicatively coupled to the one or more processors, and machine readable instructions stored in the non-transitory memory component. The remote computing device obtains information about a user proximate to the remote computing device, identifies the user based on the obtained information, obtains an action of a user, retrieves an individual profile for the user based on the identified user, determines an intended instruction related to a task based on the action of the user related to the task and the individual profile for the user, and instructs the robot to implement the task with the actuator based on the intended instruction.

Claims (57)

1. A teleoperation system comprising:

a robot comprising an actuator configured to move at least a portion of the robot; and

a remote computing device configured to:

obtain information about a user proximate to the remote computing device with one or more sensors;

identify the user based on the obtained information;

obtain an action of the user;

retrieve an individual profile among a plurality of profiles based on the identified user, each of the plurality of profiles including intended instructions associated with corresponding actions;

determine an intended instruction related to a task by interpreting the action of the user related to the task based on the intended instruction associated with the action of the user; and

instruct the robot to implement the task with the actuator based on the intended instruction,

wherein the individual profile includes intent parameters related to the action of the user, and

wherein the individual profile includes a first maximum allowable deviation for implementing the task that is different from a second maximum allowable deviation included in another individual profile, the first maximum allowable deviation is a maximum deviation between a position of an object to be placed by the robot and a position designated by the user, and the robot is instructed to implement the task of placing the object within the first maximum allowable deviation.

2. The teleoperation system of claim 1 , wherein the individual profile includes a workspace scaling value.

3. The teleoperation system of claim 2 , wherein the robot further comprises an arm,

wherein the remote computing device is configured to cause the robot to operate the arm with the actuator based on the action of the user and the workspace scaling value.

4. The teleoperation system of claim 1 , wherein the individual profile includes one or more operating gains of the robot.

5. The teleoperation system of claim 1 , wherein the one or more sensors includes one or more imaging devices configured to obtain one or more images of the user.

6. The teleoperation system of claim 1 , wherein:

the remote computing device comprises an output device configured to provide feedback to the user based on an operation of the robot, and

the remote computing device is configured to determine the feedback based on the individual profile for the user.

7. The teleoperation system of claim 1 , wherein:

the remote computing device comprises a display configured to display a view related to the robot, and

the remote computing device is configured to determine a type of the view based on the individual profile for the user.

8. The teleoperation system of claim 1 , wherein the remote computing device is configured to identify a location related to the task based on the action of the user, and

implement the task within the first maximum allowable deviation from the location.

9. The teleoperation system of claim 1 , further comprising a screen on the remote computing device configured to display information related to the intended instruction.

10. A robot system comprising:

one or more processors;

one or more sensors communicatively coupled to the one or more processors;

a non-transitory memory component communicatively coupled to the one or more processors; and

machine readable instructions stored in the non-transitory memory component that cause the robot system to perform at least the following when executed by the one or more processors:

obtain information about a user proximate to the robot system with the one or more sensors;

identify the user based on the obtained information;

obtain an action of the user;

retrieve an individual profile among a plurality of profiles based on the identified user, each of the plurality of profiles including intended instructions associated with corresponding actions;

determine an intended instruction related to a task by interpreting the action of the user related to the task based on the intended instruction associated with the action of the user; and

implement the task based on the intended instruction,

wherein the individual profile includes intent parameters related to the action of the user, and

wherein the individual profile includes a first maximum allowable deviation for implementing the task that is different from a second maximum allowable deviation included in another individual profile, the first maximum allowable deviation is a maximum deviation between a position of an object to be placed by the robot system and a position designated by the user, and the robot system implements the task of placing the object within the first maximum allowable deviation.

11. The robot system of claim 10 , wherein the individual profile includes a workspace scaling value.

12. The robot system of claim 11 , further comprising an arm,

wherein the robot system is configured to operate the arm based on the action of the user and the workspace scaling value.

13. The robot system of claim 11 , wherein the robot system is configured to receive information on the action of the user from a wearable device.

14. A method for operating a robot, the method comprising:

obtaining, by one or more sensors of a remote computing device, information about a user proximate to the remote computing device;

identifying, by a controller of the remote computing device, the user based on the obtained information;

obtaining, by the controller of the remote computing device, an action of the user;

retrieving, by the controller of the remote computing device, an individual profile for the user among a plurality of profiles based on the identified user, each of the plurality of profiles including intended instructions associated with corresponding actions;

determining, by the controller of the remote computing device, an intended instruction related to a task by interpreting the action of the user related to the task based on the intended instruction associated with the action of the user; and

instructing, by the controller of the remote computing device, the robot to implement the task based on the intended instruction,

wherein the individual profile includes intent parameters related to the action of the user, and

wherein the individual profile includes a first maximum allowable deviation for implementing the task that is different from a second maximum allowable deviation included in another individual profile, the first maximum allowable deviation is a maximum deviation between a position of an object to be placed by the robot and a position designated by the user, and the robot is instructed to implement the task of placing the object within the first maximum allowable deviation.

15. The method of claim 14 , wherein the individual profile includes a workspace scaling value.

16. The method of claim 14 , wherein the individual profile includes one or more operating gains of the robot.

17. The method of claim 14 , wherein the one or more sensors includes one or more imaging devices configured to obtain one or more images of the user.

18. The method of claim 14 , further comprising:

identifying a location related to the task based on the action of the user; and

implementing the task within the first maximum allowable deviation from the location.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 7, 2022
From: TOYOTA RESEARCH INSTITUTE INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 060118/0937 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 1, 2018
From: THACKSTON, ALLISON; ZAPOLSKY, SAMUEL; BOUMA, KATARINA; STELZNER, LAURA; GOLDMAN, RON
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 047020/0746 →
Continuity (1)
Related Publication 20200101614A1 · Apr 2, 2020
Cited By (2)
US 12,466,071 US 12,468,963