IP Library Granted Patent US 10,675,761
Granted Patent B2
US 10,675,761 · App. 15/785,040 · Granted Jun 9, 2020

Mode architecture for general purpose robotics

Inventors: Frederick Dennis Zyda (Redwood City, CA); Jeffrey Steven Kranski (San Jose, CA); Vikram Chauhan (San Diego, CA)
Assignee: Magic Leap, Inc.
B25J9/1682B25J9/1658G05B19/042G05B19/0426G05B2219/31076G05B2219/40397
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,675,761
App. No.
15/785,040
Granted
Jun 9, 2020
Kind
B2
Abstract

An improved method, system, and apparatus is provided to implement a general architecture for robot systems. A mode execution module is provided to universally execute execution modes on different robotic system. A system includes an execution module that receives software instructions in a normalized programming language. The system also includes an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system. The system further includes a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions. Moreover, the system includes a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions.

Claims (58)

1. A system, comprising:

an execution module that receives software instructions in a normalized programming language;

an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system;

a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions; and

a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions,

wherein the software instructions include:

identifying a selection of a selected mode,

determining whether the selected mode is acceptable in light of a current state of the robotic device,

placing the robotic device into a safe exit state or location for a current mode responsive to determining that the selected mode is acceptable,

stopping the current mode,

placing the robotic device into a safe entry state or location for the selected mode, and

starting the selected mode.

2. The system of claim 1 , wherein the software instructions comprises one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

3. The system of claim 1 , further comprising a second interface that converts the software instructions from the normalized language into second robot-specific instructions that are sent to a second controller to operatively control a second robotic device, wherein the second robotic device corresponds to a different type of device from the robot device, and the second robot-specific instructions for the second robotic device corresponds to a different set of programming instructions from the robot-specific instructions for the robotic device.

4. The system of claim 1 , wherein the execution module comprises a mode selection module, a mode instruction processing module, and a controller interface module, the mode selection module selecting a specific mode based on a mode selection instruction, the mode instruction processing module determining a specific combination or sequence of mode instructions to be issued to the controller to perform a desired operation, and the controller interface module issuing one or more API calls corresponding to the particular robotic system that are sent to the controller.

5. The system of claim 1 , wherein at least one of safe entry points or safe exit points are established for entry or exit of modes.

6. The system of claim 1 , further comprising a rulebase having a set of rules that identifies acceptable or unacceptable sequences of modes for one or more robotic systems, wherein the set of rules are accessed to determine wherein a selected mode of operation for the robotic device is acceptable as a next mode of operation.

7. The system of claim 1 , wherein the controller comprises a real-time operating system that controls a motor driver to operate the robotic device, and the execution module is loaded as an application onto the controller.

8. The system of claim 1 , wherein the controller comprises both a real-time operating system (RTOS) and a user-space operating system (OS), the RTOS controlling a motor driver to operate the robotic device, the user-space OS providing user-specified commands to the RTOS, and the interface issuing API calls that are understandable by a manufacturer-provided robot application layer within the user-space OS to execute the software instructions.

9. The system of claim 1 , wherein the controller comprises a thin interface layer and mode control functionality is located within a mode control/select layer that is located at a control application at a control computer, the interface having functionality to communicate with the mode control/select layer and to deliver the robot-specific instructions to a RTOS.

10. The system of claim 1 , wherein the execution module is implemented as a loadable kernel module (LKM) within a RTOS at the controller.

11. A method, comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device;

placing the robotic device into a safe exit state or location for a current mode responsive to determining that the selected mode is acceptable;

stopping the current mode;

placing the robotic device into a safe entry state or location for the selected mode; and

starting the selected mode.

12. The method of claim 11 , wherein the software instructions comprises one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

13. The method of claim 11 , wherein a set of rules in a rulebase are accessed to determine wherein the selected mode for the robotic device is acceptable as a next mode of operation.

14. The method of claim 13 , wherein if the selected mode is not acceptable, then a determination is made of an alternative instruction that is an acceptable alternative to the selected mode.

15. The method of claim 14 , wherein the alternative instruction is determined by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

16. The method of claim 11 , wherein within a single mode, the safe exit state or location is different from the safe entry state or location.

17. The method of claim 11 , wherein between different modes, the safe exit state or location or safe entry state or location for a first mode is different from the safe exit state or location or safe entry state or location for a second mode.

18. The method of claim 11 , wherein once a new mode is entered, a waiting period is entered for a trigger.

19. The method of claim 18 , wherein the robotic device performs a predefined mode function corresponding to the trigger.

20. The method of claim 11 , wherein the software instructions are converted from the normalized language into robot-specific instructions by generating a set of API calls that correspond to the particular robotic system corresponding to the robotic device.

21. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device;

placing the robotic device into a safe exit state or location for a current mode responsive to determining that the selected mode is acceptable;

stopping the current mode;

placing the robotic device into a safe entry state or location for the selected mode; and starting the selected mode.

22. The computer program product of claim 21 , wherein the software instructions comprises one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

23. The computer program product of claim 21 , wherein a set of rules in a rulebase are accessed to determine wherein the selected mode for the robotic device is acceptable as a next mode of operation.

24. The computer program product of claim 23 , wherein if the selected mode is not acceptable, then a determination is made of an alternative instruction that is an acceptable alternative to the selected mode.

25. The computer program product of claim 24 , wherein the alternative instruction is determined by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

26. The computer program product of claim 21 , wherein within a single mode, the safe exit state or location is different from the safe entry state or location.

27. The computer program product of claim 21 , wherein between different modes, the safe exit state or location or safe entry state or location for a first mode is different from the safe exit state or location or safe entry state or location for a second mode.

28. The computer program product of claim 21 , wherein once a new mode is entered, a waiting period is entered for a trigger.

29. The computer program product of claim 28 , wherein the robotic device performs a predefined mode function corresponding to the trigger.

30. The computer program product of claim 21 , wherein the software instructions are converted from the normalized language into robot-specific instructions by generating a set of API calls that correspond to the particular robotic system corresponding to the robotic device.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2020
From: ZYDA, FREDERICK DENNIS; KRANSKI, JEFFREY STEVEN; CHAUHAN, VIKRAM
To: MAGIC LEAP, INC.
Reel/Frame 052704/0612 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
Continuity (2)
Provisional Application 62408354 · Oct 14, 2016
Related Publication 20180104819A1 · Apr 19, 2018