IP Library Granted Patent US 11,511,434
Granted Patent B2
US 11,511,434 · App. 16/865,225 · Granted Nov 29, 2022

Mode architecture for general purpose robotics

Inventors: Frederick Dennis Zyda (Plantation, FL); Jeffrey Steven Kranski (Portland, OR); Vikram Chauhan (San Diego, CA)
Assignee: Magic Leap, Inc.
B25J9/1682B25J9/1658G05B19/042G05B19/0426G05B2219/31076G05B2219/40397
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,511,434
App. No.
16/865,225
Granted
Nov 29, 2022
Kind
B2
Abstract

An improved method, system, and apparatus is provided to implement a general architecture for robot systems. A mode execution module is provided to universally execute execution modes on different robotic system. A system includes an execution module that receives software instructions in a normalized programming language. The system also includes an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system. The system further includes a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions. Moreover, the system includes a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions.

Claims (41)

1. A system, comprising:

an execution module that receives software instructions in a normalized programming language;

an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system;

a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions; and

a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions

wherein the software instructions include:

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

placing the robotic device into a safe exit state or location for a current mode if the selected mode is acceptable.

2. The system of claim 1 , wherein the software instructions comprises one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

3. The system of claim 1 , further comprising a second interface that converts the software instructions from the normalized language into second robot-specific instructions that are sent to a second controller to operatively control a second robotic device, wherein the second robotic device corresponds to a different type of device from the robot device, and the second robot-specific instructions for the second robotic device corresponds to a different set of programming instructions from the robot-specific instructions for the robotic device.

4. The system of claim 1 , wherein the execution module comprises a mode selection module, a mode instruction processing module, and a controller interface module, the mode selection module selecting a specific mode based on a mode selection instruction, the mode instruction processing module determining a specific combination or sequence of mode instructions to be issued to the controller to perform a desired operation, and the controller interface module issuing one or more API calls corresponding to the particular robotic system that are sent to the controller.

5. The system of claim 1 , wherein at least one of safe entry points or safe exit points are established for entry or exit of modes.

6. The system of claim 1 , further comprising a rulebase having a set of rules that identifies acceptable or unacceptable sequences of modes for one or more robotic systems, wherein the set of rules are accessed to determine wherein a selected mode of operation for the robotic device is acceptable as a next mode of operation.

7. The system of claim 1 , wherein the controller comprises a real-time operating system that controls a motor driver to operate the robotic device, and the execution module is loaded as an application onto the controller.

8. The system of claim 1 , wherein the controller comprises both a real-time operating system (RTOS) and a user-space operating system (OS), the RTOS controlling a motor driver to operate the robotic device, the user-space OS providing user-specified commands to the RTOS, and the interface issuing API calls that are understandable by a manufacturer-provided robot application layer within the user-space OS to execute the software instructions.

9. The system of claim 1 , wherein the controller comprises a thin interface layer and mode control functionality is located within a mode control/select layer that is located at a control application at a control computer, the interface having functionality to communicate with the mode control/select layer and to deliver the robot-specific instructions to a RTOS.

10. The system of claim 1 , wherein the execution module is implemented as a loadable kernel module (LKM) within a RTOS at the controller.

11. A method, comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

placing the robotic device into a safe exit state or location for a current mode if the selected mode is acceptable.

12. The method of claim 11 , wherein the software instructions comprises one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

13. The method of claim 11 , further comprising

placing the robotic device into a safe entry state or location for the selected mode.

14. The method of claim 13 , wherein a set of rules in a rulebase are accessed to determine wherein the selected mode for the robotic device is acceptable as a next mode of operation.

15. The method of claim 14 , wherein if the selected mode is not acceptable, then a determination is made of an alternative instruction that is an acceptable alternative to the selected mode.

16. The method of claim 15 , wherein the alternative instruction is determined by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

17. The method of claim 13 , wherein within a single mode, the safe exit state or location is different from the safe entry state or location.

18. The method of claim 13 , wherein between different modes, the safe exit state or location or safe entry state or location for a first mode is different from the safe exit state or location or safe entry state or location for a second mode.

19. The method of claim 11 , wherein the software instructions are converted from the normalized language into robot-specific instructions by generating a set of API calls that correspond to the particular robotic system corresponding to the robotic device.

20. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

placing the robotic device into a safe exit state or location for a current mode if the selected mode is acceptable.

Assignments (4)
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
SECURITY INTEREST Recorded May 21, 2020
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 052729/0791 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2020
From: ZYDA, FREDERICK DENNIS; KRANSKI, JEFFREY STEVEN; CHAUHAN, VIKRAM
To: MAGIC LEAP, INC.
Reel/Frame 052704/0612 →