IP Library Granted Patent US 12,122,054
Granted Patent B2
US 12,122,054 · App. 18/049,586 · Granted Oct 22, 2024

Mode architecture for general purpose robotics

Inventors: Frederick Dennis Zyda (Plantation, FL); Jeffrey Steven Kranski (Portland, OR); Vikram Chauhan (San Diego, CA)
Assignee: Magic Leap, Inc.
B25J9/1682B25J9/1658G05B19/042G05B19/0426G05B2219/31076G05B2219/40397
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,122,054
App. No.
18/049,586
Granted
Oct 22, 2024
Kind
B2
Abstract

An improved method, system, and apparatus is provided to implement a general architecture for robot systems. A mode execution module is provided to universally execute execution modes on different robotic system. A system includes an execution module that receives software instructions in a normalized programming language. The system also includes an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system. The system further includes a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions. Moreover, the system includes a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions.

Claims (45)

1. A system, comprising:

an execution module that receives software instructions in a normalized programming language;

an interface having a translation layer that converts the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system;

a controller that is communicatively coupled to the interface, wherein the controller receives the robot-specific instructions; and

a robotic device that is operatively controlled by the controller by execution of the robot-specific instructions, wherein the software instructions comprise:

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

upon a determination that the selected mode is not acceptable, then determining an alternative instruction that is an acceptable alternative to the robot-specific instructions for the selected mode, wherein the alternative instruction comprises a first instruction that places the robotic device in a safe exist location or state and a second instruction that causes the robotic device to execute the alternative instruction from the safe exit location or state.

2. The system of claim 1 , wherein the software instructions comprise:

upon a determination that the selected mode is acceptable, modifying the software instructions based at least in part on the alternative instruction.

3. The system of claim 1 , further comprising a rulebase having a set of rules that identifies acceptable or unacceptable sequences of modes for one or more robotic systems, wherein the set of rules are accessed to determine whether a selected mode of operation for the robotic device is acceptable as a next mode of operation.

4. The method of claim 3 , wherein the alternative instruction is determined by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

5. The system of claim 1 , wherein the software instructions comprise one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

6. The system of claim 1 , further comprising a second interface that converts the software instructions from the normalized programming language into second robot-specific instructions that are sent to a second controller to operatively control a second robotic device, wherein the second robotic device corresponds to a different type of device from the robotic device, and the second robot-specific instructions for the second robotic device correspond to a different set of programming instructions from the robot-specific instructions for the robotic device.

7. The system of claim 1 , wherein the execution module comprises a mode selection module, a mode instruction processing module, and a controller interface module, the mode selection module selecting a specific mode based at least in part on a mode selection instruction, the mode instruction processing module determining a specific combination or sequence of mode instructions to be issued to the controller to perform an operation, and the controller interface module issuing one or more Application Programming Interface (API) calls corresponding to the particular robotic system that are sent to the controller.

8. The system of claim 1 , wherein the controller comprises a real-time operating system that controls a motor driver to operate the robotic device, and the execution module is loaded as an application onto the controller.

9. The system of claim 1 , wherein the controller comprises both a real-time operating system (RTOS) and a user-space operating system (OS), the RTOS controlling a motor driver to operate the robotic device, the user-space OS providing user-specified commands to the RTOS, and the interface issuing API (Application Programming Interface) calls that are understandable by a manufacturer-provided robot application layer within the user-space OS to execute the software instructions.

10. The system of claim 1 , wherein the controller comprises a thin interface layer and mode control functionality is located within a mode control or select layer that is located at a control application at a control computer, the interface having functionality to communicate with the mode control or select layer and to deliver the robot-specific instructions to an RTOS (real-time operating system).

11. The system of claim 1 , wherein the execution module is implemented as a loadable kernel module (LKM) within a RTOS at the controller.

12. A method, comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

upon a determination that the selected mode is not acceptable, then determining an alternative instruction that is an acceptable alternative to the robot-specific instructions for the selected mode, wherein the alternative instruction comprises a first instruction that places the robotic device in a safe exist location or state and a second instruction that causes the robotic device to execute the alternative instruction from the safe exit location or state.

13. The method of claim 12 , further comprising:

upon a determination that the selected mode is acceptable, modifying the software instructions based at least in part on the alternative instruction.

14. The method of claim 12 , further comprising:

accessing a set of rules in a rulebase to determine whether the selected mode for the robotic device is acceptable as a next mode of operation.

15. The method of claim 14 , wherein the alternative instruction is determined by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

16. The method of claim 12 , wherein the software instructions comprise one or more execution modes corresponding to at least one of an enter mode, an exit mode, and a trigger mode.

17. The method of claim 12 , wherein the software instructions are converted from the normalized programming language into robot-specific instructions by generating a set of Application Programming Interface (API) calls that correspond to the particular robotic system corresponding to the robotic device.

18. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method comprising:

receiving software instructions for a robotic device in a normalized programming language;

converting the software instructions from the normalized language into robot-specific instructions that operate in a particular robotic system corresponding to the robotic device;

controlling the robotic device by execution of the robot-specific instructions;

identifying a selection of a selected mode;

determining whether the selected mode is acceptable in light of a current state of the robotic device; and

upon a determination that the selected mode is not acceptable, then determining an alternative instruction that is an acceptable alternative to the robot-specific instructions for the selected mode, wherein the alternative instruction comprises a first instruction that places the robotic device in a safe exist location or state and a second instruction that causes the robotic device to execute the alternative instruction from the safe exit location or state.

19. The computer program product of claim 18 , wherein the method comprises:

upon a determination that the selected mode is acceptable, modifying the software instructions based at least in part on the alternative instruction.

20. The computer program product of claim 18 , wherein the method comprises:

accessing a set of rules in a rulebase to determine whether the selected mode for the robotic device is acceptable as a next mode of operation; and

determining the alternative instruction by at least one of checking the rulebase, performing manual intervention, or using a machine learning system.

Assignments (3)
SECURITY INTEREST Recorded Oct 29, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073430/0225 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2022
From: ZYDA, FREDERICK DENNIS; KRANSKI, JEFFREY STEVEN; CHAUHAN, VIKRAM
To: MAGIC LEAP, INC.
Reel/Frame 061533/0912 →
Continuity (4)
Continuation 16865225 · May 1, 2020
Continuation 15785040 · Oct 16, 2017
Provisional Application 62408354 · Oct 14, 2016
Related Publication 20230061945A1 · Mar 2, 2023