IP Library › Granted Patent US 12,325,594
Granted Patent B2
US 12,325,594 · App. 18/375,742 · Granted Jun 10, 2025

Storage systems and methods for robotic picking

Inventor: Simon Kalouche (San Francisco, CA)
Assignee: Nimble Robotics, Inc.
B65G1/1373B25J5/007B25J5/02B25J9/042B25J9/14B25J9/161B25J9/1612B25J9/162B25J9/163B25J9/1689B25J9/1697B25J13/00B25J13/006B25J13/08B25J13/089B25J15/0061B25J15/02B25J15/0441B25J15/0616B25J15/0625B25J17/02B25J19/0025B25J19/005B25J19/02B25J19/021B25J19/04B60M1/30B65G1/0464B65G1/0478B65G1/065B65G1/1375B65G47/91B65G47/917B65G47/918B65G2203/041B65G2203/042B65G2209/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,325,594
App. No.
18/375,742
Granted
Jun 10, 2025
Kind
B2
Abstract

A robotic system including a storage structure provided with vertical members supporting a grid collectively defining grid spaces and a plurality of storage containers arranged in vertical stacks for storing inventory items, each one of the vertical stacks configured to be arranged within a respective one of the grid spaces. One or more load handling robots are installed upon the grid and include a lifting device extendable in a vertical direction to extract one or more storage containers from the vertical stacks and transport the extracted storage container to a station. The robotic system further includes one or more manipulator robots installed at the station and within a single grid space. The one or more manipulator robots include a picking arm provided with a pneumatic gripping tool arranged to pick an inventory item from the transported storage container and place the picked item into an order container.

Claims (51)

1. A robotic system, comprising:

a storage structure, comprising:

a frame including vertical members supporting a grid having a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction such that the first and second set of parallel rails collectively define grid spaces; and

a plurality of storage containers arranged in vertical stacks for storing inventory items, each one of the vertical stacks configured to be arranged within a respective one of the grid spaces;

a container handling robot installed upon the grid to move along the first and second set of parallel rails, the container handling robot including a lifting device extendable in a vertical direction to extract one or more storage containers from the vertical stacks and transport the extracted storage container to a station; and

a manipulator robot installed at the station and within a single grid space, the manipulator robot including a picking arm provided with a pneumatic gripping tool arranged to pick an inventory item from the storage container located at the station and place the picked item into an order container.

2. The robotic system of claim 1 , further comprising a pneumatic supply line coupling the manipulator robot to an external pneumatic source.

3. The robotic system of claim 2 , wherein the pneumatic supply line extends at least partially through the storage structure.

4. The robotic system of claim 2 , wherein the pneumatic supply line extends at least partially in the vertical direction toward the grid and the pneumatic supply line is at least partially external of the frame.

5. The robotic system of claim 1 , wherein the pneumatic gripping tool comprises a plurality of gripping elements.

6. The robotic system of claim 1 , wherein the container handling robot equipped with a gripper plate.

7. The robotic system of claim 1 , wherein the container handling robot defines a vehicle body having a container receiving cavity.

8. The robotic system of claim 1 , further comprising an imaging sensor to capture one or more images of the inventory item.

9. The robotic system of claim 8 , wherein the manipulator robot further comprises:

an interface configured to send processor readable data to a central computing system and receive processor executable instruction from the central computing system; and

an onboard processor in communication with at least one of the imaging sensor or the picking arm.

10. The robotic system of claim 9 , further comprising a tool holder provided on the manipulator robot or coupled to the frame, the tool holder being arranged to hold another pneumatic gripping tool.

11. The robotic system of claim 10 , further comprising a teleoperator interface including one or more input device configured to receive one or more inputs from an operator to assist the manipulator robot in picking the inventory item from the storage container and/or placing the inventory item in a selected orientation within the order container.

12. The robotic system of claim 11 , wherein the picking arm is configured to interchangeably couple to the pneumatic gripping tool and the another pneumatic gripping tool upon receiving instructions from one of the central computing system, the onboard processor, or the teleoperator interface.

13. A robotic system, comprising:

a storage structure, comprising:

a frame including vertical members supporting a grid having a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction such that the first and second set of parallel rails collectively define grid spaces; and a plurality of storage containers arranged in vertical stacks for storing inventory items, each one of the vertical stacks configured to be arranged within a respective one of the grid spaces;

one or more container handling robots installed upon the grid and arranged to move along the first and second set of parallel rails, each of the container handling robots including a lifting device extendable in a vertical direction to extract one or more storage containers from the vertical stacks and transport the extracted storage container to a station;

one or more manipulator robots installed at the station on the grid, each of the one or more manipulator robots including a picking arm provided with a gripping tool arranged to pick an inventory item from the storage container located at the station and place the picked item into an order container;

an imaging sensor to capture one or more images of the inventory item within the storage container;

a central computing system for coordinating operation of the one or more container handling robots about the grid and transmitting manipulation task instructions to the one or more manipulator robots; and

a teleoperator interface configured to:

receive the one or more images, the teleoperator interface including one or more input devices configured to receive one or more inputs from an operator; and

transmit, based on the one or more inputs, instructions to the central computing system for assisting the manipulator robot in performing the manipulation task instructions, the manipulation task instructions including picking the inventory item from the transported storage container and/or placing the inventory item in a selected orientation within the order container.

14. The robotic system of claim 13 , wherein each one of the one or more manipulator robots is disposed within a respective single grid space.

15. The robotic system of claim 13 , further comprising a pneumatic supply line coupling at least one of the one or more manipulator robots to an external pneumatic source, wherein the pneumatic supply line extends at least partially through the storage structure.

16. The robotic system of claim 13 , wherein the gripping tool comprises a plurality of gripping elements.

17. The robotic system of claim 13 , further comprising a tool holder provided on at least one of the one or more manipulator robots or coupled to the frame, the tool holder being arranged to hold another gripping tool.

18. The robotic system of claim 17 , wherein the manipulation task instructions include at least one of a partial pose for the gripping tool, a grasping region of the inventory item, or a selection of the gripping tool or the another gripping tool.

19. A method for assisting a manipulator robot installed at a station on a grid-based storage structure with a manipulation task, the method comprising:

receiving, by a central computing system, data corresponding to the manipulation task, the manipulation task including picking an inventory item from a transported storage container;

transmitting, by the central computing system to a teleoperator interface, and after receiving the data, one or more images of the inventory item within the storage container disposed within the grid-based storage structure, for assisting the manipulator robot with the manipulation task;

receiving, by the central computing system after transmitting the one or more images, instructions from the teleoperator interface, the instructions including at least one of a partial pose for a gripping tool of the manipulator robot, an identified grasping region of the inventory item, or a selection of a gripping tool to be used by the manipulator robot in performing the manipulation task; and

transmitting, by the central computing system, manipulation task instructions to the manipulator robot for performing the manipulation task.

20. The method of claim 19 , wherein the manipulation task further includes placing the inventory item in a selected orientation within an order container.

21. The method of claim 19 , further comprising:

coordinating, by the central computing system, operation of one or more container handling robots installed upon the grid-based storage structure, the operation of the one or more container handling robots including movement along a grid of the grid-based storage structure and/or extraction of the storage container and transportation of the extracted storage container to the station.

22. The method of claim 19 , wherein the data comprises product information, grasp success rate incurred when attempting to grasp a respective inventory item, or a level of difficulty in grasping the respective inventory item.

23. A robotic system, comprising:

a storage structure, comprising:

a frame including vertical members supporting a grid having a first set of parallel rails extending in a first direction and a second set of parallel rails extending in a second direction substantially perpendicular to the first direction such that the first and second set of parallel rails collectively define grid spaces; and

a plurality of storage containers arranged in vertical stacks for storing inventory items, each one of the vertical stacks configured to be arranged within a respective one of the grid spaces;

a container handling robot installed upon the grid to move along the first and second set of parallel rails, the container handling robot including a device extendable in a vertical direction to extract one or more storage containers from the vertical stacks and transport the extracted storage container to a station;

a manipulator robot installed at the station on the grid, the manipulator robot including a picking arm provided with a pneumatic gripping tool arranged to pick an inventory item from the storage container located at the station and place the picked item into an order container; and

a pneumatic supply line coupling the manipulator robot to an external pneumatic source, the pneumatic supply line extending at least partially through the storage structure.

24. The robotic system of claim 23 , wherein the pneumatic supply line is located at least partially external of the frame.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2023
From: KALOUCHE, SIMON
To: NIMBLE ROBOTICS, INC.
Reel/Frame 065105/0084 →
Continuity (6)
Continuation 16856409 · Apr 23, 2020
Continuation 16831963 · Mar 27, 2020
Continuation 16804251 · Feb 28, 2020
Provisional Application 62961390 · Jan 15, 2020
Provisional Application 62879843 · Jul 29, 2019
Related Publication 20240025030A1 · Jan 25, 2024
References Cited (131)
US 4660405A · Widegren et al. · 1987 [cited by applicant]
US 4809747A · Choly et al. · 1989 [cited by applicant]
US 4836111A · Kaufmann · 1989 [cited by applicant]
US 5993365A · Stagnitto et al. · 1999 [cited by applicant]
US 6270017B1 · Vennell · 2001 [cited by applicant]
US 6654662B1 · Hognaland · 2003 [cited by applicant]
US 7181314B2 · Zhang et al. · 2007 [cited by applicant]
US 8240726B2 · Subotincic · 2012 [cited by applicant]
US 8434414B2 · Wessel et al. · 2013 [cited by applicant]
US 8718814B1 · Clark et al. · 2014 [cited by applicant]
US 9465386B2 · Huang et al. · 2016 [cited by applicant]
US 9486926B2 · Kawano · 2016 [cited by applicant]
US 9540171B2 · Elazary et al. · 2017 [cited by applicant]
US 9550624B2 · Khodl et al. · 2017 [cited by applicant]
US 9623569B2 · McCollum et al. · 2017 [cited by applicant]
US 9650215B2 · Girtman · 2017 [cited by applicant]
US 9669543B1 · Stubbs et al. · 2017 [cited by applicant]
US 9682822B2 · Lindbo et al. · 2017 [cited by applicant]
US 9688472B1 · Stubbs et al. · 2017 [cited by applicant]
US 9785911B2 · Galluzzo et al. · 2017 [cited by applicant]
US 9802317B1 · Watts et al. · 2017 [cited by applicant]
US 9815625B2 · DeWitt et al. · 2017 [cited by applicant]
US 9940604B2 · Galluzzo et al. · 2018 [cited by applicant]
US 9975256B1 · Swiercz · 2018 [cited by applicant]
US 9987748B2 · Stubbs et al. · 2018 [cited by applicant]
US 10000337B2 · Lindbo et al. · 2018 [cited by applicant]
US 10086519B2 · Wagner et al. · 2018 [cited by applicant]
US 10112771B2 · D'Andrea et al. · 2018 [cited by applicant]
US 10147069B2 · Galluzzo et al. · 2018 [cited by applicant]
US 10189641B2 · Hognaland · 2019 [cited by applicant]
US 10272566B2 · Wellman et al. · 2019 [cited by applicant]
US 10315315B2 · Wagner et al. · 2019 [cited by applicant]
US 10322506B2 · Ding et al. · 2019 [cited by applicant]
US 10350755B2 · Wagner et al. · 2019 [cited by applicant]
US 10360531B1 · Stallman et al. · 2019 [cited by applicant]
US 10435241B2 · Lert et al. · 2019 [cited by applicant]
US 10438034B2 · Wagner et al. · 2019 [cited by applicant]
US 10500735B1 · Menon et al. · 2019 [cited by applicant]
US 10549928B1 · Chavez et al. · 2020 [cited by applicant]
US 10576507B2 · Jain et al. · 2020 [cited by applicant]
US 10583560B1 · Rodrigues et al. · 2020 [cited by applicant]
US 10611037B1 · Polido et al. · 2020 [cited by applicant]
US 10759597B2 · Lindbo et al. · 2020 [cited by applicant]
US 11220400B2 · Lisso · 2022 [cited by applicant]
US 11407118B1 · Augenbraun · 2022 [cited by examiner]
US 11534923B1 · De Arruda Camargo Polido · 2022 [cited by applicant]
US 20060112034A1 · Okamoto et al. · 2006 [cited by applicant]
US 20080003092A1 · Baclija · 2008 [cited by examiner]
US 20080181759A1 · Gaegauf · 2008 [cited by applicant]
US 20080229861A1 · Inoue · 2008 [cited by examiner]
US 20130076891A1 · Childress · 2013 [cited by examiner]
US 20130110280A1 · Folk · 2013 [cited by applicant]
US 20140014470A1 · Razumov · 2014 [cited by applicant]
US 20150098775A1 · Razumov · 2015 [cited by applicant]
US 20150230369A1 · Danek · 2015 [cited by examiner]
US 20150241872A1 · Huang · 2015 [cited by examiner]
US 20160167228A1 · Wellman et al. · 2016 [cited by applicant]
US 20160272421A1 · Hognaland · 2016 [cited by applicant]
US 20160297610A1 · Grosse · 2016 [cited by applicant]
US 20160347545A1 · Lindbo et al. · 2016 [cited by applicant]
US 20160361821A1 · Lessing et al. · 2016 [cited by applicant]
US 20170067488A1 · Cho · 2017 [cited by examiner]
US 20170080579A1 · Wagner et al. · 2017 [cited by applicant]
US 20170136632A1 · Wagner et al. · 2017 [cited by applicant]
US 20170157649A1 · Wagner et al. · 2017 [cited by applicant]
US 20170197316A1 · Wagner et al. · 2017 [cited by applicant]
US 20170313514A1 · Lert, Jr. et al. · 2017 [cited by applicant]
US 20180029797A1 · Hance et al. · 2018 [cited by applicant]
US 20180057263A1 · Beer · 2018 [cited by applicant]
US 20180127219A1 · Wagner et al. · 2018 [cited by applicant]
US 20180133968A1 · Zeulner · 2018 [cited by applicant]
US 20180148259A1 · Gravelle et al. · 2018 [cited by applicant]
US 20180251300A1 · Kapuria et al. · 2018 [cited by applicant]
US 20180265291A1 · Wagner et al. · 2018 [cited by applicant]
US 20180265311A1 · Wagner et al. · 2018 [cited by applicant]
US 20180284760A1 · Gupta et al. · 2018 [cited by applicant]
US 20180319590A1 · Lindbo et al. · 2018 [cited by applicant]
US 20180346243A1 · Lindbo et al. · 2018 [cited by applicant]
US 20180354723A1 · Peebles et al. · 2018 [cited by applicant]
US 20190009414A1 · Lipay et al. · 2019 [cited by applicant]
US 20190026687A1 · Voloskov · 2019 [cited by applicant]
US 20190084152A1 · Deacon · 2019 [cited by applicant]
US 20190135555A1 · Wagner et al. · 2019 [cited by applicant]
US 20190176348A1 · Bingham et al. · 2019 [cited by applicant]
US 20190217477A1 · Paepcke et al. · 2019 [cited by applicant]
US 20190245366A1 · Coady et al. · 2019 [cited by applicant]
US 20190291277A1 · Oleynik · 2019 [cited by applicant]
US 20190291282A1 · Marchese et al. · 2019 [cited by applicant]
US 20190291955A1 · Bastian, II · 2019 [cited by applicant]
US 20190337733A1 · Wehner et al. · 2019 [cited by applicant]
US 20190339693A1 · Menon et al. · 2019 [cited by applicant]
US 20190361672A1 · Odhner et al. · 2019 [cited by applicant]
US 20190389082A1 · Higo · 2019 [cited by applicant]
US 20200017314A1 · Rose et al. · 2020 [cited by applicant]
US 20200017317A1 · Yap · 2020 [cited by examiner]
US 20200030994A1 · Wagner · 2020 [cited by examiner]
US 20200031575A1 · Hognaland · 2020 [cited by applicant]
US 20200094997A1 · Menon et al. · 2020 [cited by applicant]
US 20200164517A1 · Dick et al. · 2020 [cited by applicant]
US 20200189122A1 · Polido · 2020 [cited by examiner]
US 20200290804A1 · Fjeldheim et al. · 2020 [cited by applicant]
US 20200354147A1 · Lindbo · 2020 [cited by examiner]
US 20200363259A1 · Bergstra et al. · 2020 [cited by applicant]
US 20200407165A1 · Roth et al. · 2020 [cited by applicant]
US 20210031364A1 · Groz · 2021 [cited by applicant]
US 20210101744A1 · Hognaland · 2021 [cited by applicant]
US 20210122054A1 · Sun et al. · 2021 [cited by applicant]
US 20210163223A1 · Lindbo et al. · 2021 [cited by applicant]
US 20210171282A1 · Solovianenko · 2021 [cited by applicant]
US 20220002076A1 · Austrheim · 2022 [cited by applicant]
US 20220002077A1 · Austrheim et al. · 2022 [cited by applicant]
US 20220073279A1 · Austrheim et al. · 2022 [cited by applicant]
US 20220193925A1 · Honarpardaz · 2022 [cited by examiner]
CN 103122771B · 2015 [cited by applicant]
DE 102010002317A1 · 2011 [cited by applicant]
EP 2708335A1 · 2014 [cited by applicant]
JP 2017132002A · 2017 [cited by applicant]
KR 20170052784A · 2017 [cited by applicant]
NO 317366B1 · 2004 [cited by applicant]
WO 2012083333A2 · 2012 [cited by applicant]
WO 2014040843A1 · 2014 [cited by applicant]
WO 2015019055A1 · 2015 [cited by applicant]
WO 2016063197A1 · 2016 [cited by applicant]
WO 2018146687A1 · 2018 [cited by applicant]
WO 2018234155A1 · 2018 [cited by applicant]
WO 2019084457A9 · 2019 [cited by applicant]
WO 2020018468A1 · 2020 [cited by applicant]
WO 2020023861A1 · 2020 [cited by applicant]
Partial International Search Report including Provisional Opinion for Application No. PCT/US2020/043873, mailed Nov. 2, 2020, pp. 1-37. [cited by applicant]
International Search Report and Written Opinion in Appln. No. PCT/US2022/042896 mailed Dec. 9, 2022 (19 pages). [cited by applicant]
Petri Honkamaa et al : “A lightweight approach for augmented reality on camera phones using 2D images to simulate 3D”, Proceedings of the 6th International Conference on Mobile and Ubiquitous Multimedia, MUM 2007, Oulu … [cited by applicant]