IP Library › Granted Patent US 12,491,646
Granted Patent B2
US 12,491,646 · App. 18/376,890 · Granted Dec 9, 2025

Systems and methods for providing dynamic communicative lighting in a robotic environment

Inventors: Thomas Wagner (Concord, MA); Kevin Ahearn (Nebo, NC); Michael Dawson-Haggerty (Pittsburgh, PA); Christopher Geyer (Arlington, MA); Thomas Koletschka (Somerville, MA); Kyle Maroney (Saunderstown, RI); Matthew T. Mason (Atlanta, GA); Gene Temple Price (Somerville, MA); Joseph Romano (Arlington, MA); Daniel Smith (Wexford, PA); Siddhartha Srinivasa (Seattle, WA); Prasanna Velagapudi (Pittsburgh, PA); Thomas Allen (Reading, MA)
Assignee: Berkshire Grey Operating Company, Inc.
B25J19/06B25J9/1676G08B5/36G05B2219/40202
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,491,646
App. No.
18/376,890
Granted
Dec 9, 2025
Kind
B2
Abstract

A robotic system is disclosed that includes an articulated arm with an end effector. The robotic system is for use in a robotic environment requiring interaction with persons in the robotic environment, and includes a plurality of lights that are illuminated responsive to known near-future movements of the articulated arm to convey the known near-future movements of the articulated arm to the persons in the robot environment.

Claims (47)

1 . A robotic system comprising:

an articulated arm with an end effector;

a first plurality of lights proximate the end effector; and

a second plurality of lights proximate a plurality of destination bins,

said first plurality of lights being engageable in a grasp quality mode to indicate a grasp quality of the end effector's grasp on an object, and the first plurality of lights being further engageable in a movement destination mode to indicate a planned destination of the end effector by illuminating one or more of the first plurality of lights in association with one or more of the second plurality of lights provided proximate a plurality of destination bins,

each of the second plurality of lights being associated with one of the plurality of destination bins, said second plurality of lights being engageable in a destination bin mode to indicate the association with the one or more of the first plurality of lights, and each of the second plurality of lights being further engageable in a bin complete mode to indicate that a particular destination bin is ready for removal.

2 . The robotic system of claim 1 , wherein the first plurality of lights is further engageable in a movement direction mode to indicate a planned direction of movement of the end effector by illuminating one or more of the first plurality of lights that are on a side of a ring that is associated with the planned direction of movement of the end effector.

3 . The robotic system of claim 1 , wherein the first plurality of lights is further engageable in an unknown object mode to indicate that information regarding to where the object is to be moved is not known.

4 . The robotic system of claim 1 , wherein the first plurality of lights includes multi-colored lights.

5 . The robotic system of claim 1 , wherein the one or more of the first plurality of lights that are illuminated in association with the one or more of the second plurality of lights are illuminated in a flashing mode.

6 . The robotic system of claim 1 , wherein the second plurality of lights include multi-colored lights.

7 . The robotic system of claim 1 , wherein the one or more of the first plurality of lights that are illuminated in association with the one or more of the second plurality of lights are illuminated in a same color.

8 . The robotic system of claim 1 , wherein the robotic system further includes a third plurality of lights that are engageable in a processing mode to indicate that a processing station that includes the robotic system is engaged in processing a plurality of objects, and the third plurality of lights is further engageable in an alert mode to indicate that an error has occurred at the processing station.

9 . The robotic system as claimed in claim 8 , wherein the third plurality of lights include multi-colored lights.

10 . The robotic system as claimed in claim 8 , wherein the third plurality of lights include flashing lights.

11 . A robotic system comprising:

an articulated arm with an end effector; and

a first plurality of lights proximate the end effector and a second plurality of lights proximate a plurality of destination bins,

said first plurality of lights including at least a first light that is engageable in a grasp mode to indicate a grasp quality of a grasp of an object by the end effector;

said first plurality of lights and said second plurality of lights each being engageable in an association mode such that at least one of the first plurality of lights and at least one of the second plurality of lights are both engaged in association to indicate that the articulated arm is planning to move the object grasped by the end effector to a selected destination bin of the plurality of destination bins, said at least one of the second plurality of lights being associated with the selected destination bin; and

said second plurality of lights including a plurality of bin status lights, each bin status light being associated with a respective one of the plurality of destination bins, each bin status light being engageable in a status operational mode to indicate that a destination bin among the plurality of destination bins is open to receiving any of a plurality of objects, and each bin status light being further engageable in a status completed mode to indicate that the destination bin is completed and ready to be further processed.

12 . The robotic system of claim 11 , wherein the first plurality of lights is further engageable in a movement direction mode to indicate a planned direction of movement of the end effector by illuminating one or more of the first plurality of lights that are on a side of a ring that is associated with the planned direction of movement of the end effector.

13 . The robotic system of claim 11 , wherein the first plurality of lights is further engageable in an unknown object mode to indicate that information regarding to where the object is to be moved is not known.

14 . The robotic system of claim 11 , wherein the first plurality of lights includes multi-colored lights.

15 . The robotic system of claim 11 , wherein the at least one of the first plurality of lights that are illuminated in association with the at least one of the second plurality of lights are illuminated in a flashing mode.

16 . The robotic system of claim 11 , wherein the second plurality of lights include multi-colored lights.

17 . The robotic system of claim 11 , wherein the at least one of the first plurality of lights that are illuminated in association with the at least one of the second plurality of lights are illuminated in a same color.

18 . The robotic system of claim 11 , wherein the robotic system further includes a third plurality of lights that are engageable in a processing mode to indicate that a processing station that includes the robotic system is engaged in processing a plurality of objects, and the third plurality of lights is further engageable in an alert mode to indicate that an error has occurred at the processing station.

19 . The robotic system as claimed in claim 18 , wherein the third plurality of lights include multi-colored lights.

20 . The robotic system as claimed in claim 18 , wherein the third plurality of lights include flashing lights.

21 . A robotic system comprising:

an articulated arm with an end effector;

a first plurality of lights proximate the end effector;

a second plurality of lights proximate a plurality of destination bins; and

a third plurality of lights associated with an object processing station that includes the articulated arm and the plurality of destination bins,

said first plurality of lights including at least a first light that is engageable in a grasp mode to indicate a grasp quality of a grasp of an object by the end effector;

said first plurality of lights and said second plurality of lights each being engageable in an association mode such that at least one of the first plurality of lights and at least one of the second plurality of lights are both engaged in association to indicate that the articulated arm is planning to move the object grasped by the end effector to a selected destination bin of the plurality of destination bins;

said second plurality of lights including a plurality of bin status lights, each bin status light being associated with a respective one of the plurality of destination bins, each bin status light being engageable in a status completed mode to indicate that the destination bin among the plurality of destination bins is completed and ready to be further processed; and

said third plurality of lights being engageable in an alert mode to indicate that an error has occurred at the object processing station.

22 . The robotic system of claim 21 , wherein the first plurality of lights is further engageable in a movement direction mode to indicate a planned direction of movement of the end effector by illuminating one or more of the first plurality of lights that are on a side of a ring that is associated with the planned direction of movement of the end effector.

23 . The robotic system of claim 21 , wherein the first plurality of lights is further engageable in an unknown object mode to indicate that information regarding to where the object is to be moved is not known.

24 . The robotic system of claim 21 , wherein the first plurality of lights includes multi-colored lights.

25 . The robotic system of claim 21 , wherein the one or more of the first plurality of lights that are illuminated in association with the at least one of the second plurality of lights are illuminated in a flashing mode.

26 . The robotic system of claim 21 , wherein the second plurality of lights include multi-colored lights.

27 . The robotic system of claim 21 , wherein the at least one of the first plurality of lights that are illuminated in association with the one or more of the second plurality of lights are illuminated in a same color.

28 . The robotic system as claimed in claim 21 , wherein the third plurality of lights include multi-colored lights.

29 . The robotic system as claimed in claim 21 , wherein the third plurality of lights include flashing lights.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: WAGNER, THOMAS; AHEARN, KEVIN; DAWSON-HAGGERTY, MICHAEL; GEYER, CHRISTOPHER; KOLETSCHKA, THOMAS; MARONEY, KYLE; MASON, MATTHEW T.; PRICE, GENE TEMPLE; ROMANO, JOSEPH; SMITH, DANIEL; SRINIVASA, SIDDHARTHA; VELAGAPUDI, PRASANNA; ALLEN, THOMAS
To: BERKSHIRE GREY OPERATING COMPANY, INC.
Reel/Frame 065849/0354 →
Continuity (6)
Continuation 17395929 · Aug 6, 2021
Continuation 16826819 · Mar 23, 2020
Continuation 16243753 · Jan 9, 2019
Continuation 15259961 · Sep 8, 2016
Provisional Application 62216017 · Sep 9, 2015
Related Publication 20240033951A1 · Feb 1, 2024
References Cited (124)
US 4412775A · Molitor et al. · 1983 [cited by applicant]
US 4557659A · Scaglia · 1985 [cited by applicant]
US 4704694A · Czerniejewski · 1987 [cited by applicant]
US 4846335A · Hartlepp · 1989 [cited by applicant]
US 4896357A · Hatano et al. · 1990 [cited by applicant]
US 5328169A · Mandel · 1994 [cited by examiner]
US 5794789A · Payson et al. · 1998 [cited by applicant]
US 5865487A · Gore et al. · 1999 [cited by applicant]
US 5881890A · Wiley · 1999 [cited by applicant]
US 6079570A · Oppliger et al. · 2000 [cited by applicant]
US 6520342B1 · Dumke · 2003 [cited by examiner]
US 6865487B2 · Charron · 2005 [cited by applicant]
US 7313464B1 · Perreault et al. · 2007 [cited by applicant]
US 7474939B2 · Oda et al. · 2009 [cited by applicant]
US 7516848B1 · Shakes et al. · 2009 [cited by applicant]
US 7677622B2 · Dunkmann et al. · 2010 [cited by applicant]
US 7861844B2 · Hayduchok et al. · 2011 [cited by applicant]
US 8718814B1 · Clark et al. · 2014 [cited by applicant]
US 8874270B2 · Ando · 2014 [cited by applicant]
US 8952284B1 · Wong et al. · 2015 [cited by applicant]
US 9061868B1 · Paulsen et al. · 2015 [cited by applicant]
US 9120622B1 · Elazary et al. · 2015 [cited by applicant]
US 9227323B1 · Konolige et al. · 2016 [cited by applicant]
US 9259844B2 · Xu et al. · 2016 [cited by applicant]
US 9492923B2 · Wellman et al. · 2016 [cited by applicant]
US 9604363B2 · Ban · 2017 [cited by applicant]
US 10265872B2 · Wagner et al. · 2019 [cited by applicant]
US 10632631B2 · Wagner et al. · 2020 [cited by applicant]
US 11117271B2 · Wagner et al. · 2021 [cited by applicant]
US 20010045755A1 · Schick et al. · 2001 [cited by applicant]
US 20010056316A1 · Johnson et al. · 2001 [cited by applicant]
US 20020027652A1 · Paromtchik et al. · 2002 [cited by applicant]
US 20030135300A1 · Lewis · 2003 [cited by applicant]
US 20060177295A1 · Frueh et al. · 2006 [cited by applicant]
US 20060242785A1 · Cawley et al. · 2006 [cited by applicant]
US 20070005179A1 · Mccrackin et al. · 2007 [cited by applicant]
US 20080179224A1 · Van Bossuyt · 2008 [cited by applicant]
US 20080181485A1 · Beis et al. · 2008 [cited by applicant]
US 20090019818A1 · Gilmore et al. · 2009 [cited by applicant]
US 20100125361A1 · Mougin et al. · 2010 [cited by applicant]
US 20100180711A1 · Kilibarda et al. · 2010 [cited by applicant]
US 20100234857A1 · Itkowitz et al. · 2010 [cited by applicant]
US 20110054689A1 · Nielsen et al. · 2011 [cited by applicant]
US 20110144798A1 · Freudelsperger · 2011 [cited by applicant]
US 20110160910A1 · Preisinger et al. · 2011 [cited by applicant]
US 20110176148A1 · Briggs et al. · 2011 [cited by applicant]
US 20110184555A1 · Kosuge et al. · 2011 [cited by applicant]
US 20110320036A1 · Freudelsperger · 2011 [cited by applicant]
US 20120215346A1 · Gingher et al. · 2012 [cited by applicant]
US 20130232918A1 · Lomerson, Jr. · 2013 [cited by applicant]
US 20130245824A1 · Barajas et al. · 2013 [cited by applicant]
US 20130343640A1 · Buehler et al. · 2013 [cited by applicant]
US 20140005831A1 · Naderer et al. · 2014 [cited by applicant]
US 20140067121A1 · Brooks et al. · 2014 [cited by applicant]
US 20140067127A1 · Gotou · 2014 [cited by applicant]
US 20140195979A1 · Branton et al. · 2014 [cited by applicant]
US 20140291112A1 · Lyon et al. · 2014 [cited by applicant]
US 20140305847A1 · Kudrus · 2014 [cited by applicant]
US 20140316570A1 · Sun et al. · 2014 [cited by applicant]
US 20150057793A1 · Kawano · 2015 [cited by applicant]
US 20150073589A1 · Khodl et al. · 2015 [cited by applicant]
US 20150081090A1 · Dong · 2015 [cited by applicant]
US 20150217937A1 · Marquez · 2015 [cited by applicant]
US 20150224650A1 · Xu et al. · 2015 [cited by applicant]
US 20150369618A1 · Barnard et al. · 2015 [cited by applicant]
US 20150375401A1 · Dunkmann et al. · 2015 [cited by applicant]
US 20160027093A1 · Crebier · 2016 [cited by applicant]
US 20160136816A1 · Pistorino · 2016 [cited by applicant]
US 20160199884A1 · Lykkegaard et al. · 2016 [cited by applicant]
US 20160243704A1 · Vakanski et al. · 2016 [cited by applicant]
US 20160244262A1 · O'Brien et al. · 2016 [cited by applicant]
US 20170043953A1 · Battles et al. · 2017 [cited by applicant]
US 20170050315A1 · Henry et al. · 2017 [cited by applicant]
US 20170066597A1 · Hiroi · 2017 [cited by applicant]
US 20170080566A1 · Stubbs et al. · 2017 [cited by applicant]
US 20170087731A1 · Wagner et al. · 2017 [cited by applicant]
US 20170106532A1 · Wellman et al. · 2017 [cited by applicant]
US 20170276472A1 · Becker et al. · 2017 [cited by applicant]
US 20190143543A1 · Wagner et al. · 2019 [cited by applicant]
US 20200223075A1 · Wagner et al. · 2020 [cited by applicant]
US 20210362356A1 · Wagner et al. · 2021 [cited by applicant]
US 20220144545A1 · Harder · 2022 [cited by applicant]
CA 2998403C · 2021 [cited by applicant]
CH 701886A2 · 2011 [cited by applicant]
CN 104870147A · 2015 [cited by applicant]
CN 108349096A · 2018 [cited by applicant]
CN 114248272A · 2022 [cited by applicant]
DE 102007038834A1 · 2009 [cited by applicant]
DE 102013020137A1 · 2015 [cited by applicant]
EP 0613841A1 · 1994 [cited by applicant]
EP 1256421A1 · 2008 [cited by applicant]
EP 1995192A2 · 2008 [cited by applicant]
EP 2960024A2 · 2015 [cited by applicant]
EP 3006379A2 · 2016 [cited by applicant]
EP 3347175B1 · 2022 [cited by applicant]
EP 3988263B1 · 2024 [cited by applicant]
JP H09244730A · 1997 [cited by applicant]
WO 2010034044A2 · 2010 [cited by applicant]
WO 2010099873A1 · 2010 [cited by applicant]
WO 2014111483A1 · 2014 [cited by applicant]
WO 2015162390A1 · 2015 [cited by applicant]
WO 2017044630A1 · 2017 [cited by applicant]
Decision on Rejection issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 202210026563.2 on Apr. 29, 2024, 21 pages. [cited by applicant]
Cipolla et al., “Visually Guided Grasping in Unstructured Environments,” J. of Robotics and Autonomous Sys., pp. 1-20. [cited by applicant]
Communication pursuant to Rules 161(1) and 162EPC issued by the European Patent Office on Apr. 17, 2018 in related European Patent Application No. 16767467.0, 3 pages. [cited by applicant]
Examiner's Report issued by the Canadian Intellectual Property Office on May 16, 2019 in related Canadian Patent Application No. 2,998,403, 3 pages. [cited by applicant]
Extended European Search Report issued by the European Patent Office in related European Patent Application No. 21214194.9 on Mar. 4, 2022, 5 pages. [cited by applicant]
Final Office Action issued by the U.S. Patent and Trademark Office on Feb. 13, 2018 in related U.S. Appl. No. 15/259,961, 33 pages. [cited by applicant]
First Office Action, and its English translation, issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 201680062748.1 on Jul. 22, 2020, 20 pages. [cited by applicant]
Herbert et al., “A Robotic Gripper System for Limp Material Manipulation: Hardware and Software Development and Integration,” Proc. of the 1997 IEEE, Int'l Conf. on Robotics and Automation, Apr. 1997, pp. 15-21. [cited by applicant]
International Preliminary Report on Patentability issued by the International Bureau of WIPO on Mar. 13, 2018 for International Application No. PCT/US2016/050786, 7 pages. [cited by applicant]
International Search Report and Written Opinion issued by the International Searching Authority on Nov. 24, 2016 for International Application No. PCT/US2016/050786, 9 pages. [cited by applicant]
Klingbeil et al., “Grasping with Application to an Autonomous Checkout Robot,” Proc. of IEEE, Int'l Conf. on Robotics and Automation, Jun. 2011, 9 pages. [cited by applicant]
Liu et al., “Hand Arm Coordination for a Tomato Harvesting Robot Based on Commercial Manipulator,” Proc. of the IEEE, Int'l Conf. on Robotics and Biometrics, Dec. 2013, pp. 2715-2720. [cited by applicant]
Non-Final Office Action issued by the U.S. Patent and Trademark Office on Jul. 10, 2017 in related U.S. Appl. No. 15/259,961, 31 pages. [cited by applicant]
Non-Final Office Action issued by the U.S. Patent and Trademark Office on Aug. 21, 2018 in related U.S. Appl. No. 15/259,961, 31 pages. [cited by applicant]
Non-Final Office Action issued by the U.S. Patent and Trademark Office on Dec. 20, 2019 in related U.S. Appl. No. 16/243,753, 35 pages. [cited by applicant]
Non-Final Office Action issued by the U.S. Patent and Trademark Office in related U.S. Appl. No. 16/826,819 on May 5, 2021, 16 pages. [cited by applicant]
Non-Final Office Action issued by the United States Patent and Trademark Office in related U.S. Appl. No. 17/395,929 on Apr. 7, 2023, 37 pages. [cited by applicant]
Notice on the First Office Action, along with its English translation, issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 202210026563.2 on Mar. 31, 2023, 25 page… [cited by applicant]
Notice on the Second Office Action, along with its English translation, issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 202210026563.2 on Dec. 19, 2023, 21 pag… [cited by applicant]
Notice on the Third Office Action and the Third Office Action, along with its English translation, issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 201680062748… [cited by applicant]
Second Office Action, and its English translation, issued by the China National Intellectual Property Administration in related Chinese Patent Application No. 201680062748.1 on Feb. 20, 2021, 18 pages. [cited by applicant]
Vitton et al., “A Flexible Robotic Gripper for Automation of Assembly Tasks,” Proc. of the ASME—Dynamic Systems & Control Div., 2003, vol. 2, 7 pages. [cited by applicant]