Granted Patent B2

US 12,617,616 · App. 18/363,268 · Granted May 5, 2026

Inventors: Mark R Cass (Boise, ID); Amey Sunil Kulkarni (Boise, ID); Shelby Lynne Cass (Corvallis, OR); Eric Michael Smith (Dallas, TX); Paola Andrea Gutierrez Guzman (Boise, ID); Ruthvik Vaila (Boise, ID)

Assignee: Bastian Solutions, LLC

B65G1/1373B25J9/1612B25J9/1697B25J19/023

Abstract

Claims (64)

1 . A system, comprising:

a robotic station including a robot configured to pick from and/or place into a tote one or more items;

a robot camera positioned proximal to the robot to capture one or more images of the items in the tote;

a computer configured to determine if the tote is pickable by the robot based on the images from the robot camera;

an operator station located remote from the robotic station;

an operator camera located at the operator station to monitor the operator station;

wherein the computer is configured to determine when a human operator at the operator station is idle with the operator camera;

wherein the computer is configured to route the tote from the robotic station to the operator station when the human operator is idle and the tote is unpickable by the robot;

wherein the operator station is configured to facilitate manual rearrangement of the items in the tote by the human operator; and

wherein the computer is configured to route the tote back from the operator station to the robotic station after determining the items in the tote are pickable with the operator camera.

2 . The system of claim 1 , further comprising:

wherein the computer is configured to monitor tote rearrangement via the operator camera;

an input/output (I/O) device including an indicator of proper tote arrangement; and

wherein the I/O device is configured to provide instructions for proper tote arrangement to the human operator.

3 . The system of claim 1 , wherein:

the computer includes an artificial intelligence (AI) system;

the AI system is configured to determine if the tote is pickable by the robot based on the images from the robot camera; and

the AI system is configured to learn operations from captured image data.

4 . The system of claim 3 , wherein the AI system is configured to learn how to pick from and/or place into the tote the items based on the captured image data.

5 . The system of claim 1 , wherein:

the computer is configured to rank totes based on probability of successfully picking items from the totes; and

the computer is configured to route totes to the robot based on the rankings.

6 . The system of claim 1 , further comprising:

a network; and

wherein the computer is configured to distribute previously captured imaging to the robot via the network.

7 . The system of claim 1 , further comprising:

a second camera aimed towards the same items in the tote.

8 . The system of claim 1 , wherein:

the computer is configured to determine the items in the tote at the operator station are pickable with the operator camera; and

the computer is configured to activate an indicator when the items in the tote at the operator station are pickable.

9 . The system of claim 1 , wherein:

the computer includes an artificial intelligence (AI) system;

the AI system is configured to learn from the manual rearrangement of the items in the tote by the human operator captured by the operator camera; and

the computer via learning by the AI system from the manual rearrangement is configured to change operation of the robot to enhance picking of the items.

10 . A method, comprising:

capturing an image of a tote with a robot camera positioned proximal to a robot at a robotic station;

determining the tote is unpickable by the robot based on the image with a computer;

monitoring a human operator at an operator station with an operator camera;

determining the human operator is idle at the operator station via the computer as a result of the monitoring;

routing the tote from the robotic station to the operator station in response to the determining the tote in unpickable and the determining the human operator is idle;

wherein the operator station is located remote from the robotic station;

wherein the operator station is configured to facilitate manual rearrangement of the tote by the human operator; and

routing the tote from the operator station to the robotic station after the human operator manually rearranges the tote to be pickable.

11 . The method of claim 10 , further comprising:

wherein the tote contains one or more items; and

picking at least one of the items in the tote with the robot.

12 . The method of claim 10 , further comprising:

determining a chance of success of the robot rearranging items in the tote is above a threshold with the computer; and

attempting to rearrange the items in the tote using the robot before the routing the tote to the human operator.

13 . The method of claim 10 , further comprising:

determining the tote ranks ahead other totes with the computer based on probability of picking success; and

routing the tote to the robot in response to the determining the tote ranks ahead other totes.

14 . A method, comprising:

determining a first tote is unpickable by a first robot at a robotic station with a robot camera;

routing the first tote to an operator station in response to the determining the first tote is unpickable;

capturing one or more images of a human operator manually rearranging the first tote by hand with an operator camera at the operator station;

routing the first tote from the operator station to the robotic station after the human operator manually rearranges the first tote to be pickable;

distributing the images to one or more computers using a network;

training at least one artificial intelligence (AI) system on at least one of the computers to perform a manipulation of one or more items using the images; and

rearranging the items in a second tote with a second robot based on the training.

15 . The method of claim 14 , wherein the rearranging of the items includes picking and/or placing the items in the second tote with the second robot.

16 . The method of claim 14 , further comprising routing the first tote to the human operator who is idle.

17 . The method of claim 14 , further comprising:

labeling the images associated with the manipulation with the AI system.

Assignments (2)

Continuity (1)

References Cited (20)