IP Library › Granted Patent US 10,814,489
Granted Patent B1
US 10,814,489 · App. 16/804,277 · Granted Oct 27, 2020

System and method of integrating robot into warehouse management software

Inventors: Simon Kalouche (San Francisco, CA); Siva Chaitanya Mynepalli (San Francisco, CA); John Vashchhenko (Apple Valley, CA); Suyash Nigam (San Francisco, CA); Jordan Dawson (San Francisco, CA); George Marchman (San Francisco, CA)
Assignee: Nimble Robotics, Inc.
B25J9/1697B25J9/161B25J9/1679B25J13/08B25J19/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,814,489
App. No.
16/804,277
Granted
Oct 27, 2020
Kind
B1
Abstract

A parsing system includes an imaging device and one or more processors. The imaging device may capture visual instructions displayed on a monitor. The one or more processors may be configured to receive, from the imaging device, the captured visual instructions, generate functions for a robot to perform based on the captured visual instructions, and transmit, to a robot, the functions to perform.

Claims (55)

1. A parsing system, comprising:

an imaging device configured to capture visual instructions provided by a warehouse or fulfillment center system and displayed on a monitor, the warehouse or fulfillment center system being distinct from the parsing system; and

one or more processors, wherein the one or more processors are configured to:

receive, from the imaging device, captured visual instructions;

perform visual and/or optical character recognition on the received visual instructions to determine pick and place instructions included in the received visual instructions;

generate functions for a robot to perform based on the determined pick and place instructions, the functions configured to cause the robot to perform the determined pick and place instructions;

transmit, to a robot, the functions to perform; and

transmit a response indicating completion of the determined pick and place instructions.

2. The parsing system of claim 1 , further comprising:

the robot, wherein the robot includes a picking arm configured to grasp an item from a first location and place the item in a second location.

3. The parsing system of claim 2 , wherein the robot is further configured to:

receive the functions to perform from the one or more processors; and

perform the received functions.

4. The parsing system of claim 3 , wherein:

the robot is further configured to provide a confirmation of completion of the received functions to the one or more processors after performing the received functions, and

the one or more processors are further configured to transmit the response after receiving the confirmation of completion from the robot.

5. The parsing system of claim 1 , wherein the parsing system further includes a physical button-pushing device, wherein the physical button-pushing device pushes a button connected to the warehouse or fulfillment center system to provide a completion response to the warehouse or fulfillment center system.

6. The parsing system of claim 1 , wherein the parsing system further includes a custom device, wherein the custom device is configured to emulate signals generated by input devices within the warehouse or fulfillment center system.

7. The parsing system of claim 1 , wherein the imaging device is a camera configured to capture an image of the visual instructions displayed on the monitor.

8. The parsing system of claim 1 , wherein the imaging device is an image or video capture card.

9. A system, comprising:

a robot having a picking arm configured to grasp an item from a first location and place the grasped item in a second location spaced from the first location;

an imaging device to capture visual instructions provided by a warehouse or fulfillment center system and displayed on a monitor, the warehouse or fulfillment center system being distinct from the parsing system; and

one or more processors, wherein the one or more processors are configured to:

receive, from the imaging device, captured visual instructions;

generate functions for the robot to perform based on the captured visual instructions;

transmit, to the robot, the functions to perform;

receive, from the robot, confirmation of performance of the functions;

transmit, after receiving confirmation of performance of the functions, a response indicating completion of the determined pick and place instructions.

10. The system of claim 9 , wherein the robot is further configured to:

receive the functions to perform from the one or more processors;

perform the received functions; and

provide the confirmation of completion of the received functions to the one or more processors after performing the received functions.

11. The system of claim 9 , further comprising:

a virtual mouse, wherein the virtual mouse is configured to, in response to receiving the response from the one or more processors, provide a completion response to the warehouse or fulfillment center system.

12. A pick and place method, the method comprising:

receiving, by one or more processors, an image including visual instructions displayed on a monitor and provided by a warehouse or fulfillment center system;

executing, by the one or more processors, vision or optical character recognition algorithms on the received image to determine pick and place instructions included in the visual instructions;

generating, by the one or more processors, pick and place functions for a robot, the pick and place functions configured to cause the robot to perform the determined pick and place instructions;

providing, by the one or more processors, the pick and place functions to the robot; and

transmitting, by the one or more processors, a response indicating completion of the determined pick and place instructions.

13. The method of claim 12 , further comprising:

performing, by the robot, the pick and place functions.

14. The method of claim 12 , wherein the pick and place functions are further configured to cause the robot to pick an item from a first section of a first container and place the item into a second container, the first section of the first container and/or the second container being determined from the pick and place instructions.

15. The parsing system of claim 1 , wherein the functions configured to cause the robot to perform the determined pick and place instructions include one or more functions configured to cause the robot to pick an item from a first container and place the item in a second container.

16. The parsing system of claim 5 , wherein the physical button-pushing device is further configured to receive the response from the one or more processors, and

wherein the completion response notifies the warehouse or fulfillment center system of completion of the determined pick and place instructions.

17. The parsing system of claim 6 , wherein, the custom device is further configured to transmit the emulated control signals to the warehouse or fulfillment center system to provide, in response to receiving the response from the one or more processors, a completion response to the warehouse or fulfillment center system notifying the warehouse or fulfillment center system of completion of the visual instructions.

18. The system of claim 9 , wherein the first location is a first container and the second location is a second container.

19. The system of claim 10 , wherein the robot is further configured to:

receive the response from the one or more processors; and

provide, after receiving the response, an input into the warehouse or fulfillment center system indicating the completion of the determined pick and place instructions.

20. The method of claim 13 , further comprising:

transmitting, by the robot, a confirmation of completion of the pick and place functions to the one or more processors after performing the pick and place functions,

wherein, the one or more processors transmit the response after receiving the confirmation of completion from the robot.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2020
From: KALOUCHE, SIMON; MYNEPALLI, SIVA CHAITANYA; VASHCHHENKO, JOHN; NIGAM, SUYASH; DAWSON, JORDAN; MARCHMAN, GEORGE
To: NIMBLE ROBOTICS, INC.
Reel/Frame 053694/0072 →
Cited By (1)
US 12,246,354