Inventory management by mobile robot
A mobile robot is configured for operation in a commercial or industrial setting, such as an office building or retail store. The robot can patrol one or more routes within a building, and can detect violations of security policies by objects, building infrastructure and security systems, or individuals. In response to the detected violations, the robot can perform one or more security operations. The robot can include a removable fabric panel, enabling sensors within the robot body to capture signals that propagate through the fabric. In addition, the robot can scan RFID tags of objects within an area, for instance coupled to store inventory. Likewise, the robot can generate or update one or more semantic maps for use by the robot in navigating an area and for measuring compliance with security policies.
1. A mobile robot, comprising:
a motorized base configured to move the mobile robot throughout a floor of a retail environment;
a wireless transceiver configured to communicatively couple to an inventory system of the retail environment;
a robot body on the motorized base comprising:
a mechanical arm including a camera, the mechanical arm configured to contract and extend relative to the robot body; and
a controller configured to:
cause the motorized base to navigate within the retail environment;
capture images of inventory on display within the retail environment using the camera on the mechanical arm, wherein at least one image is captured in response to determining that inventory on display is obstructing a view of other inventory by extending the mechanical arm into a space above the inventory but below a shelf immediately above the inventory and capturing an image of inventory not visible from in front of the shelf;
select an inventory operation to perform based on the captured images wherein the selected inventory operation comprises a restocking operation in response to determining that a number of items of a product type currently on display is less than a threshold number of the items required to be on display;
in response to a sufficient number of items to restock the product type being available at the retail environment, causing the mobile robot to restock the product type; and
in response to a sufficient number of items to restock the product type not being available at the retail environment, autonomously ordering, by the mobile robot, additional items of the product type from a remote location for delivery to the retail environment.
2. The mobile robot of claim 1 , wherein performing the restocking operation further comprises:
retrieving, using the mechanical arm, one or more additional items of the product type; and
placing, using the mechanical arm, the one or more retrieved additional items of the product type on a display area adjacent to the number of items of the product type currently on display.
3. The mobile robot of claim 2 , wherein placing the one or more retrieved additional items of the product type on a display area adjacent to the number of items of the product type currently on display further comprises:
determining a target location for the one or more retrieved additional items based on the captured images; and
placing the one or more retrieved additional items at the determined target location.
4. The mobile robot of claim 1 , wherein determining the number of items of the product type currently on display further comprises:
moving the mechanical arm to a location immediately above the items of the product type currently on display before capturing the images; and
performing one or more image processing operations on the captured images to determine the number of items.
5. The mobile robot of claim 1 , wherein selecting the inventory operation to perform further comprises:
determining an orientation of an item of a product type on display within the retail environment based on the captured images;
comparing the determined orientation of the item of the product type to a target orientation of the product type; and
rotating, using the mechanical arm, the item based on the comparison until the item has substantially a same orientation as the target orientation.
6. The mobile robot of claim 1 , wherein selecting the inventory operation to perform further comprises:
detecting a defect in an item of the inventory on display within the retail environment based on the captured images; and
responsive to detecting the defect, removing, using the mechanical arm, the item with the defect from display.
7. The mobile robot of claim 1 , wherein the controller is further configured to track movement of individuals within the retail environment.
8. The mobile robot of claim 1 , wherein the controller is further configured to compare the captured images of the inventory on display to a placement request associated with the inventory received from a third party system to determine if the inventory on display satisfies one or more requirements associated with the placement request, and to flag to a human operator if the requirements associated with the placement request are not satisfied.
9. A method comprising:
navigating, by a mobile robot, within a retail environment;
capturing, by a camera mounted to a mechanical arm of the mobile robot, images of inventory on display within the retail environment, wherein at least one image is captured in response to determining that inventory on display is obstructing a view of other inventory by extending the mechanical arm into a space above the inventory but below a shelf immediately above the inventory and capturing an image of inventory not visible from in front of the shelf;
selecting, by the mobile robot, an inventory operation to perform based on the captured images, wherein the selected inventory operation comprises a restocking operation in response to determining that a number of items of a product type currently on display is less than a threshold number of the items required to be on display;
in response to a sufficient number of items to restock the product type being available at the retail environment restocking, by the mobile robot, the product type; and
in response to a sufficient number of items to restock the product type not being available at the retail environment, autonomously ordering, by the mobile robot, additional items of the product type from a remote location for delivery to the retail environment.
10. The method of claim 9 , wherein performing the restocking operation further comprises:
retrieving, using the mechanical arm, one or more additional items of the product type; and
placing, using the mechanical arm, the one or more retrieved additional items of the product type on a display area adjacent to the number of items of the product type currently on display.
11. The method of claim 10 , wherein placing the one or more retrieved additional items of the product type on a display area adjacent to the number of items of the product type currently on display further comprises:
determining a target location for the one or more retrieved additional items based on the captured images; and
placing the one or more retrieved additional items at the determined target location.
12. The method of claim 9 , wherein determining the number of items of the product type currently on display further comprises:
moving the mechanical arm to a location immediately above the items of the product type currently on display before capturing the images; and
performing one or more image processing operations on the captured images to determine the number of items.
13. The method of claim 9 , wherein selecting the inventory operation to perform further comprises:
determining an orientation of an item of a product type on display within the retail environment based on the captured images;
comparing the determined orientation of the item of the product type to a target orientation of the product type; and
rotating, using the mechanical arm, the item based on the comparison until the item has substantially a same orientation as the target orientation.
14. The method of claim 9 , wherein selecting the inventory operation to perform further comprises:
detecting a defect in an item of the inventory on display within the retail environment based on the captured images; and
responsive to detecting the defect, removing, using the mechanical arm, the item with the defect from display.
15. The method of claim 9 , further comprising:
tracking, by the mobile robot, movement of individuals within the retail environment.
16. The method of claim 9 , further comprising:
compare the captured images of the inventory on display to a placement request associated with the inventory received from a third party system to determine if the inventory on display satisfies one or more requirements associated with the placement request, and to flag to a human operator if the requirements associated with the placement request are not satisfied.