Counting stacked inventory using image analysis
Described is a system for counting stacked items using image analysis. In one implementation, an image of an inventory location with stacked items is obtained and processed to determine the number of items stacked at the inventory location. In some instances, the item closest to the camera that obtains the image may be the only item viewable in the image. Using image analysis, such as depth mapping or Histogram of Oriented Gradients (HOG) algorithms, the distance of the item from the camera and the shelf of the inventory location can be determined. Using this information, and known dimension information for the item, a count of the number of items stacked at an inventory location may be determined.
1. A computing system, comprising:
a processor; and
a memory coupled to the processor and storing program instructions that when executed by the processor causes the processor to at least:
receive from a first camera a first image of an inventory location, wherein the first image includes a representation of a plurality of inventory items vertically stacked and located at the inventory location;
determine from an inventory location data store, an item type corresponding to the inventory location;
select a first histogram of oriented gradients (“HOG”) model and a second HOG model, wherein:
the first HOG model corresponds to the item type and is representative of a first vertical stack of a first quantity of an item of the item type;
the second HOG model corresponds to the item type and is representative of a second vertical stack of a second quantity of the item of the item type, wherein the first quantity and the second quantity are different;
process the first image to generate an inventory item feature vector representative of the plurality of inventory items vertically stacked and represented in the first image;
compare the inventory item feature vector with the first HOG model and the second HOG model;
determine that the inventory item feature vector substantially matches the first HOG model; and
in response to a determination that the inventory item feature vector substantially matches the first HOG model, determine an inventory item count representative of a number of inventory items vertically stacked at the inventory location to be the first quantity.
2. The computing system of claim 1 , wherein the program instructions, that when executed by the processor to cause the processor to determine the inventory item count, further include instructions that when executed by the processor cause the processor to at least:
determine a first distance between the first camera and a first item of the plurality of inventory items vertically stacked and located at the inventory location;
determine a second distance between the first camera and a shelf of the inventory location;
determine a dimension of an inventory item of the plurality of inventory items; and
compute the inventory item count based at least in part on a difference between the first distance and the second distance and further based at least in part on the dimension of the inventory item.
3. The computing system of claim 1 , wherein the inventory item count identifies an increase or a decrease in inventory at the inventory location.
4. The computing system of claim 1 , wherein the program instructions, that when executed by the processor to cause the processor to determine the inventory item count, further include instructions that when executed by the processor cause the processor to at least:
determine a defined distance between a front edge of an inventory shelf of the inventory location and the plurality of inventory items;
determine a depth of the inventory shelf from the front edge of the inventory shelf to a back of the inventory shelf;
determine a dimension of an inventory item of the plurality of inventory items; and
compute an inventory item count based at least in part on a difference between the defined distance and the depth of the inventory shelf and further based at least in part on the dimension of the inventory item.
5. A computer-implemented method for counting stacked items, comprising:
under control of one or more computing systems configured with executable instructions,
receiving from a camera an image of a plurality of stacked items positioned at an inventory location;
determining from the image, and based at least in part on a histogram of oriented gradients (HOG) model determined to correspond to the image, a first distance between an item of the plurality of stacked items and the camera, wherein the HOG model is representative of at least one of the plurality of stacked items, or the item;
determining an item dimension representative of at least one of a height, a width, or a length of the item; and
computing an item count based at least in part on:
the first distance between the item and the camera; and
the item dimension.
6. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a shelf of the inventory location and the camera; and
wherein the item count is further based at least in part on:
a difference between the first distance between the item and the camera and the second distance; and
the item dimension.
7. The computer-implemented method of claim 6 , wherein:
the second distance is maintained in an inventory location data store; and
the first distance is further determined based at least in part on depth information received with the image.
8. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a front edge of a shelf of the inventory location and the item;
determining a depth of the shelf from the front edge of the shelf to a back of the shelf; and
computing the item count based at least in part on:
a difference between the second distance between the front edge of the shelf and the item,
the depth of the shelf; and
the item dimension of the item.
9. The computer-implemented method of claim 5 , wherein the first distance is further determined based at least in part on a comparison of the image with a plurality of images obtained by the camera to obtain depth information for the item.
10. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a shelf of the inventory location and the camera;
determining a third distance between a front edge of the shelf and the item;
determining a fourth distance between the front edge of the shelf and a back of the shelf;
computing the item count based at least in part on:
a first quotient of:
a difference between the first distance and the second distance; and
the item dimension; and
a second quotient of:
a difference between the third distance and the fourth distance; and
a second item dimension.
11. The computer-implemented method of claim 5 , wherein the camera is mounted to an underneath side of a shelf positioned above the inventory location and oriented downward toward the inventory location.
12. The computer-implemented method of claim 5 , wherein the item count identifies at least one of a number of items stacked vertically at the inventory location or a number of items stacked horizontally at the inventory location.
13. A computer-implemented method, comprising:
under control of one or more computing systems configured with executable instructions,
receiving from a camera an image of a plurality of stacked items positioned at an inventory location;
determining from the image a first distance between an item of the plurality of stacked items and the camera wherein:
the first distance is determined based at least in part on a change in a magnification level of the item represented in the image when the item is compared with a histogram of oriented gradients model (“HOG”); and
the magnification level corresponds with the first distance between the item and the camera;
determining an item dimension representative of at least one of a height, a width, or a length of the item; and
computing an item count based at least in part on:
the first distance between the item and the camera; and
the item dimension.
14. The computer-implemented method of claim 13 , further comprising:
determining a second distance between a shelf of the inventory location and the camera; and
wherein the item count is further based at least in part on:
a difference between the first distance and the second distance; and
the item dimension.
15. The computer-implemented method of claim 13 , further comprising:
determining a second distance between a front edge of a shelf of the inventory location and the item;
determining a depth of the shelf from the front edge of the shelf to a back of the shelf; and
computing the item count based at least in part on:
a difference between the second distance between the front edge of the shelf and the item,
the depth of the shelf; and
the item dimension of the item.
16. The computer-implemented method of claim 13 , further comprising:
determining a second distance between a shelf of the inventory location and the camera;
determining a third distance between a front edge of the shelf and the item;
determining a fourth distance between the front edge of the shelf and a back of the shelf;
computing the item count based at least in part on:
a first quotient of:
a difference between the first distance and the second distance; and
the item dimension; and
a second quotient of:
a difference between the third distance and the fourth distance; and
a second item dimension.
17. A computing system, comprising:
a processor; and
a memory coupled to the processor and storing program instructions that when executed by the processor causes the processor to at least:
receive from a camera an image of a plurality of stacked items located at an inventory location,
receive depth information corresponding to a position of the plurality of stacked items represented in the image with respect to the camera;
generate a point cloud that represents at least a portion of the image and based at least in part on the depth information;
determine a cluster boundary based at least in part on item information stored in an item information data store, wherein the item information corresponds to the plurality of stacked items, wherein a base of the cluster boundary corresponds to a position of an inventory shelf with respect to the point cloud;
obtain from the item information data store, a dimension corresponding to an item type associated with the inventory location; and
compute a number of items in the plurality of stacked items located on the inventory shelf of the inventory location based at least in part on the cluster boundary and the dimension of the item type.
18. The computing system of claim 17 , wherein the dimension of the item type is at least one of a height, a width, or a length.
19. The computing system of claim 17 , wherein:
a distance between the camera and the inventory shelf is known; and
the point cloud is generated at least in part by transposing the received depth information to coordinates associated with the inventory shelf of the inventory location.
20. The computing system of claim 17 , wherein the depth information is determined based at least in part on one or more of an ultrasonic sensor, a laser sensor, a radar distance measurement sensor, a stadiametric sensor, a parallax sensor, a Lidar sensor, a Sonar sensor, a time-of-flight sensor, or a stereo image.
21. The computing system of claim 17 , wherein the program instructions that when executed by the processor further cause the processor to at least:
process the image using at least one of a histogram of oriented gradients algorithm, an edge detection algorithm or an object detection algorithm to determine a feature vector representative of an item of the plurality of stacked items represented in the image.