Granted Patent B2

US 12,211,004 · App. 18/064,698 · Granted Jan 28, 2025

Inventors: Dilip Kumar (Seattle, WA); Elie Micah Kornfield (Seattle, WA); Alexander Clark Prater (Seattle, WA); Sridhar Boyapati (Sammamish, WA); Xiaofeng Ren (Yarrow Point, WA); Chang Yuan (Seattle, WA)

Assignee: Amazon Technologies, Inc.

G06Q10/087G07G1/0036

Abstract

Claims (85)

1. A computer-implemented method for providing assistance to users within a materials handling facility, comprising:

under control of one or more computing systems configured with executable instructions,

determining a location of a user within the materials handling facility;

receiving, from at least one of a plurality of cameras positioned within the materials handling facility, each camera of the plurality of cameras configured to generate images of objects located within the materials handling facility, a first image of the location;

detecting, based at least in part on data received from a sensor at an inventory location, a change in a weight at the inventory location, wherein the inventory location is near the location;

receiving, from at least one of the plurality of cameras, a second image of the location;

determining, based at least in part on one or more of the first image, the second image, or the change in the weight, a quantity of an item removed from the inventory location; and

updating a user item list associated with the user to indicate the quantity of the item removed from the inventory location.

2. The computer-implemented method of claim 1 , wherein determining the change in the quantity of the item removed from the inventory location, further includes:

determining that a first object was removed from the inventory location.

3. The computer-implemented method of claim 1 , wherein the sensor is at least one of a pressure sensor, a scale, a weight sensor, or a volume displacement sensor.

4. The computer-implemented method of claim 1 , wherein determining the quantity of the item removed from the inventory location further includes:

comparing the first image and the second image to identify the quantity of the item removed based at least in part on a pixel comparison or an object difference determination.

5. The computer-implemented method of claim 1 , wherein determining the quantity of the item removed, further includes:

receiving an input from an input component, wherein the input component is at least one of an image capture device, an RFID reader, a barcode reader, or a QR code reader; and

determining, based at least in part on the input and on one or more of the first image, the second image, or the change in the weight, the quantity of the item removed from the inventory location.

6. The computer-implemented method of claim 1 , the method further comprising:

determining iems a plurality of items included on the user item list;

identifying the item based at least in part on:

data received from at least one input device positioned within the materials handling facility, the first image, the second image, or the change in the weight; and

the items included on the user item list; and

determining that a confidence score of an identification of the item removed from the inventory location exceeds a threshold; and

wherein updating the user item list includes adding, to the user item list, an item identifier for the identified item.

7. The computer-implemented method of claim 6 , further comprising:

sending a message that identifies the item to a portable device of the user.

8. The computer-implemented method of claim 1 , wherein determining the change in the quantity of the item removed from the inventory location further includes:

determining that an object has been removed from the inventory location; and

determining that the object corresponds with the item stored at the inventory location.

9. The computer-implemented method of claim 8 , further comprising:

determining the item associated with the inventory location;

wherein determining the quantity of the item removed from the inventory location is further based at least in part on:

data received from at least one input device at the location; and

the items associated with the inventory location; and

determining that a confidence score of an identification of the item removed from the inventory location exceeds a threshold; and

wherein updating the user item list includes adding an item identifier for the item to the user item list.

10. The computer-implemented method of claim 1 , further comprising:

determining, based at least in part on a portable device in a possession of the user, a user profile of the user; and

wherein the user item list is associated with at least one of the user or the user profile.

11. A system, comprising:

a plurality of cameras positioned within a materials handling facility, each camera of the plurality of cameras configured to generate images of objects located within the materials handling facility;

one or more processors; and

a memory storing program instructions that, when executed by the one or more processors, cause the one or more processors to at least:

determine, based at least in part on a portable device in a possession of a user or an application executing on the portable device, a user profile of the user;

determine an inventory location within a defined distance of the user;

detect, based at least in part on data received from a sensor at the inventory location, a change in a weight at the inventory location;

obtain, from at least one of the plurality of cameras, a first image of the inventory location generated before the change;

obtain, from at least one of the plurality of cameras, a second image of the inventory location generated subsequent to the change;

determine, based at least in part on one or more of the first image, the second image, or the change in the weight, a quantity of an item placed at the inventory location; and

update a user item list associated with the user profile to indicate the quantity of the item placed at the inventory location.

12. The system of claim 11 , wherein the

sensor is at least one of a pressure sensor, a scale, a weight sensor, or a volume displacement sensor.

13. The system of claim 11 , wherein the program instructions that, when executed by the one or more processors to determine the quantity of the item placed at the inventory location, further include program instructions that cause the one or more processors to at least:

receive an input from an input component, wherein the input component is at least one of an image capture device, an RFID reader, a barcode reader, or a QR code reader; and

determine, based at least in part on the input and one or more of the first image, the second image, or the change in the weight, the quantity of the item placed at the inventory location.

14. The system of claim 11 , wherein the program instructions that, when executed by the one or more processors to determine the quantity of the item placed at the inventory location, further include program instructions that cause the one or more processors to at least:

determine that an object has been placed at the inventory location; and

determine that the object corresponds with the items item stored at the inventory location.

15. The system of claim 14 , wherein the program instructions that, when executed by the one or more processors, further cause the one or more processors to at least:

determine an item identifier included on the user item list;

identify the item based at least in part on:

data received from at least one input device, the first image, the second image, or the change in the weight; and

the item identifier included on the user item list;

determine that a confidence score of an identification of the item placed at the inventory location exceeds a threshold; and

remove, from the user item list, the item identifier for the identified item.

16. The system of claim 14 , wherein the program instructions that, when executed by the one or more processors, further cause the one or more processors to at least:

determine that the item does not correspond with the inventory location; and

send a message to an associate to remove the item from the inventory location.

17. A method, comprising:

receiving, from at least one of a plurality of cameras positioned within a materials handling facility, each camera of the plurality of cameras configured to generate images of objects located within the materials handling facility, a first image of a location within the materials handling facility;

detecting, based at least in part on data received from a sensor at an inventory location, a change in a weight at the location;

receiving, from at least one of the plurality of cameras, a second image of the location;

determining, based at least in part on one or more of the first image, the second image, or the change in the weight, a change in a quantity of an item associated with the location; and

updating a user item list associated with a user, wherein the updating is based at least in part on the change in the quantity of the item.

18. The method of claim 17 , further comprising:

determining that the user is within a defined distance of the location.

19. The method of claim 17 , wherein determining the change in the quantity of the item further includes:

determining that an object has been removed from the location; and

determining that the object corresponds with the item associated with the location.

20. The method of claim 19 , further comprising:

determining a plurality of items associated with the location;

wherein determining the change in the quantity of the item is further based at least in part on:

data received from at least one input device; and

the plurality of items associated with the location; and

determining that a confidence score of an identification of the item removed from the location exceeds a threshold; and

wherein updating the user item list includes adding an item identifier for the item to the user item list.

Assignments (1)

Continuity (3)

References Cited (120)