Associating shoppers together
A store having an automated checkout can automatically associate multiple shoppers with a single purchase. For example, a mother and her children can walk independently through the store and select items. Upon checkout, the items selected by the mother and children can all be charged to the mother's credit card in a single purchase. The system and method discussed herein can capture images of multiple shoppers from one or more video streams of a store area, can associate the multiple shoppers with one another from the captured images when the multiple shoppers appear together in a check-in area of the store, can track the movement of the multiple shoppers within the store from the captured images, can track items selected by the multiple shoppers from the captured images, and can initiate a transaction that charges the selected items to an account associated with one of the multiple shoppers.
1. A method, comprising:
receiving, at a check-in location of a store, selection of first and second accounts account to be charged for a purchase purchases, the first and second accounts account being associated with respective first and second shoppers shopper;
identifying, with a video surveillance system of the store, the first and second shoppers shopper as being one of a group of shoppers that are present in the check-in location of the store when the selection is received;
assigning a unique personal identifier to each respective member of the group of shoppers, the personal identifier based on a feature vector of the first shopper who is a particular member of the group of shoppers to which the personal identifier is assigned, the feature vector being a multi-dimensional vector created from a first image including at least the first shopper by applying a deep-learning autoencoder;
assigning a common cart to the group of shoppers and associating the common cart identifier with all members of the group of shoppers;
tracking with the video surveillance system of the store, movement of each member of the group of shoppers throughout the store and items in the store selected for purchase by any of the shoppers in the group of shoppers, wherein the tracking includes sensing when the first or second shopper picks up a particular item in the store, wherein the sensing is made by sending a second image to the deep-learning autoencoder, retrieving a second feature vector from the second image, and comparing the second feature vector with the feature vector, and wherein the second image includes at least one of the first and second shopper;
specifying that a cost of the purchase of the selected items is to be split between the first shopper and the second shopper;
charging, through a frictionless check-out of the store, the selected account first and second accounts for respective first and second portions of the cost of the purchase to be split between the first and second shoppers for the purchase of the selected items; and
charging, through the frictionless check-out of the store, the second account for a second portion of the cost of the purchase to be split between the first shopper and the second shopper for the purchase of the selected items.
2. The method of claim 1 , wherein the video surveillance system tracks the items in the store by tracking items that are handled by any of the group of shoppers.
3. The method of claim 2 , wherein tracking the items in the store selected for purchase by any of the shoppers in the group of shoppers comprises:
adding the particular item to a virtual shopping cart.
4. The method of claim 3 , wherein tracking the items in the store selected for purchase by any of the shoppers in the group of shoppers further comprises:
sensing, with the video surveillance system of the store, that the first shopper has returned the particular item to the shelf or rack; and
removing the particular item from the virtual shopping cart.
5. The method of claim 1 , wherein receiving the selection of the account comprises:
receiving input of data corresponding to a credit card number from the shopper.
6. The method of claim 5 , wherein receiving the data corresponding to the credit card number comprises:
receiving a credit card input.
7. The method of claim 1 , wherein receiving the selection of the account comprises:
receiving a radiofrequency identifier that corresponds in stored data to the shopper; and
invoking a charging process that charges a purchase to a bankcard account represented in and associated with the shopper in stored data.
8. The method of claim 1 , wherein receiving the selection of the account comprises:
scanning a visual code that corresponds in stored data to the shopper; and
invoking a charging process that charges a purchase to a bankcard account represented in and associated with the shopper in stored data.
9. The method of claim 1 , wherein receiving the selection of the account comprises:
scanning a biometric indicator that corresponds in stored data to the shopper; and
invoking a charging process that charges a purchase to a bankcard account represented in and associated with the shopper in stored data.
10. The method of claim 1 , wherein identifying the shopper as being one of the group of shoppers comprises:
receiving at least one image from the video surveillance system, the at least one image including the check-in location of the store.
11. The method of claim 1 , wherein identifying a shopper as being one of the group of shoppers comprises:
forming cropped images of bodies of identified shoppers in the group of shoppers;
sending the cropped images to at least one processor configured to execute the deep-learning autoencoder; and
storing, on a storage device coupled to the at least one processor, the multi-dimensional feature vector in a nearest-neighbors database.
12. The method of claim 11 , wherein tracking the items in the store selected for purchase comprises:
sensing, with the video surveillance of the store, that a particular item has been picked up;
sending, to the deep-learning autoencoder, a cropped image of the person who picked up the first item;
retrieving, from the deep-learning autoencoder, a feature vector corresponding to the sent cropped image;
calculating a Euclidean distance metric between the retrieved feature vector and feature vectors stored on the storage device; and
identifying, from the stored feature vector having the shortest Euclidean distance to the retrieved feature vector, a particular shopper as the person who picked up the first item.
13. A system, comprising:
at least one video camera positioned to capture at least one video stream of a store;
a video interface configured to receive the at least one video stream; and
a processor coupled to the video interface and configured to execute computing instructions to perform data processing activities, the data processing activities comprising:
receiving, at a check-in location of a store, selection of first and second accounts account to be charged for purchases, the first and second accounts being associated with respective first and second shoppers;
identifying, with a video surveillance system of the store, the first and second shoppers as being a group of shoppers that are present in the check-in location of the store when the selection is received;
assigning a unique personal identifier to each respective member of the group of shoppers, the personal identifier based on a feature vector of the first shopper who is a particular member of the group of shoppers to which the personal identifier is assigned, the feature vector being a multi-dimensional vector created from a first image including at least the first shopper by applying a deep-learning autoencoder;
assigning a common cart to the group of shoppers and associating the common cart identifier with all members of the group of shoppers;
tracking with the video surveillance system of the store, movement of each member of the group of shoppers throughout the store and items in the store selected for purchase by any of the shoppers in the group of shoppers, wherein the tracking includes sensing when the first or second shopper picks up a particular item in the store, wherein the sensing is made by sending a second image to the deep-learning autoencoder, retrieving a second feature vector from the second image, and comparing the second feature vector with the feature vector, and wherein the second image includes at least one of the first and second shopper;
specifying that a cost of the purchase of the selected items is to be split between the first shopper and the second shopper;
charging, through a frictionless check-out of the store, the first and second accounts for respective first and second portions of the cost of the purchase to be split between the first and second shoppers for the purchase of the selected items.
14. The system of claim 13 , wherein tracking the items in the store comprises tracking items that are handled by any of the group of shoppers.
15. The system of claim 14 , wherein tracking the items in the store comprises:
adding the particular item to a virtual shopping cart.
16. The system of claim 15 , wherein tracking the items in the store further comprises:
sensing, from the at least one video stream, that a shopper of the group of shoppers has returned the particular item to the shelf or rack; and
removing the first item from the virtual shopping cart.
17. The system of claim 13 , wherein receiving the selection of the account comprises:
receiving input of data corresponding to a credit card number from the shopper.
18. The system of claim 13 , wherein identifying the shopper as being one of the group of shoppers comprises:
receiving at least one image from the at least one video stream, the at least one image including the check-in location of the store.
19. The system of claim 13 , wherein identifying the shopper as being one of the group of shoppers comprises:
forming cropped images of bodies of the identified shoppers in the group;
sending the cropped images to the deep-learning autoencoder configured to execute on the processor; and
storing, on a storage device coupled to the processor, the multi-dimensional feature vector in a nearest-neighbors database.