IP Library Granted Patent US 11,068,949
Granted Patent B2
US 11,068,949 · App. 15/826,508 · Granted Jul 20, 2021

Distributed and automated transaction systems

Inventors: Paul McDonald (Diablo, CA); Ashwath Rajan (San Francisco, CA); Tyler Garvin (San Francisco, CA)
Assignee: 365 Retail Markets, LLC
G06Q30/06G06K9/00006G06K9/00355G06K9/00375G06K9/00744G06K9/00771G06K9/00979G06K9/2054G06K9/4628G06K9/627G06Q10/087G06Q20/102G06Q20/14G06Q20/20G06Q20/208G06Q20/22G06Q30/0201G06Q30/0202G06Q30/0207G06Q30/0241G06T7/74G06T7/75G07G1/0036G07G1/0054G06K9/00288G06K9/183G06K2009/00738G06T2207/10016G06T2207/10024G06T2207/20081G06T2207/20084G06T2207/30196H04L67/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,068,949
App. No.
15/826,508
Granted
Jul 20, 2021
Kind
B2
Abstract

Methods and apparatus are described that use machine vision and machine learning to eliminate many of the steps of a typical retail transaction. For example, implementations are contemplated in which a user simply removes an item from a retail location to effect the transaction.

Claims (38)

1. A system comprising:

a product enclosure including one or more shelves for placement of a plurality of products, one or more doors, one or more cameras, one or more processors, and a network connection, the one or more processors being configured to:

unlock one or more of the doors in response to an open door message from a remote back end infrastructure;

using the one or more cameras, capture a sequence of images associated with a transaction involving one or more products; and

using the network connection, transmit the sequence of images to a remote back end infrastructure in response to one or more of the doors locking, or in response to passage of a predetermined period of time; and

the remote back end infrastructure including one or more computing devices operating in a network, the one or more computing devices configured to:

receive the sequence of images from the product enclosure via the network;

process the sequence of images to (1) identify a hand reaching into the product enclosure, (2) determine a location of the hand within the enclosure, and (3) identify a first product grasped by the hand; and

charge an account associated with the user a transaction amount for the transaction, the transaction amount including a first amount corresponding to the first product.

2. The system of claim 1 , wherein the one or more computing devices or the one or more processors are further configured to identify the user by one of (1) biometric identification, (2) detection of a machine readable code corresponding to the user, or (3) receiving a code corresponding to the product enclosure transmitted from a device corresponding to the user.

3. The system of claim 1 , wherein the one or more processors are further configured to, using the one or more cameras, determine that a human is in front of the product enclosure.

4. The system of claim 1 , wherein the one or more computing devices are further configured to process the sequence of images to discard one or more subsets of the images that do not include the hand.

5. The system of claim 1 , wherein the one or more computing devices are configured to process the sequence of images to determine a location of the hand within the product enclosure based on a model of a spatial context of the product enclosure.

6. The system of claim 5 , wherein the one or more computing devices are configured to process the sequence of images to identify the first product grasped by the hand based on a product location schema associated with the spatial context.

7. The system of claim 1 , wherein the one or more computing devices are configured to process the sequence of images using a multibox localization model to focus on portions of individual images in which the hand appears.

8. The system of claim 1 , wherein the one or more computing devices are further configured to process the sequence of images to identify one or more changes within the product enclosure between successive images.

9. The system of claim 1 , wherein the product enclosure further comprises one or more doors, and wherein the one or more processors are further configured to:

determine that the one or more doors have been open longer than a predetermined period of time;

using the one or more cameras, passively record video; and

using the network connection, periodically transmit a frame of the video to the back end infrastructure.

10. The system of claim 9 , wherein the one or more computing devices are further configured to:

process the frames of the video received from the product enclosure to identify one or more changes within the product enclosure between successive ones of the frames; and

request upload of the video from the product enclosure in response to identification of the one or more changes.

11. A product enclosure, comprising:

one or more shelves for placement of a plurality of products within the product enclosure;

one or more doors designed to allow access to the plurality of products;

one or more cameras;

a network connection; and

one or more processors configured to:

unlock one or more of the doors in response to an open door message from a remote back end infrastructure;

using the one or more cameras, capture a sequence of images associated with a transaction involving one or more of the products; and

using the network connection, transmit the sequence of images to the remote back end infrastructure in response to one or more of the doors locking, or in response to passage of a predetermined period of time.

12. The product enclosure of claim 11 , wherein the one or more processors are further configured to facilitate identification of the user by one of (1) capturing biometric identifying information, (2) detecting a machine readable code corresponding to the user, or (3) receiving a code transmitted from a device corresponding to the user.

13. The product enclosure of claim 11 , wherein the one or more processors are further configured to, using the one or more cameras, determine that a human is in front of the product enclosure.

14. The product enclosure of claim 11 , further comprising one or more doors, and wherein the one or more processors are further configured to:

determine that the one or more doors have been open longer than a predetermined period of time;

using the one or more cameras, passively record video; and

using the network connection, periodically transmit a frame of the video to a back end infrastructure.

Assignments (7)
SECURITY INTEREST Recorded Dec 23, 2020
From: 365 RETAIL MARKETS, LLC
To: ANTARES CAPITAL LP, AS ADMINISTRATIVE AGENT
Reel/Frame 054740/0326 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: STOCKWELL AI, INC.
To: 365 RETAIL MARKETS, LLC
Reel/Frame 053558/0854 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: STOCKWELL AI INC.
To: 365 RETAIL MARKETS, LLC
Reel/Frame 053563/0307 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: BODEGA AI, INC.
To: 365 RETAIL MARKETS
Reel/Frame 053557/0614 →
CHANGE OF NAME Recorded Aug 21, 2020
From: BODEGA AI, INC.
To: STOCKWELL AI, INC.
Reel/Frame 053575/0750 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: BODEGA AI, INC.
To: STOCKWELL AI INC.
Reel/Frame 053565/0606 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2018
From: MCDONALD, PAUL; RAJAN, ASHWATH; GARVIN, TYLER
To: BODEGA AI, INC.
Reel/Frame 044876/0239 →
Continuity (2)
Provisional Application 62432423 · Dec 9, 2016
Related Publication 20180165728A1 · Jun 14, 2018
Cited By (1)
US 12,406,226