Automated checkout system with video and RFID

1 . A checkout system comprising:

a first checkout lane including one or more walls defining a transition zone;

a first RFID antenna system positioned on or within the one or more walls and directed into the transition zone;

a first camera positioned above the first checkout lane and having a field of view that comprises the transition zone;

one or more processors; and

one or more non-transitory computer-readable media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving first RFID data from the first RFID antenna system;

determining, by a tag-transition component based on the first RFID data, a first set of items that passed through the checkout system;

receiving first image data from the first camera;

determining, by a lane-occupancy component based at least on the first image data, a checkout time for a user based on a timestamp associated with the user traversing the transition zone;

determining, by the lane-occupancy component based on the timestamp, a window of time associated with the user passing through the first checkout lane;

determining, by a cart-generation component based on a first portion of the first image data associated with the window of time, a second set of items associated with the user;

determining, by the cart-generation component, a third set of items associated with the user based at least in part on a subset of the first set of items and the second set of items determined based at least in part on a union between the first set of items and the second set of items;

determining, by the cart-generation component, a virtual cart for the user based on the third set of items; and

charging, by a payment component, a payment instrument associated with the first checkout lane for the virtual cart.

2 . The checkout system of claim 1 , wherein determining the third set of items comprises:

determining a first set of confidence scores associated with the first set of items;

determining a second set of confidence scores associated with the second set of items;

adjusting a confidence score value of one or more of the first set of confidence scores based on the second set of confidence scores to produce reconciled confidence score data; and

determining the third set of items based on the reconciled confidence score data.

3 . The checkout system of claim 1 , wherein the first checkout lane comprises a gate proximate the first checkout lane, the gate configured to toggle between a closed state and an open state based at least in part on receiving payment information from the user, and wherein determining the checkout time is based on the timestamp associated with the gate transitioning from the closed state to the open state as the user traverses the transition zone.

4 . The checkout system of claim 1 , wherein determining the third set of items comprises providing the first RFID data to a machine learned algorithm trained using RFID data from one or more RFID antennas of a cashierless checkout system, the RFID data tagged based on image data from one or more cameras of the cashierless checkout system to indicate a lane of a plurality of checkout lanes of the cashierless checkout system an item having an RFID tag passed through.

5 . A method comprising:

receiving first reader data from one or more first reader antennas positioned proximate and directed toward a first checkout lane of a checkout system in a facility;

determining, by a tag-transition component based on the first reader data, a first set of items that passed through the checkout system;

receiving first image data from one or more cameras positioned proximate and directed toward the first checkout lane in the facility;

determining, by a lane-occupancy component, a checkout time for a user based at least on the first image data and on a timestamp associated with the user traversing the first checkout lane;

determining, by the lane-occupancy component based on the timestamp, a window of time associated with the user passing through the first checkout lane;

determining, by a cart-generation component, a second set of items associated with the user based on a portion of the first image data associated with the window of time;

determining, by the cart-generation component, a third set of items associated with the user based at least in part on the first set of items and the second set of items;

determining, by the cart-generation component, a virtual cart for the user based on the third set of items; and

charging, by a payment component, a payment instrument associated with the virtual cart.

6 . The method of claim 5 , wherein determining the third set of items comprises:

providing the first reader data and first image data to one or more machine learned algorithms trained using a set of reader data and a set of image data associated with a cashierless checkout system, the set of reader data and the set of image data tagged based on review of items to indicate a lane of a plurality of checkout lanes of the cashierless checkout system an item having a tag readable by the one or more first reader antennas passed through.

7 . The method of claim 5 , wherein determining the virtual cart comprises:

determining a first set of confidence scores associated with the first set of items;

determining a second set of confidence scores associated with the second set of items;

adjusting a confidence score value of one or more of the first set of confidence scores based on the second set of confidence scores to produce reconciled confidence score data; and

determining the third set of items based on the reconciled confidence score data.

8 . The method of claim 5 , wherein receiving the first reader data comprises:

receiving second reader data from one or more second reader antennas positioned proximate a first zone of the first checkout lane in the facility; and

receiving third reader data from one or more third antennas positioned proximate a second zone of the first checkout lane, and wherein determining the checkout time comprises analyzing the second reader data and the third reader data to determine the timestamp associated with a tag transitioning from the first zone of the first checkout lane to the second zone of the first checkout lane.

9 . The method of claim 5 , further comprising:

receiving payment information from the user; and

causing a first physical gate proximate to the first checkout lane to transition from a closed state to an open state based at least in part on receiving the payment information, wherein charging the payment instrument comprises charging the payment information of the user.

10 . The method of claim 5 , wherein determining the third set of items comprises:

generating a user interface comprising a representation of the first set of items or the second set of items;

displaying, via the user interface, at least a part of the portion of the first image data; and

receiving, via the user interface, an indication of an item visible in the first image data, wherein determining the third set of items is further based on the indication.

11 . The method of claim 5 , further comprising:

receiving second reader data from one or more second reader antennas positioned proximate a second side of the first checkout lane, wherein the first reader antennas are positioned proximate a first side of the first checkout lane; and

determining the checkout time is based at least in part on detecting a transition of an item by the one or more first reader antennas to the one or more second reader antennas.

12 . The method of claim 5 , wherein determining the third set of items comprises:

conveying the portion of the first image data to a user device;

generating a user interface comprising a representation of the first set of items, the second set of items, the first reader data, and the first image data; and

receiving, from the user device, an indication of the third set of items.

13 . The method of claim 5 , wherein determining the virtual cart comprises:

determining a first listing of potential items based on the first set of items or the second set of items;

determining a first set of confidence values, each confidence value in the first set of confidence values associated with an item of the first listing of potential items;

determining a second set of confidence values by adjusting the first set of confidence values for the first listing of potential items based on the third set of items; and

determining the virtual cart based on the second set of confidence values.

14 . A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising:

receiving first reader data from one or more first reader antennas positioned proximate and directed toward a first checkout lane of a checkout system in a facility;

determining, by a tag-transition component based on the first reader data, a first set of items that passed through the checkout system;

receiving first image data from one or more cameras positioned proximate and directed toward the first checkout lane in the facility;

determining, by a lane-occupancy component, a checkout time for a user based at least on the first image data and on a timestamp associated with the user traversing the first checkout lane;

determining, by the lane-occupancy component based on the timestamp, a window of time associated with the user passing through the first checkout lane;

determining, by a cart-generation component, a second set of items associated with the user based on a portion of the first image data associated with the window of time;

determining, by the cart-generation component, a third set of items associated with the user based at least in part on the first set of items and the second set of items;

determining, by the cart-generation component, a virtual cart for the user based on the second set of items and the third set of items; and

charging, by a payment component, a payment instrument associated with the virtual cart.

15 . The system of claim 14 , wherein determining the virtual cart comprises:

determining a first set of confidence scores associated with the first set of items;

determining a second set of confidence scores associated with the second set of items;

adjusting a confidence score value of one or more of the first set of confidence scores based on the second set of confidence scores to produce reconciled confidence score data; and

determining the third set of items based on the reconciled confidence score data.

16 . The system of claim 14 , wherein determining the virtual cart comprises:

determining a first listing of potential items based on the first set of items;

determining a first set of confidence values, with individual confidence values associated with individual items of the first listing of potential items;

adjusting the first set of confidence values based at least in part on the second set of items to produce a second set of confidence values; and

determining the third set of items based at least in part on the second set of confidence values.

17 . The system of claim 14 , wherein determining the third set of items comprises:

providing the first reader data and the first image data to one or more machine learned algorithms trained using reader data and a set of image data associated with a cashierless checkout system, the reader data and the set of image data tagged based on review of items to indicate a lane of a plurality of checkout lanes of the cashierless checkout system an item having a tag readable by the one or more first reader antennas passed through.

18 . The system of claim 14 , wherein receiving the first reader data comprises:

receiving second reader data from one or more second reader antennas positioned proximate a first zone of the first checkout lane in the facility; and

receiving third reader data from one or more third antennas positioned proximate a second zone of the first checkout lane, and wherein determining the checkout time comprises analyzing the second reader data and the third reader data to determine the timestamp associated with a tag transitioning from the first zone of the first checkout lane to the second zone of the first checkout lane.

19 . The system of claim 14 , wherein determining the second set of items comprises:

conveying the portion of the first image data to a user device;

generating a user interface comprising a representation of the first set of items, the second set of items, the first reader data, and the first image data; and

receiving, from the user device, an indication of the third set of items.

20 . The system of claim 14 , the acts further comprising receiving second reader data from one or more second reader antennas positioned proximate an inventory location, and wherein:

the receiving the second reader data comprises receiving second reader data indicating that an item is absent from the inventory location; and

determining the second set of items is further based on the second reader data at least partly in response to the receiving the second reader data indicating that the item is absent from the inventory location.