IP Library Granted Patent US 11,494,933
Granted Patent B2
US 11,494,933 · App. 16/917,370 · Granted Nov 8, 2022

Occluded item detection for vision-based self-checkouts

Inventors: Frank Douglas Hinek (Decatur, GA); Qian Yang (Smyrna, GA)
Assignee: NCR Corporation
G06T7/73A47F9/046G06N20/00G06Q20/14G06Q20/18G06T7/194G06V10/44G06T2207/20081G06T2207/20132G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,494,933
App. No.
16/917,370
Granted
Nov 8, 2022
Kind
B2
Abstract

Item recognition of a given item is trained on a single item from different views. The item recognition is then trained on images of the given item partially occluded by a second item having same, similar, or different shapes and features to that of the given item. General features of the item are noted and used to detect the given item when the given item is presented with multiple different items having multiple different occluded views.

Claims (29)

1. A method, comprising:

training a first machine-learning algorithm on first images to identify first items based on non-occluded views of the first items present in the first images;

training a second machine-learning algorithm on second images to identify second items based on occluded views of pairs of the second items present in the second images;

receiving a transaction image of a transaction area during a transaction at a transaction terminal;

creating bounding boxes within the transaction image comprising first bounding boxes associated with the non-occluded views for the first items and second bounding boxes associated with the occluded views for the second items;

providing the first bounding boxes to the first machine-learning algorithm and receiving back first item identifiers for corresponding first items;

providing the second bounding boxes to the second machine-learning algorithm and receiving back second item identifiers for corresponding second items; and

processing the transaction with the first item identifiers and the second item identifiers.

2. The method of claim 1 wherein training the first machine-learning algorithm further includes providing the first images as multiple different camera angle views for each of the first items.

3. The method of claim 1 , wherein training the second machine-learning algorithm further includes providing the second images as multiple different occluded views for each pair of the second items.

4. The method of claim 1 , wherein receiving further includes cropping the transaction image as a cropped image by removing a known background scene present in the transaction image.

5. The method of claim 4 , wherein creating further includes identifying features from the cropped image and processing the features to identify the non-occluded views and the occluded views.

6. The method of claim 5 , wherein providing the first bounding boxes further includes providing the corresponding features associated with the first bounding boxes to the first machine-learning algorithm.

7. The method of claim 6 , wherein providing the second bounding boxes further includes providing the corresponding features associated with the second bounding boxes to the second machine-learning algorithm.

8. A system, comprising:

a camera;

a transaction terminal;

a server comprising a processor and a non-transitory computer-readable storage medium having executable instructions; and

the executable instructions when executed by the processor from the non-transitory computer-readable storage medium cause the processor to perform processing comprising:

obtaining a transaction image of a transaction area of the transaction from the camera;

cropping the transaction image as a cropped image by removing a known background scene associated with the transaction area;

extracting features from the cropped image;

identifying single items from the cropped image based on the features;

identifying multiple items that overlap and have an occluded view based on the features;

obtaining first item identifiers for the single items from a first trained-machine learning algorithm;

obtaining second item identifiers for the multiple items from a second trained-machine learning algorithm; and

processing the first item identifiers and the second item identifiers to complete a transaction at the transaction terminal.

9. The system of claim 8 , wherein the transaction terminal is a Self-Service Terminal (SST).

10. The system of claim 9 , wherein the SST is a vision-based SST that performs item recognition for transaction items based on the transaction image by providing the first item identifiers and the second item identifiers.

Assignments (3)
CHANGE OF NAME Recorded Dec 7, 2023
From: NCR CORPORATION
To: NCR VOYIX CORPORATION
Reel/Frame 065820/0704 →
SECURITY INTEREST Recorded Oct 25, 2023
From: NCR VOYIX CORPORATION
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 065346/0168 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 28, 2020
From: HINEK, FRANK DOUGLAS; YANG, QIAN
To: NCR CORPORATION
Reel/Frame 054196/0234 →
Continuity (1)
Related Publication 20210407124A1 · Dec 30, 2021