IP Library Granted Patent US 11,600,115
Granted Patent B2
US 11,600,115 · App. 16/928,836 · Granted Mar 7, 2023

Barcode scanning based on gesture detection and analysis

Inventors: Ozgur Ozserin (London, GB); Luke Walton (Bromley, GB)
Assignee: Zebra Technologies Corporation
G06V40/28G06K7/10722G06K7/1413G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,600,115
App. No.
16/928,836
Granted
Mar 7, 2023
Kind
B2
Abstract

A system for decoding a barcode based on gesture detection and analysis is disclosed herein. The system may receive an image stream of an environment. The system may determine, based on processing images of the image stream, a gesture region of the image stream. The system may detect a gesture, by a user, based on analyzing the gesture region. The system may determine a barcode region of the image stream based on a position of the user when performing the gesture. The system may identify a barcode in the environment based on analyzing the barcode region. The system may decode the barcode based on a format of the barcode.

Claims (86)

1. A method, comprising:

receiving, by a device, an image stream that depicts an environment;

monitoring, by the device and using an image processing model, the image stream to detect a set of gestures in the environment;

determining, by the device and using the image processing model, that a first section of the image stream depicts a first gesture of the set of gestures;

determining, by the device and using the image processing model, that the first gesture is associated with indicating a barcode region of the image stream;

analyzing, by the device and using the image processing model, the barcode region of the image stream to identify a barcode;

determining, by the device and using the image processing model, that a second section of the image stream depicts a second gesture of the set of gestures that is associated with decoding the barcode;

decoding, by the device, the barcode based on determining that the second section of the image stream depicts the second gesture; and

performing, by the device, an action associated with an object that is associated with the barcode.

2. The method of claim 1 , wherein analyzing the barcode region comprises:

identifying a position of a user member that performed of the first gesture;

determining that the barcode region corresponds to an area of the environment that is depicted within a threshold distance of the position of the user member; and

identifying the barcode within the barcode region based on the barcode being depicted within the threshold distance.

3. The method of claim 1 , wherein the barcode is identified based on the image processing model being configured to:

identify that the image stream depicts the barcode based on planar configurations of other barcodes that are associated with the barcode; and

process the barcode region based on determining that the barcode is depicted in association with one of the planar configurations.

4. The method of claim 1 , wherein the second gesture is associated with a gesture movement that corresponds to a user transitioning a user member from a gesture position of the first gesture to a gesture position of the second gesture.

5. The method of claim 1 , wherein decoding the barcode comprises:

causing a laser scanner to scan the barcode; and

processing scan information received from the laser scanner to decode the barcode.

6. The method of claim 1 , wherein decoding the barcode comprises:

processing, using the image processing model, the barcode region to extract an image of the barcode from the image stream;

formatting, using a barcode scanning model, the image for decoding to form a formatted image; and

decoding, using the barcode scanning model, the barcode based on the formatted image.

7. The method of claim 1 , wherein performing the action comprises at least one of:

providing status information associated with the object relative to the environment,

providing location information associated with the object relative to the environment,

providing transaction information associated with the object relative to the environment, and

providing user information associated with a user processing the object.

8. A device, comprising:

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, configured to:

receive an image stream that depicts an environment;

monitor, using an image processing model, the image stream to detect a set of gestures in the environment;

determine, using the image processing model, that a first section of the image stream depicts a first gesture of the set of gestures;

determine, using the image processing model, that the first gesture is associated with indicating a barcode region of the image stream;

analyze, using the image processing model, the barcode region of the image stream to identify a barcode that is in the environment;

determine, using the image processing model, that a second section of the image stream depicts a second gesture of the set of gestures that is associated with decoding the barcode; and

decode the barcode based on the image stream depicting the second gesture.

9. The device of claim 8 , wherein the one or more processors, when analyzing the barcode region, are configured to:

determine the barcode region based on a position of a user member that performed of the first gesture;

determine that the barcode region corresponds to an area of the environment that is depicted within a threshold distance of the position of the user member; and

identify the barcode within the barcode region based on the barcode being within the threshold distance.

10. The device of claim 8 , wherein the barcode is identified based on the image processing model being configured to:

identify that the image stream depicts the barcode based on a format of the barcode; and

process the barcode region based on the format of the barcode to verify that the barcode region depicts the barcode,

wherein the barcode is decoded based on the barcode region being processed to verify that the barcode region depicts the barcode.

11. The device of claim 8 ,

wherein the image processing model is configured to:

detect the set of gestures in a foreground of the image stream, and

identify the barcode in a background of the image stream.

12. The device of claim 8 , wherein the one or more processors, prior to determining that the second section depicts the second gesture, are configured to:

augment a barcode identifier in association with the barcode via a display of a user interface that is associated with the environment.

13. The device of claim 8 ,

wherein the one or more processors, when decoding the barcode, are configured to:

process, using the image processing model, the barcode region to extract an image of the barcode from the image stream; and

decode, using a barcode scanning model, the barcode based on the image of the barcode.

14. The device of claim 8 , wherein the one or more processors are further configured to:

perform, based on decoding the barcode, an action associated with an object,

wherein the object is associated with the barcode, and

wherein the one or more processors, when performing the action, are configured to at least one of:

provide status information associated with the object relative to the environment,

provide location information associated with the object relative to the environment,

provide transaction information associated with the object relative to the environment, and

provide user information associated with a user processing the object.

15. A tangible machine-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the device to:

receive an image stream of an environment;

determine, based on processing images of the image stream, a gesture region of the image stream;

detect a gesture, by a user, based on analyzing the gesture region;

determine a barcode region of the image stream based on a position of the user when performing the gesture;

identify a barcode in the environment based on analyzing the barcode region; and

decode the barcode based on a format of the barcode.

16. The tangible machine-readable medium of claim 15 , wherein the gesture region is determined based on the images depicting a user member of the user in a foreground of the image stream.

17. The tangible machine-readable medium of claim 15 , wherein the barcode region is determined based on the barcode region being in a background, relative to the gesture region, of the image stream.

18. The tangible machine-readable medium of claim 15 , wherein the gesture region and the barcode region correspond to three-dimensional zones of the image stream, and wherein the format of the barcode corresponds to a two-dimensional depiction of the barcode within the barcode region.

19. The tangible machine-readable medium of claim 15 ,

wherein the one or more instructions, that cause the one or more processors to decode the barcode, cause the one or more processors to at least one of:

decode the barcode using at least one of:

a laser scan of the barcode; and

a barcode scanning model that utilizes an image processing technique to decode the barcode using an image from the image stream.

20. The tangible machine-readable medium of claim 15 , wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to at least one of:

provide status information associated with an object that is associated with the barcode,

provide location information associated with the object,

provide transaction information associated with the object, and

provide user information associated with the user and the object.

Assignments (4)
SECURITY INTEREST Recorded Apr 12, 2021
From: ZEBRA TECHNOLOGIES CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 056471/0868 →
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2020
From: OZSERIN, OZGUR; WALTON, LUKE
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 053330/0535 →