IP Library Granted Patent US 11,538,262
Granted Patent B2
US 11,538,262 · App. 16/827,000 · Granted Dec 27, 2022

Multiple field of view (FOV) vision system

Inventors: Edward Barkan (Miller Place, NY); Mark Drzymala (Saint James, NY); Darran Michael Handshaw (Sound Beach, NY)
Assignee: Zebra Technologies Corporation
G06V30/2247G06K7/1413G06K19/06028G06N7/005G06V10/225G06V10/443
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,538,262
App. No.
16/827,000
Granted
Dec 27, 2022
Kind
B2
Abstract

Multiple field of view (FOV) systems are disclosed herein. An example system includes a bioptic barcode reader having a target imaging region. The bioptic barcode reader includes at least one imager having a first FOV and a second FOV and is configured to capture an image of a target object from each FOV. The example system includes one or more processors configured to receive the images and a trained object recognition model stored in memory communicatively coupled to the one or more processors. The memory includes instructions that, when executed, cause the one or more processors to analyze the images to identify at least a portion of a barcode and one or more features associated with the target object. The instructions further cause the one or more processors to determine a target object identification probability and to determine whether a predicted product identifies the target object.

Claims (57)

1. A bioptic barcode reader having a target object imaging region, the bioptic barcode reader comprising:

at least one imager having a first field of view (FOV) and a second FOV, wherein the at least one imager is configured to capture (i) a first image of a target object placed in a target imaging region within the first FOV and (ii) a second image of the target object within the second FOV;

one or more processors configured to receive the first image and the second image captured by the at least one imager; and

a trained object recognition model stored in a memory communicatively coupled to the one or more processors,

the memory storing instructions that, when executed by the one or more processors, cause the one or more processors to:

analyze the first image and the second image to identify at least a portion of a barcode associated with the target object in either the first image or the second image,

analyze the first image and the second image to identify one or more features associated with the target object in each of the first image and the second image,

determine, with the trained object recognition model, a target object identification probability for the target object based on (i) a set of barcode data extracted from the at least the portion of the barcode and (ii) the one or more features, wherein the target object identification probability associates the target object with a predicted product,

compare the target object identification probability to an identification probability threshold to determine whether the predicted product identifies the target object,

determine, with the trained object recognition model, a first target object identification for the target object based on the first image and a second target object identification for the target object based on the second image, wherein each of the first target object identification and the second target object identification is based on at least one of (i) the set of barcode data extracted from the at least the portion of the barcode, (ii) the one or more features, and (iii) the target object identification probability; and

responsive to determining that the first target object identification differs from the second target object identification, at least one of (i) generate a first alert for display to a user, (ii) generate a second alert for transmission to a remote monitoring system, and (iii) transmit a notification including the first target object identification and the second target object identification to a point-of-sale station.

2. The bioptic barcode reader of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

responsive to determining that the target object identification probability exceeds the identification probability threshold, compare the at least the portion of the barcode to a predicted barcode associated with the predicted product; and

responsive to determining a match between the at least the portion of the barcode and the predicted barcode, at least one of (i) identify the target object as the predicted product and (ii) train the trained object recognition model using the at least the portion of the barcode and the predicted product.

3. The bioptic barcode reader of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

generate, with the trained object recognition model, a first set of target object identification probabilities based on the first image, wherein the first set of target object identification probabilities includes a first probability associating the target object with the predicted product;

generate, with the trained object recognition model, a second set of target object identification probabilities based on the second image, wherein the second set of target object identification probabilities includes a second probability associating the target object with the predicted product; and

determine whether the predicted product identifies the target object by comparing the first probability and the second probability to the identification probability threshold.

4. The bioptic barcode reader of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

generate, with the trained object recognition model, a set of target object identification probabilities based on the first image and the second image, wherein the set of target object identification probabilities includes a first probability associating the target object with the predicted product; and

determine whether the predicted product identifies the target object by comparing the first probability to the identification probability threshold.

5. The bioptic barcode reader of claim 1 , wherein the second FOV has a relative orientation with respect to the first FOV, the at least the portion of the barcode has a barcode orientation within either the first image or the second image, and wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

determine, with the trained object recognition model, a target object identification probability for the target object based on (i) a set of barcode data extracted from the at least the portion of the barcode, (ii) the barcode orientation, (iii) the one or more features, and (iv) the relative orientation, wherein the target object identification probability associates the target object with a predicted product.

6. The bioptic barcode reader of claim 5 , wherein the first FOV has a first central axis and the second FOV has a second central axis that is substantially orthogonal to the first central axis, such that the relative orientation is substantially similar to 90°.

7. The bioptic barcode reader of claim 6 , wherein one of the first central axis or the second central axis extends in a substantially horizontal direction away from the bioptic barcode reader, and wherein one of the first central axis and the second central axis extends in a substantially vertical direction away from the bioptic barcode reader.

8. The bioptic barcode reader of claim 6 , wherein the bioptic barcode reader has no other fields of view.

9. The bioptic barcode reader of claim 1 , wherein the at least one imager comprises one imager, and the bioptic barcode reader has no other imagers configured to capture the first image and the second image.

10. The bioptic barcode reader of claim 1 , wherein the trained object recognition model comprises a convolutional neural network, and the convolutional neural network is trained with image data representative of the target object.

11. The bioptic barcode reader of claim 10 , wherein the convolutional neural network is further trained with image data representative of a barcode corresponding to the target object.

12. The bioptic barcode reader of claim 1 , wherein the one or more features includes at least one of (i) a graphic on the target object, (ii) an alphanumeric character on the target object, (iii) an edge of the target object, or (iv) a color of the target object.

13. A target object recognition method for a bioptic barcode reader having a target object imaging region, the method comprising:

capturing, with at least one imager of the bioptic barcode reader, a first image of a target object placed in a target imaging region from a first FOV and a second image of the target object from a second FOV;

analyzing the first image and the second image to identify at least a portion of a barcode associated with the target object in either the first image or the second image;

analyzing the first image and the second image to identify one or more features associated with the target object in each of the first image and the second image;

determining, with a trained object recognition model, a target object identification probability for the target object based on (i) a set of barcode data extracted from the at least the portion of the barcode and (ii) the one or more features, wherein the target object identification probability associates the target object with a predicted product;

comparing the target object identification probability to an identification probability threshold to determine whether the predicted product identifies the target object,

determine, with the trained object recognition model, a first target object identification for the target object based on the first image and a second target object identification for the target object based on the second image, wherein each of the first target object identification and the second target object identification is based on at least one of (i) the set of barcode data extracted from the at least the portion of the barcode, (ii) the one or more features, and (iii) the target object identification probability; and

responsive to determining that the first target object identification differs from the second target object identification, at least one of (i) generate a first alert for display to a user, (ii) generate a second alert for transmission to a remote monitoring system, and (iii) transmit a notification including the first target object identification and the second target object identification to a point-of-sale station.

14. The target object recognition method of claim 13 , further comprising:

responsive to determining that the target object identification probability exceeds the identification probability threshold, comparing the at least the portion of the barcode to a predicted barcode associated with the predicted product; and

responsive to determining a match between the at least the portion of the barcode and the predicted barcode, at least one of (i) identifying the target object as the predicted product and (ii) training the trained object recognition model using the at least the portion of the barcode and the predicted product.

15. The target object recognition method of claim 13 , further comprising:

generating, with the trained object recognition model, a first set of target object identification probabilities based on the first image, wherein the first set of target object identification probabilities includes a first probability associating the target object with the predicted product;

generating, with the trained object recognition model, a second set of target object identification probabilities based on the second image, wherein the second set of target object identification probabilities includes a second probability associating the target object with the predicted product; and

determining whether the predicted product identifies the target object by comparing the first probability and the second probability to the identification probability threshold.

16. The target object recognition method of claim 13 , further comprising:

generating, with the trained object recognition model, a set of target object identification probabilities based on the first image and the second image, wherein the set of target object identification probabilities includes a first probability associating the target object with the predicted product; and

determining whether the predicted product identifies the target object by comparing the first probability to the identification probability threshold.

17. The target object recognition method of claim 13 , wherein the second FOV has a relative orientation with respect to the first FOV, the at least the portion of the barcode has a barcode orientation within either the first image or the second image, and the method further comprising:

determining, with the trained object recognition model, a target object identification probability for the target object based on (i) a set of barcode data extracted from the at least the portion of the barcode, (ii) the barcode orientation, (iii) the one or more features, and (iv) the relative orientation, wherein the target object identification probability associates the target object with a predicted product.

18. The target object recognition method of claim 17 , wherein the first FOV has a first central axis and the second FOV has a second central axis that is substantially orthogonal to the first central axis, such that the relative orientation is substantially similar to 90°.

19. The target object recognition method of claim 18 , wherein one of the first central axis or the second central axis extends in a substantially horizontal direction away from the bioptic barcode reader, and wherein one of the first central axis and the second central axis extends in a substantially vertical direction away from the bioptic barcode reader.

20. The target object recognition method of claim 18 , wherein the bioptic barcode reader has no other fields of view.

21. The target object recognition method of claim 13 , wherein the at least one imager comprises one imager, and the bioptic barcode reader has no other imagers configured to capture the first image and the second image.

22. The target object recognition method of claim 13 , wherein the trained object recognition model comprises a convolutional neural network, and the convolutional neural network is trained with a first set of image data representative of the target object.

23. The target object recognition method of claim 22 , wherein the convolutional neural network is trained with and a second set of image data representative of a barcode corresponding to the target object.

24. The target object recognition method of claim 13 , wherein the one or more features includes at least one of (i) a graphic on the target object, (ii) an alphanumeric character on the target object, (iii) an edge of the target object, or (iv) a color of the target object.

Assignments (4)
SECURITY INTEREST Recorded Apr 12, 2021
From: ZEBRA TECHNOLOGIES CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 056472/0063 →
RELEASE OF SECURITY INTEREST - 364 - DAY Recorded Mar 5, 2021
From: JPMORGAN CHASE BANK, N.A.
To: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
Reel/Frame 056036/0590 →
SECURITY INTEREST Recorded Sep 1, 2020
From: ZEBRA TECHNOLOGIES CORPORATION; LASER BAND, LLC; TEMPTIME CORPORATION
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 053841/0212 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2020
From: BARKAN, EDWARD; DRZYMALA, MARK; HANDSHAW, DARRAN MICHAEL
To: ZEBRA TECHNOLOGIES CORPORATION
Reel/Frame 053333/0045 →
Continuity (1)
Related Publication 20210295078A1 · Sep 23, 2021
Cited By (2)
US 12,254,709 US 12,675,877