IP Library › Granted Patent US 12,488,211
Granted Patent B1
US 12,488,211 · App. 18/053,991 · Granted Dec 2, 2025

Label verification using image analysis

Inventors: Fabian Nater (Zurich, CH); Bernd Schoner (New York, NY); Matthias Bloch (Zurich, CH); Christian Floerkemeier (Zurich, CH)
Assignee: Scandit AG
G06K7/1491G06K7/1417G06K7/1443G06K7/1447G06T7/73G06T11/00G06V10/255G06V20/10G06V30/248
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,488,211
App. No.
18/053,991
Filed
Nov 9, 2022
Granted
Dec 2, 2025
Kind
B1
Art Unit
2876
USPC
235/462.1
Abstract

Image analysis is used to provide label verification. An image of a label within a scene is acquired. The label includes a barcode and characters. The barcode on the label is decoded to receive an identity of an object. Optical character recognition is performed to recognize the characters on the label. Recognized characters are compared to information about decoded barcode to verify the label is correct.

Claims (64)

1 . A system for optically detecting a state of a display using image analysis, the system comprising:

a camera, wherein the camera is part of a mobile device; and

one or more memory device comprising instructions that, when executed, cause one or more processors to perform the following steps:

acquiring, using the camera, a plurality of images of a scene, wherein:

the scene includes a label and a product; and

the label includes a barcode and characters of text and/or numbers;

decoding the barcode from an image of the plurality of images to receive an identity of an object and ascertain a price associated with the object based on the identity of the object;

identifying the product and a price label for the product using image recognition based on one or more images of the plurality of images; and

matching the product with the identity of the object and comparing the price label for the product with the price associated with the object to obtain one or more results.

2 . The system of claim 1 , further comprising:

receiving information about the object based on the identity of the object;

performing optical character recognition on an image of the plurality of images to recognize the characters;

comparing recognized characters to the information about the object to obtain a result of the one or more results; and

reporting the result.

3 . The system of claim 2 , wherein:

the information about the object is an intended price; and

the characters include a price on the label.

4 . The system of claim 1 , wherein the instructions, when executed, cause the one or more processors to track the barcode from frame to frame in the plurality of images.

5 . The system of claim 1 , wherein the instructions, when executed, cause the one or more processors to create a map of distances between labels, based on the plurality of images.

6 . A method for optically detecting a state of a display using image analysis, the method comprising:

acquiring a plurality of images of a scene, wherein:

the scene includes a label; and

the label includes a barcode and characters of text and/or numbers;

decoding the barcode from an image of the plurality of images to receive an identity of an object;

receiving information about the object based on the identity of the object, wherein the information is a first price associated with the object;

performing optical character recognition on an image of the plurality of images to recognize a second price associated with the object;

comparing the first price to the second price to obtain a result; and

reporting the result.

7 . The method of claim 6 , wherein:

a portion of the characters include the second price.

8 . The method of claim 6 , wherein:

the scene includes a product; and

the method comprises:

identifying the product using image recognition based on one or more of the plurality of images; and

matching the product with the identity of the object based on image recognition of the product and decoding the barcode.

9 . The method of claim 6 , comprising tracking the barcode from frame to frame in the plurality of images.

10 . The method of claim 6 , wherein the barcode is a two-dimensional barcode.

11 . The method of claim 6 , comprising providing a user interface to receive data about the label from a user.

12 . The method of claim 6 , wherein matching is performed through a web browser.

13 . The method of claim 6 , comprising creating a map of labels and distances between labels.

14 . The method of claim 6 , comprising:

saving an image of the scene around the barcode; and

saving a higher-resolution image of the barcode than the image of the scene around the barcode.

15 . The method of claim 6 , comprising:

saving image data of one or more of the plurality of images to a database; and

saving the identity of the object to the database.

16 . A memory device comprising instructions that, when executed, cause one or more processors to perform the following steps:

acquiring a plurality of images of a scene, wherein:

the scene includes a label; and

the label includes a barcode and characters of text and/or numbers;

decoding the barcode from an image of the plurality of images to receive an identity of an object;

receiving information about the object based on the identity of the object, wherein the information is a first price associated with the object;

performing optical character recognition on an image of the plurality of images to recognize a second price associated with the object;

comparing the first price to the second price to obtain a result; and

reporting the result.

17 . The memory device of claim 16 , wherein:

a portion of the characters include the second price.

18 . The memory device of claim 16 , wherein:

the scene includes a product; and

the instructions, when executed, cause the one or more processors to perform the following steps:

identifying the product using image recognition based on one or more of the plurality of images; and

matching the product with the identity of the object based on image recognition of the product and decoding the barcode.

19 . The memory device of claim 16 , wherein the instructions, when executed, cause the one or more processors to track the barcode from frame to frame in the plurality of images.

20 . The memory device of claim 16 , wherein the instructions, when executed, cause the one or more processors to create a map of labels and distances between labels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2025
From: NATER, FABIAN; ROIMELA, KIMMO; SCHONER, BERND; BLOCH, MATTHIAS; FLOERKEMEIER, CHRISTIAN
To: SCANDIT AG
Reel/Frame 070515/0764 →
Continuity (6)
Continuation 17244251 · Apr 29, 2021
Continuation In Part 17139529 · Dec 31, 2020
Continuation 16920061 · Jul 2, 2020
Provisional Application 63017493 · Apr 29, 2020
Provisional Application 63003675 · Apr 1, 2020
Provisional Application 63019818 · May 4, 2020
References Cited (197)
US 4167756A · Smith · 1979 [cited by applicant]
US D344261S · Watanabe · 1994 [cited by applicant]
US 5744815A · Gurevich et al. · 1998 [cited by applicant]
US D473872S · Ausems et al. · 2003 [cited by applicant]
US 6580453B1 · Hirasawa · 2003 [cited by applicant]
US D576197S · Takagi · 2008 [cited by applicant]
US 7457407B2 · Sun et al. · 2008 [cited by applicant]
US D654931S · Lemelman et al. · 2012 [cited by applicant]
US D658174S · Tasselli et al. · 2012 [cited by applicant]
US D659564S · Baxter · 2012 [cited by applicant]
US 8223203B2 · Ohsumi et al. · 2012 [cited by applicant]
US D667823S · Merenda · 2012 [cited by applicant]
US D670278S · Hamann · 2012 [cited by applicant]
US D672386S · Matunuma et al. · 2012 [cited by applicant]
US D678870S · Fathollahi · 2013 [cited by applicant]
US D678936S · Oliver · 2013 [cited by applicant]
US D685360S · Chen et al. · 2013 [cited by applicant]
US D688654S · Stevinson · 2013 [cited by applicant]
US 8596540B2 · Adelmann · 2013 [cited by applicant]
US D698772S · Merenda · 2014 [cited by applicant]
US 8744173B2 · Vincent et al. · 2014 [cited by applicant]
US D710343S · Chandler, Jr. et al. · 2014 [cited by applicant]
US D710346S · Smith et al. · 2014 [cited by applicant]
US 8798453B2 · Lawton · 2014 [cited by applicant]
US D716285S · Chaney et al. · 2014 [cited by applicant]
US D716785S · White · 2014 [cited by applicant]
US D717287S · Macrina et al. · 2014 [cited by applicant]
US D717304S · Yturralde et al. · 2014 [cited by applicant]
US D719166S · Brown et al. · 2014 [cited by applicant]
US D719167S · Brown et al. · 2014 [cited by applicant]
US D724573S · Stevinson · 2015 [cited by applicant]
US D726701S · Stevinson · 2015 [cited by applicant]
US 9019420B2 · Hurst et al. · 2015 [cited by applicant]
US D728551S · Saeki et al. · 2015 [cited by applicant]
US D732011S · Stevinson · 2015 [cited by applicant]
US D733112S · Chaney et al. · 2015 [cited by applicant]
US D734336S · Mistkawi et al. · 2015 [cited by applicant]
US D744470S · Stevinson · 2015 [cited by applicant]
US D748085S · Merenda · 2016 [cited by applicant]
US D754114S · Curtis et al. · 2016 [cited by applicant]
US D754650S · Curtis et al. · 2016 [cited by applicant]
US D759004S · Stevinson · 2016 [cited by applicant]
US D760209S · Weng et al. · 2016 [cited by applicant]
US D760212S · Mao et al. · 2016 [cited by applicant]
US 9378435B1 · Prulhiere et al. · 2016 [cited by applicant]
US D760710S · Ozolins et al. · 2016 [cited by applicant]
US D761240S · Ozolins et al. · 2016 [cited by applicant]
US D768617S · Merenda · 2016 [cited by applicant]
US D771631S · Fitch et al. · 2016 [cited by applicant]
US 9626577B1 · Yu et al. · 2017 [cited by applicant]
US 9654675B2 · Kessler · 2017 [cited by applicant]
US 9659204B2 · Wu et al. · 2017 [cited by applicant]
US 9836635B2 · Negro et al. · 2017 [cited by applicant]
US 9864969B2 · Yasunaga · 2018 [cited by examiner]
US 9911033B1 · Harary · 2018 [cited by examiner]
US 10013643B2 · Yellaprada et al. · 2018 [cited by applicant]
US 10191242B2 · Palmeri · 2019 [cited by applicant]
US 10200599B1 · Baldwin · 2019 [cited by applicant]
US 10229301B2 · Cumoli et al. · 2019 [cited by applicant]
US D860180S · Lehmann et al. · 2019 [cited by applicant]
US D862441S · Eppler et al. · 2019 [cited by applicant]
US 10426442B1 · Schnorr · 2019 [cited by applicant]
US 10452959B1 · Gautam et al. · 2019 [cited by applicant]
US 10455163B2 · Kanatsu · 2019 [cited by applicant]
US 10489668B2 · Quentin et al. · 2019 [cited by applicant]
US 10558844B2 · D'ercoli et al. · 2020 [cited by applicant]
US 10621435B2 · Bridges et al. · 2020 [cited by applicant]
US 10818014B2 · Xu et al. · 2020 [cited by applicant]
US 10846561B1 · Floerkemeier et al. · 2020 [cited by applicant]
US 10963658B1 · Bloch et al. · 2021 [cited by applicant]
US 11003891B2 · Chen · 2021 [cited by applicant]
US 11087105B1 · Biasini et al. · 2021 [cited by applicant]
US 11216628B2 · Scherly et al. · 2022 [cited by applicant]
US 11244147B2 · Floerkemeier et al. · 2022 [cited by applicant]
US 11290643B1 · Tullis et al. · 2022 [cited by applicant]
US 11295163B1 · Schoner et al. · 2022 [cited by applicant]
US 20030059124A1 · Center, Jr. · 2003 [cited by applicant]
US 20030202697A1 · Simard et al. · 2003 [cited by applicant]
US 20060011724A1 · Joseph · 2006 [cited by examiner]
US 20060075241A1 · Deguillaume et al. · 2006 [cited by applicant]
US 20060249581A1 · Smith · 2006 [cited by applicant]
US 20070116454A1 · Tsai · 2007 [cited by applicant]
US 20090002797A1 · Kwong et al. · 2009 [cited by applicant]
US 20090033786A1 · Finkelstein et al. · 2009 [cited by applicant]
US 20090108071A1 · Carlson · 2009 [cited by applicant]
US 20090212113A1 · Chiu et al. · 2009 [cited by applicant]
US 20090304234A1 · Kondo et al. · 2009 [cited by applicant]
US 20100102129A1 · Drzymala et al. · 2010 [cited by applicant]
US 20100252633A1 · Barkan et al. · 2010 [cited by applicant]
US 20100328420A1 · Roman · 2010 [cited by applicant]
US 20110007205A1 · Lee · 2011 [cited by applicant]
US 20110043683A1 · Beach et al. · 2011 [cited by applicant]
US 20110081946A1 · Singh · 2011 [cited by applicant]
US 20110168776A1 · Jalali et al. · 2011 [cited by applicant]
US 20130076697A1 · Goertz et al. · 2013 [cited by applicant]
US 20130112750A1 · Negro et al. · 2013 [cited by applicant]
US 20130147839A1 · Fukushima et al. · 2013 [cited by applicant]
US 20130206839A1 · Gao · 2013 [cited by applicant]
US 20130329115A1 · Palmeri · 2013 [cited by applicant]
US 20140025973A1 · Schillings et al. · 2014 [cited by applicant]
US 20140027503A1 · Kennedy et al. · 2014 [cited by applicant]
US 20140168468A1 · Levoy et al. · 2014 [cited by applicant]
US 20140171150A1 · Hurst et al. · 2014 [cited by applicant]
US 20140232930A1 · Anderson · 2014 [cited by applicant]
US 20140247278A1 · Samara et al. · 2014 [cited by applicant]
US 20140285913A1 · Palmeri · 2014 [cited by applicant]
US 20140327815A1 · Auger · 2014 [cited by applicant]
US 20140344729A1 · Blanco et al. · 2014 [cited by applicant]
US 20150048167A1 · Russell et al. · 2015 [cited by applicant]
US 20150053765A1 · Powell et al. · 2015 [cited by applicant]
US 20150116547A1 · Laroia · 2015 [cited by applicant]
US 20150220766A1 · Russell et al. · 2015 [cited by applicant]
US 20160042315A1 · Field-Darragh et al. · 2016 [cited by applicant]
US 20160070944A1 · McCloskey et al. · 2016 [cited by applicant]
US 20160077307A1 · Palmeri · 2016 [cited by applicant]
US 20160104021A1 · Negro et al. · 2016 [cited by applicant]
US 20160171707A1 · Schwartz · 2016 [cited by examiner]
US 20160188941A1 · Todeschini et al. · 2016 [cited by applicant]
US 20160219218A1 · Kinoshita · 2016 [cited by applicant]
US 20160253599A1 · Lang et al. · 2016 [cited by applicant]
US 20160307006A1 · Wang · 2016 [cited by applicant]
US 20160321819A1 · Morgan-Mar et al. · 2016 [cited by applicant]
US 20160323508A1 · Ayalasomayajula et al. · 2016 [cited by applicant]
US 20170013179A1 · Kang et al. · 2017 [cited by applicant]
US 20170032311A1 · Rizzolo et al. · 2017 [cited by applicant]
US 20170041540A1 · Foster et al. · 2017 [cited by applicant]
US 20170185400A1 · Shanbhogue et al. · 2017 [cited by applicant]
US 20170243097A1 · Loy et al. · 2017 [cited by applicant]
US 20170286732A1 · Lei et al. · 2017 [cited by applicant]
US 20180081417A1 · Chan et al. · 2018 [cited by applicant]
US 20180101813A1 · Paat · 2018 [cited by examiner]
US 20180120106A1 · Sato · 2018 [cited by applicant]
US 20180122194A1 · Schoner · 2018 [cited by applicant]
US 20180137319A1 · Giordano et al. · 2018 [cited by applicant]
US 20180139337A1 · Ghazizadeh · 2018 [cited by applicant]
US 20180157885A1 · Gurzumar · 2018 [cited by applicant]
US 20180314908A1 · Lam · 2018 [cited by applicant]
US 20180342050A1 · Fitzgerald et al. · 2018 [cited by applicant]
US 20190065803A1 · Burke et al. · 2019 [cited by applicant]
US 20190087772A1 · Medina · 2019 [cited by examiner]
US 20190122012A1 · Lei et al. · 2019 [cited by applicant]
US 20190188435A1 · Davis et al. · 2019 [cited by applicant]
US 20190213523A1 · Adato et al. · 2019 [cited by applicant]
US 20190244043A1 · Bradley et al. · 2019 [cited by applicant]
US 20190304132A1 · Yoda et al. · 2019 [cited by applicant]
US 20190325183A1 · Tscherepanow et al. · 2019 [cited by applicant]
US 20190354923A1 · Taira et al. · 2019 [cited by applicant]
US 20200042803A1 · Yamaguchi · 2020 [cited by applicant]
US 20200084375A1 · Tadano et al. · 2020 [cited by applicant]
US 20200202095A1 · Yoda · 2020 [cited by examiner]
US 20200258033A1 · Yonezawa et al. · 2020 [cited by applicant]
US 20200273066A1 · Yonezawa · 2020 [cited by examiner]
US 20200311659A1 · Yonezawa · 2020 [cited by examiner]
US 20210012272A1 · Dugar · 2021 [cited by examiner]
US 20210125141A1 · Lipsey et al. · 2021 [cited by applicant]
US 20210150618A1 · Glaser et al. · 2021 [cited by applicant]
US 20210158278A1 · Bogolea et al. · 2021 [cited by applicant]
US 20210192162A1 · Rodriguez et al. · 2021 [cited by applicant]
US 20220292446A1 · Teramoto · 2022 [cited by examiner]
US 20230139490A1 · Narendra · 2023 [cited by examiner]
CN 108596295A · 2018 [cited by examiner]
EP 3167404A1 · 2017 [cited by applicant]
JP 2004032507A · 2004 [cited by applicant]
JP 4966464B2 · 2012 [cited by applicant]
JP 2019094191A · 2019 [cited by applicant]
KR 20020077090A · 2002 [cited by applicant]
KR 20060102957A · 2006 [cited by applicant]
WO 0131893A1 · 2001 [cited by applicant]
WO 2016007662A1 · 2016 [cited by applicant]
WO 2019135163A2 · 2019 [cited by applicant]
U.S. Appl. No. 18/051,590, filed Nov. 1, 2022, Notice of Allowance mailed Aug. 30, 2023, all pages. [cited by applicant]
U.S. Appl. No. 17/139,529, filed Dec. 31, 2020, Notice of Allowance mailed Sep. 19, 2023, all pages. [cited by applicant]
U.S. Appl. No. 17/686,721, filed Mar. 4, 2022, Notice of Allowance mailed Oct. 4, 2023, all pages. [cited by applicant]
U.S. Appl. No. 17/541,957, filed Dec. 3, 2021, Notice of Allowance mailed Oct. 18, 2023, all pages. [cited by applicant]
“Code Reader 4405 User Manual”, Code Product Line Version 03, Release date: Jan. 2016,. Code Corporation, Accessed on: Dec. 22, 2020 [Online]. Retrieved from: https://web.archive.org/web/20181223184651/http://www.codeco… [cited by applicant]
“Flens—The First Flashlight Booster for Smartphones”, created by Basilico, Kickstarter, last updated Dec. 11, 2019, Accessed on Mar. 26, 2021 from https://kickstarter.com/projects/basilico/flen-the-first-flashlight-boos… [cited by applicant]
“Linea Pro Extreme Rugged Case”. Infinite Peripherals, iOS Accessories, Linea Pro and Infinea Tab Accessories, Retrieved on Dec. 22, 2020 from https://web.archive.org/web/20150825044354/http://ipcprint.com/linea-pro-ext… [cited by applicant]
Brownlee; Jason, “Deep Learning Models for Human Activity Recognition”, Deep Learning for Time Series, Navigation, last updated Aug. 5, 2019, 16 pages. [cited by applicant]
Scandit, “Scandit Augmented Reality Retail Price Label Verification”, posted Sep. 26, 2018, retrieved on Aug. 4, 2020 from https://www.scandit.com/resources/videos/scandit-augmented reality-retail-price-label-verificati… [cited by applicant]
Scandit, “Scandit Augmented Reality for Smart Devices”, posted Oct. 2, 2019, retrieved on Dec. 2, 2020 from https://www.scandit.com/resources/videos/scandit-augmented-reality-for-smart-devices/, 3 pages. [cited by applicant]
Scandit, “Scandit Augmented Retail Product Information Using AR”, posted Sep. 26, 2018, retrieved on Dec. 2, 2020 from https://www.scandit.com/resources/videos/scandit-augmented-retail-product-information-using-ar/, 5 p… [cited by applicant]
Scandit, “Scandit Augmented Reality Retail Click and Collect”, posted Sep. 26, 2018, retrieved on Dec. 2, 2020 from https://www.scandit.com/resources/videos/scandit-augmented-reality-retail-click-and-collect/, 4 pages. [cited by applicant]
Scandit, “Scandit Augmented Reality Retail Shelf Management”, posted Sep. 26, 2018, retrieved on Dec. 2, 2020 from https://www.scandit.com/resources/videos/scandit-augmented-reality-retail-shelf-management/, 4 pages. [cited by applicant]
Scandit, “Computer Vision and Augmented Reality for Enterprise Applications”, posted Jun. 6, 2018, retrieved on Dec. 2, 2020 from https://www.scandit.com/resources/videos/computer-vision-and-augmented reality-for-enterp… [cited by applicant]
Invitation to Pay Additional Fees And, Where Applicable, Protest Fee, mailed Jun. 17, 2021 in International Patent Application No. PCT/US2021/025212, 2 pages. [cited by applicant]
International Search Report and Written Opinion for PCT/US2021/025212 mailed Aug. 12, 2001, 11 pages. [cited by applicant]
U.S. Appl. No. 16/920,061 received a Pre-Interview First Office Action mailed on Sep. 16, 2020, 6 pages. [cited by applicant]
U.S. Appl. No. 16/920,061 Notice of Allowance mailed on Dec. 2, 2020, 10 pages. [cited by applicant]
U.S. Appl. No. 17/016,896 Notice of Allowance mailed Nov. 16, 2022, 15 pages. [cited by applicant]
U.S. Appl. No. 17/139,529 First Action Interview Pilot Program Pre-Interview Communication mailed Dec. 22, 2022, 6 pages. [cited by applicant]
U.S. Appl. No. 17/396,123 Non-Final Office Action mailed Dec. 2, 2021, 6 pages. [cited by applicant]
U.S. Appl. No. 17/468,376 Notice of Allowance mailed Nov. 29, 2021, 14 pages. [cited by applicant]
U.S. Appl. No. 17/549,217 Non-Final Office Action mailed Feb. 28, 2023, 8 pages. [cited by applicant]
U.S. Appl. No. 17/549,805 Non-Final Office Action mailed on Mar. 2, 2022, 10 pages. [cited by applicant]
U.S. Appl. No. 17/549,805 Notice of Allowance mailed on Sep. 29, 2022, 9 pages. [cited by applicant]
U.S. Appl. No. 17/565,131 Notice of Allowance mailed Dec. 8, 2022, 12 pages. [cited by applicant]
U.S. Appl. No. 17/837,870 Non-Final Office Action mailed on Mar. 1, 2023, 8 pages. [cited by applicant]