IP Library Granted Patent US 12,657,909
Granted Patent B2
US 12,657,909 · App. 17/689,904 · Granted Jun 16, 2026

Augmented, virtual and mixed-reality content selection and display for billboards

Inventors: Stefan W. Herzberg (Nassau, BS); Megan Herzberg (Nassau, BS)
Assignee: Techinvest Company Limited
G06V20/20G06Q30/0251G06V30/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,657,909
App. No.
17/689,904
Granted
Jun 16, 2026
Kind
B2
Abstract

A billboard information system recognizes a real world billboard object and uses augmented, mixed and/or virtual reality to display information relating to the items and/or transaction shown the billboard on a smart device such as a smartphone or AR glasses.

Claims (38)

1 . A method of presenting billboard information to a user comprising:

enabling capture by a wearable camera disposed on eyeglasses, of an image of a billboard having visible indicia printed or painted thereon; and

using at least one processor operatively coupled to the wearable camera:

translating the captured image of the billboard into a compact representation;

based on the compact representation, recognizing the printed or painted visible indicia in a portion of the captured image designated as an area of recognition on a per object basis by pattern matching at least one of color and grey scale information within the area of recognition, without requiring the recognized printed or painted visible indicia to be or comprise an encoded marker, bar code, QR code or hidden printed signature;

comparing the recognized printed or painted visible indicia with database records;

based at least in part on the pattern matching, associating the billboard with a database record;

selecting an interactive media item in response to the database record associated with the billboard; and

using augmented reality to superimpose the selected interactive media item onto a view through the eyeglasses of the billboard and enabling user interaction with the selected interactive media item.

2 . The method of claim 1 wherein the superimposing further comprises using mixed reality and/or virtual reality.

3 . The method of claim 1 wherein recognizing comprises recognizing a portion of a two-dimensional or three-dimensional billboard object with print on it.

4 . The method of claim 1 wherein the billboard has indicia printed thereon, and the recognizing comprises recognizing at least some of the printed indicia.

5 . The method of claim 4 wherein the indicia comprises visible characters printed on the billboard.

6 . The method of claim 5 wherein the billboard comprises a printed billboard advertisement.

7 . The method of claim 6 wherein the selected media item comprises a digital overlay that leads to specific action selected from the group consisting of providing specific information;

a video, tutorial, or any kind of displayable content.

8 . The method of claim 1 wherein the superimposing is performed on a user's retina or on lenses of the eyeglasses.

9 . The method of claim 1 wherein the selected media item comprises an interactive call button.

10 . The method of claim 1 further including displaying any or all of the following user-interactive action buttons in any combination or subcombination:

Price Tag,

Photo Gallery,

Videos,

Description,

Call,

Mail,

Shoplink,

Explanation,

Intro,

Social Media links,

Map,

Discount codes,

Reviews,

Tutorials,

Directions,

Test drives, and/or

Booking opportunities.

11 . The method of claim 1 wherein the billboard bears at least one visible security feature within the area of recognition, and recognizing includes using pattern matching to recognize the at least one visible security feature within the area of recognition, wherein the pattern matching does not require decoding a value indicated by the at least one visible security feature.

12 . The method of claim 1 further including enabling capture by downloading the image of the billboard over a digital network.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2023
From: AR, LLC
To: TECHINVEST COMPANY LIMITED
Reel/Frame 065957/0794 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2022
From: HERZBERG, STEFAN W.; HERZBERG, MEGAN
To: AR, LLC
Reel/Frame 059977/0548 →
Continuity (11)
Continuation In Part 17641300 · Mar 8, 2022
Continuation In Part 17228821 · Apr 13, 2021
Continuation In Part 17228814
Continuation 16895637 · Jun 8, 2020
Continuation In Part 16893236 · Jun 4, 2020
Continuation 16893236 · Jun 4, 2020
Continuation 16565234 · Sep 9, 2019
Continuation In Part 16565234 · Sep 9, 2019
Continuation In Part 16565234 · Sep 9, 2019
Provisional Application 62966503 · Jan 27, 2020
Related Publication 20220262113A1 · Aug 18, 2022
References Cited (139)
US 7170391B2 · Lane · 2007 [cited by applicant]
US 9451307B2 · McClanahan · 2016 [cited by applicant]
US 9484046B2 · Knudson · 2016 [cited by examiner]
US 9886845B2 · Rhoads · 2018 [cited by applicant]
US 10019136B1 · Ozog · 2018 [cited by applicant]
US 10026190B2 · Bostick · 2018 [cited by examiner]
US 10055714B2 · Zhou · 2018 [cited by applicant]
US 10142496B1 · Rao · 2018 [cited by applicant]
US 10147076B2 · Zhou · 2018 [cited by applicant]
US 10521777B2 · Zhou · 2019 [cited by applicant]
US 10778867B1 · Goyal · 2020 [cited by examiner]
US 10825004B1 · Walker · 2020 [cited by applicant]
US 11017164B1 · Paul · 2021 [cited by applicant]
US 11157739B1 · Iskandar · 2021 [cited by applicant]
US 11205304B2 · Miller · 2021 [cited by applicant]
US 11282323B2 · Van Horn · 2022 [cited by applicant]
US 11532051B1 · Braun · 2022 [cited by applicant]
US 20030009670A1 · Rhoads · 2003 [cited by applicant]
US 20050058318A1 · Rhoads · 2005 [cited by applicant]
US 20050132420A1 · Howard · 2005 [cited by applicant]
US 20050196013A1 · Rhoads · 2005 [cited by applicant]
US 20060038833A1 · Mallinson · 2006 [cited by applicant]
US 20080204361A1 · Scales · 2008 [cited by applicant]
US 20080260210A1 · Kobeli · 2008 [cited by applicant]
US 20090078756A1 · Oberan · 2009 [cited by applicant]
US 20110098056A1 · Rhoads · 2011 [cited by applicant]
US 20110145068A1 · King · 2011 [cited by applicant]
US 20110213664A1 · Osterhout · 2011 [cited by examiner]
US 20120092329A1 · Koo · 2012 [cited by examiner]
US 20120139847A1 · Hunt · 2012 [cited by applicant]
US 20120299961A1 · Ramkumar · 2012 [cited by examiner]
US 20130206830A1 · Kugel · 2013 [cited by applicant]
US 20130210523A1 · Arumugam · 2013 [cited by applicant]
US 20130229452A1 · Roof · 2013 [cited by applicant]
US 20130278631A1 · Border · 2013 [cited by applicant]
US 20130335784A1 · Kurtz · 2013 [cited by applicant]
US 20140080428A1 · Rhoads · 2014 [cited by applicant]
US 20140100896A1 · Du · 2014 [cited by applicant]
US 20140267409A1 · Fein · 2014 [cited by applicant]
US 20140340423A1 · Taylor · 2014 [cited by applicant]
US 20150058229A1 · Wiacek · 2015 [cited by examiner]
US 20150117701A1 · Ross · 2015 [cited by applicant]
US 20150262208A1 · Bjontegard · 2015 [cited by examiner]
US 20150287290A1 · Van Horn · 2015 [cited by applicant]
US 20150332620A1 · Sako · 2015 [cited by examiner]
US 20160044383A1 · Bonvolanta · 2016 [cited by examiner]
US 20160085955A1 · Lerner · 2016 [cited by applicant]
US 20160133054A1 · Honjo · 2016 [cited by applicant]
US 20160140427A1 · Keay · 2016 [cited by applicant]
US 20160180565A1 · Moore · 2016 [cited by examiner]
US 20160209648A1 · Haddick · 2016 [cited by applicant]
US 20160225192A1 · Jones · 2016 [cited by applicant]
US 20160240010A1 · Rosenthal · 2016 [cited by examiner]
US 20160328607A1 · Krishnan · 2016 [cited by applicant]
US 20160337505A1 · Bjontegard · 2016 [cited by applicant]
US 20160357406A1 · Lee · 2016 [cited by applicant]
US 20160358383A1 · Gauglitz · 2016 [cited by applicant]
US 20170032285A1 · Sharma · 2017 [cited by applicant]
US 20170076316A1 · Heffernan · 2017 [cited by examiner]
US 20170094196A1 · Adsumilli · 2017 [cited by applicant]
US 20170221272A1 · Li · 2017 [cited by applicant]
US 20170249491A1 · Macintosh · 2017 [cited by applicant]
US 20170270698A1 · Goslin · 2017 [cited by applicant]
US 20170344823A1 · Withrow · 2017 [cited by applicant]
US 20180018671A1 · Celia · 2018 [cited by applicant]
US 20180089899A1 · Piemonte et al. · 2018 [cited by applicant]
US 20180114364A1 · McPhee · 2018 [cited by applicant]
US 20180157398A1 · Kaehler · 2018 [cited by applicant]
US 20180174366A1 · Nishibe · 2018 [cited by applicant]
US 20180181378A1 · Bakman · 2018 [cited by applicant]
US 20180208209A1 · Al-Dahle et al. · 2018 [cited by applicant]
US 20180225921A1 · Lyons · 2018 [cited by applicant]
US 20180240220A1 · Katori · 2018 [cited by applicant]
US 20180249083A1 · Chi · 2018 [cited by applicant]
US 20180253876A1 · Lin · 2018 [cited by applicant]
US 20180276895A1 · Hodge · 2018 [cited by applicant]
US 20180330167A1 · Siggard · 2018 [cited by applicant]
US 20180345129A1 · Rathod · 2018 [cited by applicant]
US 20180350144A1 · Rathod · 2018 [cited by applicant]
US 20190122384A1 · Ertle · 2019 [cited by examiner]
US 20190124272A1 · O'Neill · 2019 [cited by applicant]
US 20190149725A1 · Adato · 2019 [cited by applicant]
US 20190188920A1 · McPhee · 2019 [cited by applicant]
US 20190197789A1 · Macauley · 2019 [cited by examiner]
US 20190205645A1 · Bates · 2019 [cited by applicant]
US 20190205962A1 · Piramuthu · 2019 [cited by applicant]
US 20190220002A1 · Huang · 2019 [cited by applicant]
US 20190236594A1 · Ehrlich-Quinn · 2019 [cited by applicant]
US 20190251303A1 · Fertakos · 2019 [cited by applicant]
US 20190258204A1 · Nelson · 2019 [cited by applicant]
US 20190266404A1 · Spivack · 2019 [cited by applicant]
US 20190297186A1 · Karani · 2019 [cited by applicant]
US 20190318168A1 · Meier · 2019 [cited by applicant]
US 20190318186A1 · Sergott · 2019 [cited by applicant]
US 20190392088A1 · Duff · 2019 [cited by applicant]
US 20200019776A1 · Lewis · 2020 [cited by applicant]
US 20200045260A1 · Averhart · 2020 [cited by applicant]
US 20200082668A1 · Foley · 2020 [cited by applicant]
US 20200089014A1 · Peng et al. · 2020 [cited by applicant]
US 20200089313A1 · Himane · 2020 [cited by applicant]
US 20200160601A1 · Shreve · 2020 [cited by applicant]
US 20200225901A1 · Boissiere et al. · 2020 [cited by applicant]
US 20200233212A1 · Peimonte et al. · 2020 [cited by applicant]
US 20200233220A1 · Strongwater et al. · 2020 [cited by applicant]
US 20200233453A1 · Hatfield et al. · 2020 [cited by applicant]
US 20200233681A1 · Garstenauer et al. · 2020 [cited by applicant]
US 20200242439A1 · Micali · 2020 [cited by applicant]
US 20200394012A1 · Wright, Jr. · 2020 [cited by applicant]
US 20210004802A1 · Rule · 2021 [cited by applicant]
US 20220391867A1 · Glaser · 2022 [cited by applicant]
US 20230300420A1 · Erskine · 2023 [cited by examiner]
WO 2013138846A1 · 2013 [cited by applicant]
WO 2018222756A1 · 2018 [cited by applicant]
Gammeter et al—Server-side object recognition and client-side object tracking for mobile augmented reality—2010—IEEE (Year: 2010). [cited by examiner]
Silverbrook et al, WO2013138846A1—Method and System of Interacting With Content Disposed on Substrates, Sep. 26, 2013, WIPO (Year: 2013). [cited by examiner]
https://appleinsider.com/articles/20/07/23/apple-plans-to-dominate-all-aspects-of-ar-with-apple-glass-and-more. [cited by applicant]
International Search Report (PCT/ISA/220 & PCT/ISA/210) and Written Opinion (PCT/ISA/237) issued in PCT Application No. PCT/IB2020/058386 dated Feb. 1, 2021 (19 pages). [cited by applicant]
U.S. Appl. No. 17/641,300. [cited by applicant]
U.S. Appl. No. 17/689,900. [cited by applicant]
U.S. Appl. No. 17/689,903. [cited by applicant]
U.S. Appl. No. 17/689,904. [cited by applicant]
U.S. Appl. No. 17/689,906. [cited by applicant]
U.S. Appl. No. 17/689,911. [cited by applicant]
U.S. Appl. No. 17/689,912. [cited by applicant]
U.S. Appl. No. 17/689,913. [cited by applicant]
U.S. Appl. No. 17/689,917. [cited by applicant]
U.S. Appl. No. 17/689,920. [cited by applicant]
U.S. Appl. No. 17/689,923. [cited by applicant]
U.S. Appl. No. 17/689,926. [cited by applicant]
U.S. Appl. No. 17/689,929. [cited by applicant]
U.S. Appl. No. 17/689,931. [cited by applicant]
U.S. Appl. No. 17/689,933. [cited by applicant]
Jan. 4, 2024 Office Action in U.S. Appl. No. 18/095,866 (applicant will submit copy upon request). [cited by applicant]
Aug. 23, 2023 Office Action in U.S. Appl. No. 17/641,300 (applicant will submit copy upon request). [cited by applicant]
U.S. Office Action issued in related U.S. Appl. No. 17/689,903 dated Mar. 7, 2024. [cited by applicant]
U.S. Office Action issued in related U.S. Appl. No. 17/689,903 dated Aug. 29, 2024. [cited by applicant]
U.S. Office Action issued in related U.S. Appl. No. 17/689,933 dated Mar. 7, 2024. [cited by applicant]
U.S. Office Action issued in related U.S. Appl. No. 18/087,588 dated Aug. 29, 2024. [cited by applicant]
U.S. Office Action issued in related U.S. Appl. No. 18/095,866 dated Jan. 4, 2024. [cited by applicant]