IP Library Granted Patent US 12,694,221
Granted Patent B2
US 12,694,221 · App. 18/597,470 · Granted Jul 28, 2026

Dynamic content rendering based on context for AR and assistant systems

Inventors: Yiming Pu (Santa Clara, CA); Christopher E Balmes (San Francisco, CA); Gabrielle Catherine Moskey (San Mateo, CA); John Jacob Blakeley (Bothell, WA); Amy Lawson Bearman (Emerald Hills, CA); Alireza Dirafzoon (San Francisco, CA); Matthew Dan Feiszli (San Francisco, CA); Ganesh Venkatesh (San Jose, CA); Babak Damavandi (San Francisco, CA); Jiwen Ren (Brooklyn, NY); Chengyuan Yan (San Bruno, CA); Guangqiang Dong (Sammamish, WA)
Assignee: Meta Platforms, Inc.
G06F40/30G06F1/3206G06F3/011G06F3/04815G06F16/122G06F16/95G06F40/205G06F40/35G06N5/02G06N5/046G06N20/00G06T19/006G06T19/20G10L15/083G10L15/1815H04L67/133G06F40/295G06T2219/2004G10L15/22G10L2015/223G10L15/26G10L15/30H04L51/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,694,221
App. No.
18/597,470
Filed
Mar 6, 2024
Granted
Jul 28, 2026
Kind
B2
Art Unit
2615
USPC
715/848
Abstract

In one embodiment, a method includes rendering a first output image comprising one or more augmented-reality (AR) objects for displays of an AR rendering device of an AR system associated with a first user. The method further includes accessing sensor signals associated with the first user. The one or more sensor signals may be captured by sensors of the AR system. The method further includes detecting a change in a context of the first user with respect to a real-world environment based on the sensor signals. The method further includes rendering a second output image comprising the AR objects for the displays of the AR rendering device. One or more of the AR objects may be adapted based on the detected change in the context of the first user.

Claims (38)

1 . A method comprising:

rendering, for a display of an augmented-reality (AR) device associated with a user, an output image comprising one or more AR objects, wherein the one or more AR objects comprise a user interface;

detecting a gaze of the user directed towards a first portion of the user interface;

in response to detecting the gaze of the user towards the first portion of the user interface, displaying an indication of a notification on a second portion of the user interface;

receiving, from the user, an interaction with the displayed indication; and

in response to receiving the interaction, performing an operation associated with content associated with the notification.

2 . The method of claim 1 , wherein the user interface comprises a notifications region, and the second portion of the user interface comprises the notifications region.

3 . The method of claim 2 , wherein the notifications region is located at a top portion of a field of view of the user.

4 . The method of claim 2 , wherein the first portion of the user interface comprises the notifications region.

5 . The method of claim 1 , wherein the output image comprises a real-world scene.

6 . The method of claim 2 , wherein the AR device is operating in a glance power state in which the notifications region is displayed relative to a real-world scene.

7 . The method of claim 6 , wherein the output image comprises a first layer and a second layer, the first layer comprises the real-world scene, and the second layer comprises the user interface,

wherein the AR device is operating in a standby power state in which the second layer is hidden by default behind the first layer, and

wherein displaying the notification on the second portion of the user interface comprises displaying the second layer in front of the first layer as an overlay.

8 . The method of claim 1 , wherein the notification is a push notification.

9 . The method of claim 1 , wherein the indication is displayed in further response to an input gesture performed by the user.

10 . A non-transitory computer-readable storage medium embodying software that is operable, when executed, to:

render, for a display of an augmented-reality (AR) device associated with a user, an output image comprising one or more AR objects, wherein the one or more AR objects comprise a user interface;

detect a gaze of the user directed towards a first portion of the user interface;

in response to detecting the gaze of the user towards the first portion of the user interface, display an indication of a notification on a second portion of the user interface;

receive, from the user, an interaction with the displayed indication; and

in response to receiving the interaction, perform an operation associated with content associated with the notification.

11 . The non-transitory computer-readable storage medium of claim 10 , wherein the user interface comprises a notifications region, and the second portion of the user interface comprises the notifications region.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein the first portion of the user interface comprises the notifications region.

13 . The non-transitory computer-readable storage medium of claim 10 ,

wherein the notification is a push notification.

14 . The non-transitory computer-readable storage medium of claim 10 , wherein the indication is displayed in further response to an input gesture performed by the user.

15 . The non-transitory computer-readable storage medium of claim 10 , wherein the output image comprises a real-world scene.

16 . A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the instructions, when executed by the processors, cause the system to:

render, for a display of an augmented-reality (AR) device associated with a user, an output image comprising one or more AR objects, wherein the one or more AR objects comprise a user interface;

detect a gaze of the user directed towards a first portion of the user interface;

in response to detecting the gaze of the user towards the first portion of the user interface, display an indication of a notification on a second portion of the user interface;

receive, from the user, an interaction with the displayed indication; and

in response to receiving the interaction, perform an operation associated with content associated with the notification.

17 . The system of claim 16 , wherein the user interface comprises a notifications region, and the second portion of the user interface comprises the notifications region.

18 . The system of claim 16 , wherein the notification is a push notification.

19 . The system of claim 16 , wherein the indication is displayed in further response to an input gesture performed by the user.

20 . The system of claim 16 , wherein the output image comprises a real-world scene.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2024
From: PU, YIMING; BALMES, CHRISTOPHER E.; MOSKEY, GABRIELLE CATHERINE; BLAKELEY, JOHN JACOB; BEARMAN, AMY LAWSON; DIRAFZOON, ALIREZA; FEISZLI, MATTHEW DAN; VENKATESH, GANESH; DAMAVANDI, BABAK; REN, JIWEN; YAN, CHENGYUAN; DONG, GUANGQIANG
To: FACEBOOK, INC.
Reel/Frame 067323/0267 →
CHANGE OF NAME Recorded May 6, 2024
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 067328/0965 →
Continuity (3)
Continuation 17391765 · Aug 2, 2021
Provisional Application 63177812 · Apr 21, 2021
Related Publication 20240296289A1 · Sep 5, 2024
References Cited (359)
US 5282267A · Woo, Jr. et al. · 1994 [cited by applicant]
US 5926537A · Birze · 1999 [cited by applicant]
US 6014439A · Walker et al. · 2000 [cited by applicant]
US 6504571B1 · Narayanaswami et al. · 2003 [cited by applicant]
US 7124123B1 · Roskind et al. · 2006 [cited by applicant]
US 7158678B2 · Nagel et al. · 2007 [cited by applicant]
US 7397912B2 · Aasman et al. · 2008 [cited by applicant]
US 7613287B1 · Stifelman et al. · 2009 [cited by applicant]
US 7889853B2 · Sutcliffe · 2011 [cited by applicant]
US 8027451B2 · Arendsen et al. · 2011 [cited by applicant]
US 8204200B2 · Wang et al. · 2012 [cited by applicant]
US 8560564B1 · Hoelzle et al. · 2013 [cited by applicant]
US 8660247B1 · Brown · 2014 [cited by applicant]
US 8677377B2 · Cheyer et al. · 2014 [cited by applicant]
US 8935192B1 · Ventilla et al. · 2015 [cited by applicant]
US 8983383B1 · Haskin · 2015 [cited by applicant]
US 9154739B1 · Nicolaou et al. · 2015 [cited by applicant]
US 9251471B2 · Pinckney et al. · 2016 [cited by applicant]
US 9299059B1 · Marra et al. · 2016 [cited by applicant]
US 9304736B1 · Whiteley et al. · 2016 [cited by applicant]
US 9338242B1 · Suchland et al. · 2016 [cited by applicant]
US 9338493B2 · Van Os et al. · 2016 [cited by applicant]
US 9390724B2 · List · 2016 [cited by applicant]
US 9418658B1 · David et al. · 2016 [cited by applicant]
US 9472206B2 · Ady · 2016 [cited by applicant]
US 9479931B2 · Ortiz, Jr. et al. · 2016 [cited by applicant]
US 9576574B2 · Van Os · 2017 [cited by applicant]
US 9639608B2 · Freeman · 2017 [cited by applicant]
US 9659577B1 · Langhammer · 2017 [cited by applicant]
US 9720955B1 · Cao et al. · 2017 [cited by applicant]
US 9747895B1 · Jansche et al. · 2017 [cited by applicant]
US 9792281B2 · Sarikaya · 2017 [cited by applicant]
US 9858925B2 · Gruber et al. · 2018 [cited by applicant]
US 9865260B1 · Vuskovic et al. · 2018 [cited by applicant]
US 9875233B1 · Tomkins et al. · 2018 [cited by applicant]
US 9875741B2 · Gelfenbeyn et al. · 2018 [cited by applicant]
US 9881077B1 · Alfonseca et al. · 2018 [cited by applicant]
US 9886953B2 · Lemay et al. · 2018 [cited by applicant]
US 9990591B2 · Gelfenbeyn et al. · 2018 [cited by applicant]
US 10042032B2 · Scott et al. · 2018 [cited by applicant]
US 10127220B2 · Bellegarda et al. · 2018 [cited by applicant]
US 10134395B2 · Typrin · 2018 [cited by applicant]
US 10162886B2 · Wang et al. · 2018 [cited by applicant]
US 10199051B2 · Binder et al. · 2019 [cited by applicant]
US 10241752B2 · Lemay et al. · 2019 [cited by applicant]
US 10276170B2 · Gruber et al. · 2019 [cited by applicant]
US 10311833B1 · Qiu et al. · 2019 [cited by applicant]
US 10387464B2 · Weston et al. · 2019 [cited by applicant]
US 10453454B2 · Homma et al. · 2019 [cited by applicant]
US 10462422B1 · Harrison et al. · 2019 [cited by applicant]
US D868793S · Germe · 2019 [cited by applicant]
US 10511808B2 · Harrison et al. · 2019 [cited by applicant]
US 10521185B1 · Kim et al. · 2019 [cited by applicant]
US D881883S · Germe · 2020 [cited by applicant]
US D882567S · Parfitt · 2020 [cited by applicant]
US D882570S · Germe · 2020 [cited by applicant]
US 10623897B1 · Wu et al. · 2020 [cited by applicant]
US 10685669B1 · Lan et al. · 2020 [cited by applicant]
US 10719786B1 · Treseler et al. · 2020 [cited by applicant]
US 10782986B2 · Martin · 2020 [cited by applicant]
US 10803050B1 · Salkola · 2020 [cited by applicant]
US 10841249B2 · Lim et al. · 2020 [cited by applicant]
US 10854206B1 · Liu et al. · 2020 [cited by applicant]
US 10878805B2 · Agarwal et al. · 2020 [cited by applicant]
US 10885819B1 · Verbeke et al. · 2021 [cited by applicant]
US 10896295B1 · Shenoy · 2021 [cited by applicant]
US 10949616B1 · Shenoy et al. · 2021 [cited by applicant]
US 10958599B1 · Penov et al. · 2021 [cited by applicant]
US 10977258B1 · Liu et al. · 2021 [cited by applicant]
US 10978056B1 · Challa et al. · 2021 [cited by applicant]
US 11003669B1 · Nelson et al. · 2021 [cited by applicant]
US 11003839B1 · Hatch · 2021 [cited by applicant]
US 11038974B1 · Koukoumidis et al. · 2021 [cited by applicant]
US 11042554B1 · Balakrishnan et al. · 2021 [cited by applicant]
US 11086858B1 · Koukoumidis et al. · 2021 [cited by applicant]
US 11175646B2 · Koch · 2021 [cited by applicant]
US 11195057B2 · Zadeh et al. · 2021 [cited by applicant]
US 11222627B1 · Qian et al. · 2022 [cited by applicant]
US 11308284B2 · Huang et al. · 2022 [cited by applicant]
US 11416556B2 · Misra et al. · 2022 [cited by applicant]
US 11437027B1 · Guo et al. · 2022 [cited by applicant]
US 11829595B1 · Aggarwal · 2023 [cited by examiner]
US 11948563B1 · Liu et al. · 2024 [cited by applicant]
US 20080240379A1 · Maislos et al. · 2008 [cited by applicant]
US 20080300884A1 · Smith · 2008 [cited by applicant]
US 20090055778A1 · Abdelazim et al. · 2009 [cited by applicant]
US 20090282033A1 · Alshawi · 2009 [cited by applicant]
US 20110246383A1 · Gibson et al. · 2011 [cited by applicant]
US 20120245944A1 · Gruber et al. · 2012 [cited by applicant]
US 20120246191A1 · Xiong · 2012 [cited by applicant]
US 20120265528A1 · Gruber et al. · 2012 [cited by applicant]
US 20120311126A1 · Jadallah et al. · 2012 [cited by applicant]
US 20130035930A1 · Ferrucci et al. · 2013 [cited by applicant]
US 20130268839A1 · Lefebvre et al. · 2013 [cited by applicant]
US 20130275138A1 · Gruber et al. · 2013 [cited by applicant]
US 20130275164A1 · Gruber et al. · 2013 [cited by applicant]
US 20130278634A1 · Xu · 2013 [cited by examiner]
US 20130311508A1 · Denker · 2013 [cited by examiner]
US 20140012926A1 · Narayanan et al. · 2014 [cited by applicant]
US 20140032659A1 · Marini et al. · 2014 [cited by applicant]
US 20140074483A1 · Van Os · 2014 [cited by applicant]
US 20140129266A1 · Perl et al. · 2014 [cited by applicant]
US 20140164506A1 · Tesch et al. · 2014 [cited by applicant]
US 20140225918A1 · Mittal et al. · 2014 [cited by applicant]
US 20140244712A1 · Walters et al. · 2014 [cited by applicant]
US 20140278343A1 · Tran · 2014 [cited by applicant]
US 20140280017A1 · Indarapu et al. · 2014 [cited by applicant]
US 20140280302A1 · Morsi et al. · 2014 [cited by applicant]
US 20140297284A1 · Gruber et al. · 2014 [cited by applicant]
US 20140314338A1 · Hamel et al. · 2014 [cited by applicant]
US 20150081674A1 · Ali et al. · 2015 [cited by applicant]
US 20150134430A1 · Ellis et al. · 2015 [cited by applicant]
US 20150142420A1 · Sarikaya et al. · 2015 [cited by applicant]
US 20150142704A1 · London · 2015 [cited by applicant]
US 20150169284A1 · Quast et al. · 2015 [cited by applicant]
US 20150169572A1 · Taubman et al. · 2015 [cited by applicant]
US 20150169744A1 · Walkingshaw et al. · 2015 [cited by applicant]
US 20150179168A1 · Hakkani-Tur et al. · 2015 [cited by applicant]
US 20150186156A1 · Brown et al. · 2015 [cited by applicant]
US 20150207765A1 · Brantingham et al. · 2015 [cited by applicant]
US 20150220888A1 · Iyer · 2015 [cited by applicant]
US 20150235447A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150347375A1 · Tremblay et al. · 2015 [cited by applicant]
US 20160019290A1 · Ratnaparkhi et al. · 2016 [cited by applicant]
US 20160037311A1 · Cho · 2016 [cited by applicant]
US 20160063118A1 · Campbell et al. · 2016 [cited by applicant]
US 20160116980A1 · George-Svahn · 2016 [cited by examiner]
US 20160179462A1 · Bjorkengren · 2016 [cited by applicant]
US 20160196491A1 · Chandrasekaran et al. · 2016 [cited by applicant]
US 20160225370A1 · Kannan et al. · 2016 [cited by applicant]
US 20160255082A1 · Rathod · 2016 [cited by applicant]
US 20160260436A1 · Lemay et al. · 2016 [cited by applicant]
US 20160293164A1 · Shi et al. · 2016 [cited by applicant]
US 20160306505A1 · Vigneras et al. · 2016 [cited by applicant]
US 20160308799A1 · Schubert et al. · 2016 [cited by applicant]
US 20160328096A1 · Tran et al. · 2016 [cited by applicant]
US 20160378849A1 · Myslinski · 2016 [cited by applicant]
US 20160378861A1 · Eledath et al. · 2016 [cited by applicant]
US 20170026318A1 · Daniel et al. · 2017 [cited by applicant]
US 20170076720A1 · Gopalan et al. · 2017 [cited by applicant]
US 20170083285A1 · Meyers et al. · 2017 [cited by applicant]
US 20170091168A1 · Bellegarda et al. · 2017 [cited by applicant]
US 20170092264A1 · Hakkani-Tur et al. · 2017 [cited by applicant]
US 20170132019A1 · Karashchuk et al. · 2017 [cited by applicant]
US 20170193390A1 · Weston et al. · 2017 [cited by applicant]
US 20170220119A1 · Potts · 2017 [cited by examiner]
US 20170235361A1 · Rigazio et al. · 2017 [cited by applicant]
US 20170286401A1 · He et al. · 2017 [cited by applicant]
US 20170337742A1 · Powderly · 2017 [cited by examiner]
US 20170353469A1 · Selekman et al. · 2017 [cited by applicant]
US 20170358304A1 · Castillo Sanchez et al. · 2017 [cited by applicant]
US 20170359707A1 · Diaconu et al. · 2017 [cited by applicant]
US 20180013699A1 · Sapoznik et al. · 2018 [cited by applicant]
US 20180018562A1 · Jung · 2018 [cited by applicant]
US 20180018987A1 · Zass · 2018 [cited by applicant]
US 20180040020A1 · Kurian et al. · 2018 [cited by applicant]
US 20180045963A1 · Hoover · 2018 [cited by examiner]
US 20180046151A1 · Ahmed · 2018 [cited by applicant]
US 20180046861A1 · Patel · 2018 [cited by examiner]
US 20180052824A1 · Ferrydiansyah et al. · 2018 [cited by applicant]
US 20180054523A1 · Zhang et al. · 2018 [cited by applicant]
US 20180096071A1 · Green · 2018 [cited by applicant]
US 20180096072A1 · He et al. · 2018 [cited by applicant]
US 20180107917A1 · Hewavitharana et al. · 2018 [cited by applicant]
US 20180121508A1 · Halstvedt · 2018 [cited by applicant]
US 20180189629A1 · Yatziv et al. · 2018 [cited by applicant]
US 20180196880A1 · Carter · 2018 [cited by applicant]
US 20180210874A1 · Fuxman et al. · 2018 [cited by applicant]
US 20180211665A1 · Park et al. · 2018 [cited by applicant]
US 20180293484A1 · Wang et al. · 2018 [cited by applicant]
US 20180308486A1 · Saddler et al. · 2018 [cited by applicant]
US 20180329998A1 · Thomson et al. · 2018 [cited by applicant]
US 20180364887A1 · Bell et al. · 2018 [cited by applicant]
US 20190012470A1 · Ross · 2019 [cited by examiner]
US 20190034412A1 · Gupta et al. · 2019 [cited by applicant]
US 20190035152A1 · Kazansky · 2019 [cited by applicant]
US 20190035390A1 · Howard et al. · 2019 [cited by applicant]
US 20190066670A1 · White et al. · 2019 [cited by applicant]
US 20190080698A1 · Miller · 2019 [cited by applicant]
US 20190082221A1 · Jain et al. · 2019 [cited by applicant]
US 20190087491A1 · Bax · 2019 [cited by applicant]
US 20190096129A1 · Rhodes et al. · 2019 [cited by applicant]
US 20190096398A1 · Sereshki · 2019 [cited by applicant]
US 20190103108A1 · Song et al. · 2019 [cited by applicant]
US 20190132265A1 · Nowak-Przygodzki et al. · 2019 [cited by applicant]
US 20190139150A1 · Brownhill et al. · 2019 [cited by applicant]
US 20190171943A1 · Pao et al. · 2019 [cited by applicant]
US 20190179611A1 · Wojogbe et al. · 2019 [cited by applicant]
US 20190180144A1 · Tsishkou et al. · 2019 [cited by applicant]
US 20190213490A1 · White et al. · 2019 [cited by applicant]
US 20190311036A1 · Shanmugam et al. · 2019 [cited by applicant]
US 20190311720A1 · Pasko · 2019 [cited by applicant]
US 20190324527A1 · Presant et al. · 2019 [cited by applicant]
US 20190324553A1 · Liu et al. · 2019 [cited by applicant]
US 20190324780A1 · Zhu et al. · 2019 [cited by applicant]
US 20190325042A1 · Yu et al. · 2019 [cited by applicant]
US 20190325080A1 · Natarajan et al. · 2019 [cited by applicant]
US 20190325081A1 · Liu et al. · 2019 [cited by applicant]
US 20190325084A1 · Peng et al. · 2019 [cited by applicant]
US 20190325864A1 · Anders et al. · 2019 [cited by applicant]
US 20190327330A1 · Natarajan et al. · 2019 [cited by applicant]
US 20190327331A1 · Natarajan et al. · 2019 [cited by applicant]
US 20190347837A1 · Donalek et al. · 2019 [cited by applicant]
US 20190348033A1 · Chen et al. · 2019 [cited by applicant]
US 20190361408A1 · Tokuchi · 2019 [cited by applicant]
US 20200013380A1 · Kakoyiannis et al. · 2020 [cited by applicant]
US 20200027427A1 · Allen · 2020 [cited by examiner]
US 20200103963A1 · Kelly et al. · 2020 [cited by applicant]
US 20200143562A1 · Jiang · 2020 [cited by examiner]
US 20200211546A1 · Schairer et al. · 2020 [cited by applicant]
US 20200213663A1 · Okamoto · 2020 [cited by applicant]
US 20200221223A1 · Zhou · 2020 [cited by applicant]
US 20200228692A1 · Wakamatsu et al. · 2020 [cited by applicant]
US 20200241646A1 · Hebbalaguppe et al. · 2020 [cited by applicant]
US 20200274773A1 · Mortsolf et al. · 2020 [cited by applicant]
US 20200285855A1 · Brebner · 2020 [cited by applicant]
US 20200304880A1 · Diaz Delgado et al. · 2020 [cited by applicant]
US 20200314330A1 · Takayama · 2020 [cited by applicant]
US 20200341546A1 · Yuan et al. · 2020 [cited by applicant]
US 20200394843A1 · Ramachandra Iyer · 2020 [cited by applicant]
US 20200409457A1 · Terrano et al. · 2020 [cited by applicant]
US 20200410012A1 · Moon et al. · 2020 [cited by applicant]
US 20200410891A1 · Baeuerle et al. · 2020 [cited by applicant]
US 20210048883A1 · Kelly et al. · 2021 [cited by applicant]
US 20210081698A1 · Lindeman et al. · 2021 [cited by applicant]
US 20210082403A1 · Agarwal et al. · 2021 [cited by applicant]
US 20210090343A1 · Godi · 2021 [cited by examiner]
US 20210117479A1 · Liu et al. · 2021 [cited by applicant]
US 20210117623A1 · Aly et al. · 2021 [cited by applicant]
US 20210117712A1 · Huang et al. · 2021 [cited by applicant]
US 20210117780A1 · Malik et al. · 2021 [cited by applicant]
US 20210118440A1 · Peng et al. · 2021 [cited by applicant]
US 20210118442A1 · Poddar et al. · 2021 [cited by applicant]
US 20210120206A1 · Liu · 2021 [cited by examiner]
US 20210247846A1 · Shriram et al. · 2021 [cited by applicant]
US 20210334565A1 · Roche et al. · 2021 [cited by applicant]
US 20220121884A1 · Zadeh et al. · 2022 [cited by applicant]
US 20220129123A1 · Nair et al. · 2022 [cited by applicant]
US 20220214743A1 · Dascola · 2022 [cited by examiner]
US 20220219206A1 · Narayan · 2022 [cited by applicant]
US 20220269762A1 · Zhao et al. · 2022 [cited by applicant]
US 20240054156A1 · Vincent et al. · 2024 [cited by applicant]
CA 3142624A1 · 2020 [cited by applicant]
CN 107978313A · 2018 [cited by applicant]
CN 111344780A · 2020 [cited by applicant]
EP 3557504A1 · 2019 [cited by applicant]
EP 3792914A2 · 2021 [cited by applicant]
GB 2581621A · 2020 [cited by applicant]
TW 201712669A · 2017 [cited by applicant]
TW 201730734A · 2017 [cited by applicant]
TW 201738877A · 2017 [cited by applicant]
TW 201935270A · 2019 [cited by applicant]
TW 202001597A · 2020 [cited by applicant]
WO 9401824A1 · 1994 [cited by applicant]
WO 2012116241A2 · 2012 [cited by applicant]
WO 2017027339A1 · 2017 [cited by applicant]
WO 2019118367A1 · 2019 [cited by applicant]
US 11,915,021 B1, 02/2024, Liu et al. (withdrawn) [cited by applicant]
Co-pending U.S. Appl. No. 16/914,966, inventor Behar; Noam Yakob, filed Jun. 29, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/921,665, inventors Liu; Honglei et al., filed Jul. 6, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/998,423, inventors Armen; Aghajanyan et al., filed Aug. 20, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/006,260, inventors William; Presant et al., filed Aug. 28, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/006,339, inventors Shivani; Poddar et al., filed Aug. 28, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/006,377, inventors Shivani; Poddar et al., filed Aug. 28, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/009,542, inventor Kottur; Satwik , filed Sep. 1, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/035,253, inventors Khemka; Piyush et al., filed Sep. 28, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/120,013, inventors Botros; Fadi et al., filed Dec. 11, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/136,636, inventors Greenberg; Michael et al., filed Dec. 29, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/139,363, inventors Daniel; Manhon Cheng et al., filed Dec. 31, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 17/186,459, inventors Liu; Bing et al., filed Feb. 26, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/336,716, inventors Chaland; Christophe et al., filed Jun. 2, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/351,501, inventors Sethi; Pooja et al., filed Jun. 18, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/394,096, inventors Wang; Emily et al., filed Aug. 4, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/394,159, inventors Santoro; Elizabeth Kelsey et al., filed Aug. 4, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/407,922, inventors Pu; Yiming et al., filed Aug. 20, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/504,276, inventors Satwik; Kottur et al., filed Oct. 18, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/512,478, inventors Chen; Zhiyu et al., filed Oct. 27, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/512,490, inventors Presant; William Crosby et al., filed Oct. 27, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/521,623, inventors Martinson; Leif Haven et al., filed Nov. 8, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/524,598, inventors Hanson; Michael Robert et al., filed Nov. 11, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/525,510, inventors Shrivastava; Akshat et al., filed Nov. 12, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/543,178, inventors Desai; Shrey et al., filed Dec. 6, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/566,467, inventors Goel; Swati et al., filed Dec. 30, 2021. [cited by applicant]
Co-pending U.S. Appl. No. 17/725,540, inventors Surkov; Alexey Gennadyevich et al., filed Apr. 20, 2022. [cited by applicant]
Co-pending U.S. Appl. No. 17/732,104, inventors Sheng; Bi et al., filed Apr. 28, 2022. [cited by applicant]
Huang J., et al., “An Augmented Reality Sign-Reading Assistant for Users with Reduced Vision,” Plos One, vol. 14, No. 1,Jan. 16, 2019, 9 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/024163, mailed Oct. 10, 2022, 9 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/025653, mailed Jul. 21, 2022, 11 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/025675, mailed Jul. 22, 2022, 14 pages. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2022/025844, mailed Jul. 25, 2022, 14 pages. [cited by applicant]
Khan M.D., et al., “An AI-Based Visual Aid With Integrated Reading Assistant for the Completely Blind,” IEEE TRA1'ISACTIONS on Human-Machine Systems, vol. 50, No. 6, Dec. 2020, pp. 507-517. [cited by applicant]
Pintado D., et al., “Deep Learning Based Shopping Assistant for the Visually Impaired,” IEEE International Conference Onconsumer Electronics (ICCE), Jan. 11, 2019, 6 pages. [cited by applicant]
U.S. Appl. No. 62/660,876, inventors Kumar; Anuj et al., filed Apr. 20, 2018. [cited by applicant]
U.S. Appl. No. 62/675,090, inventors Hanson; Michael Robert et al., filed May 22, 2018. [cited by applicant]
U.S. Appl. No. 62/747,628, inventors Liu; Honglei et al., filed Oct. 18, 2018. [cited by applicant]
U.S. Appl. No. 62/749,608, inventors Challa; Ashwini et al., filed Oct. 23, 2018. [cited by applicant]
U.S. Appl. No. 62/750,746, inventors Liu; Honglei et al., filed Oct. 25, 2018. [cited by applicant]
U.S. Appl. No. 62/923,342, inventors Hanson; Michael Robert et al., filed Oct. 18, 2019. [cited by applicant]
USPTO—Office Action mailed on Dec. 16, 2024 for related U.S. Appl. No. 17/725,540; 72 pages. [cited by applicant]
Co-pending U.S. Appl. No. 15/953,957, inventors Moujahid; Kemal El et al., filed Apr. 16, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 15/967,193, inventors Testuggine; Davide et al., filed Apr. 30, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 15/967,279, inventors Fuchun; Peng et al., filed Apr. 30, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/025,317, inventors Gupta; Sonal et al., filed Jul. 2, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/036,827, inventors Emmanouil; Koukoumidis et al., filed Jul. 16, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/038,120, inventors Schissel; Jason et al., filed Jul. 17, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/048,049, inventor Markku; Salkola, filed Jul. 27, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/048,072, inventor Salkola; Markku , filed Jul. 27, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/048,101, inventor Salkola; Markku , filed Jul. 27, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/057,414, inventors Jeremy; Gillmor Kahn et al., filed Aug. 7, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/103,775, inventors Zheng; Zhou et al., filed Aug. 14, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/107,601, inventor Rajesh; Krishna Shenoy, filed Aug. 21, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/107,847, inventors Rajesh; Krishna Shenoy et al., filed Aug. 21, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/121,393, inventors Zhou; Zheng et al., filed Sep. 4, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/127,173, inventors Zheng; Zhou et al., filed Sep. 10, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/129,638, inventors Vivek; Natarajan et al., filed Sep. 12, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/150,184, inventors Francislav; P. Penov et al., filed Oct. 2, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/151,040, inventors Brian; Nelson et al., filed Oct. 3, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/168,536, inventors Dumoulin; Benoit F. et al., filed Oct. 23, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/176,081, inventors Anusha; Balakrishnan et al., filed Oct. 31, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/176,312, inventors Koukoumidis; Emmanouil et al., filed Oct. 31, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/182,542, inventors Michael; Hanson et al., filed Nov. 6, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/183,650, inventors Xiaohu; Liu et al., filed Nov. 7, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/192,538, inventor Koukoumidis; Emmanouil, filed Nov. 15, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/222,923, inventors Jason; Schissel et al., filed Dec. 17, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/222,957, inventors Emmanouil; Koukoumidis et al., filed Dec. 17, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/229,828, inventors Xiaohu; Liu et al., filed Dec. 21, 2018. [cited by applicant]
Co-pending U.S. Appl. No. 16/247,439, inventors Xiaohu; Liu et al., filed on Jan. 14, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/264,173, inventors Ashwini; Challa et al., filed Jan. 31, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/376,832, inventors Liu; Honglei et al., filed Apr. 5, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/389,634, inventors Crook; Paul Anthony et al., filed Apr. 19, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/389,728, inventors William; Presant et al., filed Apr. 19, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/389,738, inventors Peng; Fuchun et al., filed Apr. 19, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/389,769, inventors Honglei; Liu et al., filed Apr. 19, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/434,010, inventors Dogaru; Sergiu et al., filed Jun. 6, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/552,559, inventors Seungwhan; Moon et al., filed Aug. 27, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/557,055, inventors Moon; Seungwhan et al., filed Aug. 30, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/659,070, inventors Lisa; Xiaoyi Huang et al., filed Oct. 21, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/659,203, inventors Huang; Lisa Xiaoyi et al., filed Oct. 21, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/659,419, inventor Huang; Lisa Xiaoyi, filed Oct. 21, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/703,700, inventors Ahmed; Aly et al., filed Dec. 4, 2019. [cited by applicant]
Co-pending U.S. Appl. No. 16/733,044, inventors Francislav; P. Penov et al., filed Jan. 2, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/741,630, inventors Crook; Paul Anthony et al., filed Jan. 13, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/741,642, inventors Fuchun; Peng et al., filed Jan. 13, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/742,668, inventors Xiaohu; Liu et al., filed Jan. 14, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/742,769, inventors Liu; Xiaohu et al., filed Jan. 14, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/790,497, inventors Gao; Yang et al., filed Feb. 13, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/815,960, inventors Kshitiz; Malik et al., filed Mar. 11, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/815,990, inventors Kshitiz; Malik et al., filed Mar. 11, 2020. [cited by applicant]
Co-pending U.S. Appl. No. 16/842,366, inventors Sravani; Kamisetty et al., filed Apr. 7, 2020. [cited by applicant]
EPO—Office Action mailed Jan. 2, 2025 for European Patent Application No. 22729316.4, filed on Apr. 20, 2022, 9 pages. [cited by applicant]
EPO—Office Action mailed Jan. 3, 2025 for European Patent Application No. 22729836.1, filed on Apr. 21, 2022, 5 pages. [cited by applicant]
EPO—Office Action mailed Dec. 23, 2024 for European Patent Application No. 22726559.2, filed on Apr. 21, 2022, 9 pages. [cited by applicant]
Notice of Allowance mailed Apr. 2, 2025 for U.S. Appl. No. 17/725,540, filed Apr. 20, 2022, 14 pages. [cited by applicant]
Office Action mailed Oct. 6, 2025 for European Patent Application No. 22729836.1, filed on Apr. 21, 2022, 11 pages. [cited by applicant]
Office Action mailed May 16, 2025 for European Application No. 22729836.1, filed Apr. 21, 2022, 10 pages. [cited by applicant]
Office Action mailed May 20, 2025 for European Application No. 22729316.4, filed Apr. 20, 2022, 11 pages. [cited by applicant]
Office Action mailed Sep. 30, 2025 for Taiwan Application No. 111107738, filed Mar. 3, 2022, 8 pages. [cited by applicant]
Office Action mailed Jul. 31, 2025 for Taiwan Application No. 111107737, filed Mar. 3, 2022, 10 pages. [cited by applicant]
Non-Final Office Action mailed Apr. 20, 2026 for U.S. Appl. No. 18/894,872, filed Sep. 24, 2024, 15 pages. [cited by applicant]