IP Library Granted Patent US 12,333,122
Granted Patent B2
US 12,333,122 · App. 18/591,983 · Granted Jun 17, 2025

Light-emitting user input device for calibration or pairing

Inventors: James M. Powderly (Ft. Lauderdale, FL); Savannah Niles (Ft. Lauderdale, FL); Christopher David Nesladek (San Francisco, CA); Isioma Osagbemwenorue Azu (Ft. Lauderdale, FL); Marshal Ainsworth Fontaine (St. Augustine, FL); Haney Awad (Miami, FL); William Wheeler (Ft. Lauderdale, FL); Brian David Schwab (Sunrise, FL); Brian Edward Oliver Bucknor (Miramar, FL)
Assignee: Magic Leap, Inc.
G06F3/04815G02B27/0172G06F3/011G06F3/016G06F3/017G06F3/02G06F3/0308G06F3/0346G06F3/03547G06F3/038G06F3/041G06F3/04847G06F3/04883G06F3/04886G06T19/006G06V10/145G06V20/20G06V40/18G09G3/32G02B2027/0178G06F2203/0384G06F2203/04103G06F2203/04105G09G2320/0626G09G2354/00H04W4/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,333,122
App. No.
18/591,983
Granted
Jun 17, 2025
Kind
B2
Abstract

A light-emitting user input device can include a touch sensitive portion configured to accept user input (e.g., from a user's thumb) and a light-emitting portion configured to output a light pattern. The light pattern can be used to assist the user in interacting with the user input device. Examples include emulating a multi-degree-of-freedom controller, indicating scrolling or swiping actions, indicating presence of objects nearby the device, indicating receipt of notifications, assisting pairing the user input device with another device, or assisting calibrating the user input device. The light-emitting user input device can be used to provide user input to a wearable device, such as, e.g., a head mounted display device.

Claims (38)

1. A light-emitting user input device for providing user input to a display system including a wearable display device, the light-emitting user input device comprising:

a plurality of interactable regions, including a central touch surface, configured to receive user inputs;

a light-emitting assembly comprising a light guide outside of and at least partly surrounding the central touch surface; and

a hardware processor communicatively coupled to the plurality of interactable regions and the light-emitting assembly, the hardware processor programmed to:

detect an actuation of the light-emitting user input device by a user of the wearable display device;

determine an interactable region of the plurality of interactable regions that corresponds to the actuation;

translate the actuation into a user input for performing a user interface operation for interacting with a virtual environment presented through the wearable display device, wherein the virtual environment includes virtual content generated by an application executing on the display system, and wherein the user interface operation is determined based at least on a type of the actuation and the interactable region corresponding with the actuation; and

instruct the light-emitting assembly to illuminate a light pattern in response to the user input, wherein the light pattern is based at least partly on the particular application that is currently executing on the display system such that the hardware processor is programmed to instruct the light-emitting assembly to illuminate a first light pattern while a first application is executing and the hardware processor is programmed to instruct the light-emitting assembly to illuminate a second light pattern while a second application is executing, wherein the second light pattern is different than the first light pattern.

2. The light-emitting user input device of claim 1 , wherein:

the plurality of interactable regions comprises a first interactable region and a second interactable region;

in response to determining that the user actuated the first interactable region, the hardware processor is programmed to cause the light-emitting assembly to illuminate a third light pattern associated with the user input in the first interactable region; and

wherein in response to determining that the user actuated the second interactable region, the hardware processor is programmed to cause the light-emitting assembly to illuminate a fourth light pattern associated with the user input in the second interactable region.

3. The light-emitting user input device of claim 1 , wherein the actuation comprises at least one of: a swipe, a tap, a press, or a touch gesture.

4. The light-emitting user input device of claim 1 , wherein the light pattern comprises an arcuate region having one or more of a color, an arcuate length, or a visual effect.

5. The light-emitting user input device of claim 1 , wherein the hardware processor is further programmed to communicate with the display system causing the display system to perform the user interface operation based on the user input.

6. The light-emitting user input device of claim 1 , wherein the light guide is substantially circular.

7. The light-emitting user input device of claim 1 , wherein the light guide is touch sensitive to receive user input.

8. The light-emitting user input device of claim 1 , wherein the central touch surface is substantially circular.

9. The light-emitting user input device of claim 1 , wherein the light guide fully surrounds the central touch surface.

10. A method comprising:

under control of a hardware processor:

detecting an actuation of a light-emitting user input device by a user of a wearable display device, wherein the light-emitting user input device is configured to provide user input to a display system including the wearable display device, and wherein the light-emitting user input device comprises a plurality of interactable regions, including a central touch surface, configured to receive inputs, and a light-emitting assembly comprising a light guide outside of and at least partly surrounding the central touch surface;

determining an interactable region of the plurality of interactable regions that corresponds to the actuation;

translating the actuation into a user input for performing a user interface operation for interacting with a virtual environment presented through the wearable display device, wherein the virtual environment includes virtual content generated by an application executing on the display system, and wherein the user interface operation is determined based at least on a type of the actuation and the interactable region corresponding with the actuation; and

instructing the light-emitting assembly to illuminate a light pattern in response to the user input, wherein the light pattern is based at least partly on the particular application that is currently executing on the display system, such that the light-emitting assembly is instructed to illuminate a first light pattern while a first application is executing and the light-emitting assembly is instructed to illuminate a second light pattern while a second application is executing, wherein the second light pattern is different than the first light pattern.

11. The method of claim 10 , wherein the plurality of interactable regions comprises a first interactable region and a second interactable region, the method further comprising:

in response to determining that the user actuated the first interactable region, causing the light-emitting assembly to illuminate a third light pattern associated with the user input in the first interactable region; and

in response to determining that the user actuated the second interactable region, causing the light-emitting assembly to illuminate a fourth light pattern associated with the user input in the second interactable region.

12. The method of claim 10 , wherein the actuation comprises at least one of: a swipe, a tap, a press, or a touch gesture.

13. The method of claim 10 , wherein the light pattern comprises an arcuate region having one or more of a color, an arcuate length, or a visual effect.

14. The method of claim 10 , further comprising:

causing the display system to perform the user interface operation based on the user input.

15. The method of claim 10 , wherein the light guide is substantially circular.

16. The method of claim 10 , wherein the light guide is touch sensitive to receive user input.

17. The method of claim 10 , wherein the central touch surface is substantially circular.

18. The method of claim 10 , wherein the light guide fully surrounds the central touch surface.

19. The light-emitting user input device of claim 1 , wherein the wearable display device is separate from the light-emitting user input device.

20. The method of claim 10 , wherein the wearable display device is separate from the light-emitting user input device.

Assignments (4)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073439/0168 →
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2024
From: POWDERLY, JAMES M.; NILES, SAVANNAH; NESLADEK, CHRISTOPHER DAVID; AZU, ISIOMA OSAGBEMWENORUE; FONTAINE, MARSHAL A.; AWAD, HANEY; WHEELER, WILLIAM; SCHWAB, BRIAN DAVID; BUCKNOR, BRIAN
To: MAGIC LEAP, INC.
Reel/Frame 066992/0649 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2024
From: POWDERLY, JAMES M.; NILES, SAVANNAH; NESLADEK, CHRISTOPHER DAVID; AZU, ISIOMA OSAGBEMWENORUE; FONTAINE, MARSHAL AINSWORTH; AWAD, HANEY; SCHWAB, BRIAN DAVID; BUCKNOR, BRIAN EDWARD OLIVER
To: MAGIC LEAP, INC.
Reel/Frame 067000/0913 →
Continuity (6)
Continuation 18148316 · Dec 29, 2022
Continuation 17384663 · Jul 23, 2021
Continuation 15965696 · Apr 27, 2018
Continuation 15965702 · Apr 27, 2018
Provisional Application 62490863 · Apr 27, 2017
Related Publication 20240201823A1 · Jun 20, 2024
References Cited (172)
US 5704151A · West et al. · 1998 [cited by applicant]
US D432131S · Jones et al. · 2000 [cited by applicant]
US 6850221B1 · Tickle · 2005 [cited by applicant]
US D514570S · Ohta · 2006 [cited by applicant]
US D563480S · Blaseflug et al. · 2008 [cited by applicant]
US D565668S · Baseflug et al. · 2008 [cited by applicant]
US D567243S · Ashida et al. · 2008 [cited by applicant]
US D567287S · Anand · 2008 [cited by applicant]
US D587713S · Sutton · 2009 [cited by applicant]
US D589515S · Brunner et al. · 2009 [cited by applicant]
US D594066S · Oikawa et al. · 2009 [cited by applicant]
US D611477S · Brunner et al. · 2010 [cited by applicant]
US D638841S · Musick et al. · 2011 [cited by applicant]
US D658184S · Chang et al. · 2012 [cited by applicant]
US 8179604B1 · Gomez et al. · 2012 [cited by applicant]
US D665461S · Ikeda et al. · 2012 [cited by applicant]
US D675644S · Frost et al. · 2013 [cited by applicant]
US 8523675B2 · Young et al. · 2013 [cited by applicant]
US D698358S · Beams · 2014 [cited by applicant]
US 8950867B2 · Macnamara · 2015 [cited by applicant]
US D725041S · Bailey et al. · 2015 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US D740369S · Cho et al. · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US D749044S · Huang · 2016 [cited by applicant]
US D752529S · Loretan et al. · 2016 [cited by applicant]
US D753095S · Jou et al. · 2016 [cited by applicant]
US 9310559B2 · Macnamara · 2016 [cited by applicant]
US D755761S · Reynolds et al. · 2016 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US D758367S · Natsume · 2016 [cited by applicant]
US D759657S · Kujawski et al. · 2016 [cited by applicant]
US D763359S · Kwong · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US D769876S · Aoyagi et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US D797743S · Awad et al. · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski et al. · 2018 [cited by applicant]
US D828337S · Li · 2018 [cited by applicant]
US 10152141B1 · Lohse et al. · 2018 [cited by applicant]
US D844608S · Chen et al. · 2019 [cited by applicant]
US 10444849B2 · Li · 2019 [cited by applicant]
US 10452133B2 · Miller et al. · 2019 [cited by applicant]
US 10534447B2 · Li · 2020 [cited by applicant]
US D884700S · Shao et al. · 2020 [cited by applicant]
US 10705619B2 · Johri · 2020 [cited by applicant]
US D918176S · Natsume et al. · 2021 [cited by applicant]
US D924204S · Natsume et al. · 2021 [cited by applicant]
US D930614S · Natsume et al. · 2021 [cited by applicant]
US 11112932B2 · Powderly et al. · 2021 [cited by applicant]
US 11163416B2 · Powderly et al. · 2021 [cited by applicant]
US 20020024675A1 · Foxlin · 2002 [cited by applicant]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20060103633A1 · Gioeli · 2006 [cited by applicant]
US 20060181521A1 · Perreault et al. · 2006 [cited by applicant]
US 20060279554A1 · Shin et al. · 2006 [cited by applicant]
US 20060284792A1 · Foxlin · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20080036744A1 · Hartl · 2008 [cited by applicant]
US 20090002218A1 · Rigazio et al. · 2009 [cited by applicant]
US 20090090568A1 · Min · 2009 [cited by examiner]
US 20090267898A1 · Yamamoto et al. · 2009 [cited by applicant]
US 20090284950A1 · Rubio · 2009 [cited by examiner]
US 20090295722A1 · Yamamoto · 2009 [cited by applicant]
US 20090299686A1 · Ho · 2009 [cited by applicant]
US 20100102980A1 · Troy et al. · 2010 [cited by applicant]
US 20100194687A1 · Corson et al. · 2010 [cited by applicant]
US 20110121950A1 · Izadi et al. · 2011 [cited by applicant]
US 20110159813A1 · Mallinson et al. · 2011 [cited by applicant]
US 20110159814A1 · Mallinson et al. · 2011 [cited by applicant]
US 20110159959A1 · Mallinson et al. · 2011 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120133583A1 · Ramrattan · 2012 [cited by examiner]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130104085A1 · Mlyniec et al. · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140176418A1 · Ramachandran et al. · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140320531A1 · Elvesjö · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150138101A1 · Park et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150193949A1 · Katz et al. · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150220110A1 · Brawer · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150258432A1 · Stafford et al. · 2015 [cited by applicant]
US 20150261291A1 · Mikhailov et al. · 2015 [cited by applicant]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150317833A1 · Ebstyne et al. · 2015 [cited by applicant]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · TeKolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160062489A1 · Li · 2016 [cited by applicant]
US 20160100817A1 · Hussain · 2016 [cited by applicant]
US 20160239096A1 · Okuno et al. · 2016 [cited by applicant]
US 20160259404A1 · Woods · 2016 [cited by applicant]
US 20160364910A1 · Higgins et al. · 2016 [cited by applicant]
US 20160378204A1 · Chen et al. · 2016 [cited by applicant]
US 20170076502A1 · Chen et al. · 2017 [cited by applicant]
US 20170109936A1 · Powderly et al. · 2017 [cited by applicant]
US 20170182957A1 · Watson et al. · 2017 [cited by applicant]
US 20170205903A1 · Miller et al. · 2017 [cited by applicant]
US 20170290290A1 · Trottier et al. · 2017 [cited by applicant]
US 20170307889A1 · Newman · 2017 [cited by applicant]
US 20170307891A1 · Bucknor et al. · 2017 [cited by applicant]
US 20170308258A1 · Xu et al. · 2017 [cited by applicant]
US 20170336964A1 · Kim et al. · 2017 [cited by applicant]
US 20170352184A1 · Poulos et al. · 2017 [cited by applicant]
US 20170357333A1 · Balan et al. · 2017 [cited by applicant]
US 20180005443A1 · Poulos et al. · 2018 [cited by applicant]
US 20180053284A1 · Rodriguez et al. · 2018 [cited by applicant]
US 20180091304A1 · Brook et al. · 2018 [cited by applicant]
US 20180097975A1 · Osman · 2018 [cited by applicant]
US 20180120936A1 · Keller et al. · 2018 [cited by applicant]
US 20180173417A1 · Foresti · 2018 [cited by applicant]
US 20180217663A1 · Chandrasekhar et al. · 2018 [cited by applicant]
US 20180314406A1 · Powderly et al. · 2018 [cited by applicant]
US 20180314416A1 · Powderly et al. · 2018 [cited by applicant]
US 20180330521A1 · Samples et al. · 2018 [cited by applicant]
US 20190018567A1 · Murphy et al. · 2019 [cited by applicant]
US 20190170510A1 · Robinson · 2019 [cited by applicant]
US 20190187779A1 · Miller · 2019 [cited by applicant]
US 20190325274A1 · Balan et al. · 2019 [cited by applicant]
US 20220019333A1 · Powderly et al. · 2022 [cited by applicant]
US 20230221831A1 · Powderly et al. · 2023 [cited by applicant]
CN 105117016A · 2015 [cited by applicant]
EP 2516026A1 · 2012 [cited by applicant]
JP 2005092435B · 2005 [cited by applicant]
JP 2011024613A · 2011 [cited by applicant]
JP 2012094010A · 2012 [cited by applicant]
JP 2015072582A · 2015 [cited by applicant]
JP 2016162368A · 2016 [cited by applicant]
WO 2011078926A1 · 2011 [cited by applicant]
WO 2018201067A1 · 2018 [cited by applicant]
https://web.archive.org/web/20111025233711/http://onemansstyle.wordpress.com/2010/06/06/one-guys-daily-tip-blackberrys-neither-seen-nor-heard/ (Year: 2010). [cited by examiner]
“Force Touch”, Wikipedia, retrieved Jan. 29, 2017, in 3 pages. URL: https://en.wikipedia.org/wiki/Force_Touch. [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, (Aug. 4, 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu/ azuma/ARpresence.pdf. [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality-Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/raskar/book/BimberRaskarAugmentedRealityBook.pdf. [cited by applicant]
CA3139648 Office Action dated Jan. 10, 2023. [cited by applicant]
CN2018800425864 Office Action dated Mar. 14, 2023. [cited by applicant]
EP22197905.7 Extended European Search Report dated Jan. 30, 2023. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US18/29988, dated Oct. 29, 2019. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US18/29988, dated Aug. 30, 2018. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
Nilsson, L.G, “Another touch screen technology tips up”, SemiAccurate, Dec. 30, 2009, retrieved Jan. 29, 2017, in 4 pages. URL: http://semiaccurate.com/2009/12/30/newtouchscreentechnologytips/. [cited by applicant]
Sandisk Clip Sport Plus MP3 Player at least at https://www.youtube.com/watch?v=HPrnBTFIxhY (Year: 2016). [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA—paper/Proc. AMC CHI 2000 Human Factors in Computin… [cited by applicant]
EP24177515.4 Extended European Search Report dated Oct. 15, 2024. [cited by applicant]
JP2023-180164 Office Action mailed Jul. 10, 2024. [cited by applicant]