IP Library Granted Patent US 12,449,656
Granted Patent B2
US 12,449,656 · App. 18/312,886 · Granted Oct 21, 2025

Methods and apparatuses for providing input for head-worn image display devices

Inventors: Lorena Pazmino (Wilton Manors, FL); Andrea Isabel Montoya (Plantation, FL); Savannah Niles (Fort Lauderdale, FL); Alexander Rocha (Boca Raton, FL); Mario Antonio Bragg (Lake Worth, FL); Parag Goel (Coral Springs, FL); Jeffrey Scott Sommers (Mountain View, CA); David Charles Lundmark (Los Altos, CA)
Assignee: Magic Leap, Inc.
G02B27/017G06F3/011G06F3/016G06F3/017G06F3/04812G06T19/006G02B2027/0138G02B2027/014G02B2027/0178G06F3/012G06F3/04886G06F2203/0381G06F2203/04808
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,449,656
App. No.
18/312,886
Granted
Oct 21, 2025
Kind
B2
Abstract

An apparatus for use with an image display device configured for head-worn by a user, includes: a screen; and a processing unit configured to assign a first area of the screen to sense finger-action of the user; wherein the processing unit is configured to generate an electronic signal to cause a change in a content displayed by the display device based on the finger-action of the user sensed by the assigned first area of the screen of the apparatus.

Claims (27)

1. An apparatus comprising:

a touch-sensitive display;

a feedback component configured to provide non-visual feedback; and

one or more processors that are operatively coupled to the touch-sensitive display and the feedback component and are communicatively coupled to a wearable device, the one or more processors configured to:

receive, from the wearable device, data indicating whether the apparatus is within a field of view of a camera of the wearable device or not; and

based on the data indicating whether the apparatus is within the field of view of the camera of the wearable device or not, switch between a first mode and a second mode, wherein in the first mode, the one or more processors are configured to convey one or more portions of a user interface to a user via the feedback component, wherein in the second mode, the one or more processors are configured to present the user interface on the touch-sensitive display of the apparatus without utilizing the feedback component of the apparatus, wherein the one or more processors are configured to switch to the second mode if the data indicates that the apparatus is within the field of view of the camera of the wearable device, and to switch to the first mode if the data indicates that the apparatus is outside the field of view of the camera of the wearable device.

2. The apparatus of claim 1 , wherein the feedback component is a haptic actuator.

3. The apparatus of claim 1 , wherein the feedback component is a speaker.

4. The apparatus of claim 1 , wherein user interface is a graphical user interface for controlling one or more functions of the wearable device.

5. The apparatus of claim 4 , wherein the graphical user interface comprises a plurality of graphical control elements.

6. The apparatus of claim 5 , wherein when in the mode, the one or more processors are configured to refrain from presenting one or more of the plurality of graphical control elements on the touch-sensitive display.

7. The apparatus of claim 5 , wherein when in the first mode, the one or more processors are configured to present a limited version of the graphical user interface on the touch-sensitive display.

8. The apparatus of claim 7 , wherein the limited version of the graphical user interface on the touch-sensitive display is one in which one or more of the plurality of graphical control elements are not shown, one in which brightness levels of one or more of the plurality of graphical control elements are reduced, or a combination thereof.

9. The apparatus of claim 5 , wherein when in the first mode, the one or more processors are configured to convey one or more locations on the touch-sensitive display at which one or more of the plurality of graphical control elements are otherwise presented in the second mode.

10. The apparatus of claim 9 , wherein the one or more locations on the touch-sensitive display that are conveyed in the first mode correspond to one or more outer boundaries of the plurality of graphical control elements that are otherwise presented in the second mode.

11. The apparatus of claim 5 , wherein the graphical control elements correspond to a plurality of physical user input components of a dedicated input controller that is associated with the wearable device.

12. The apparatus of claim 1 , wherein the one or more processors are configured to assign an area of the touch-sensitive display to sense different finger-gliding actions of the user.

13. The apparatus of claim 12 , wherein the apparatus is configured to monitor a peripheral part of the assigned area for provision of a control signal to operate the feedback component, wherein the peripheral part of the assigned area being monitored for provision of the control signal extends continuously across at least a majority of a side of the touch-sensitive display.

14. The apparatus of claim 1 , wherein the data comprises camera data.

15. The apparatus of claim 1 , wherein the camera is configured to view away from the user.

16. A processor-implemented method includes:

receiving, from a wearable device, data indicating whether an apparatus having a touch-sensitive display is within a field of view of a camera of the wearable device or not; and

based on the data, switching between a first mode and a second mode, wherein in the first mode, one or more portions of a user interface are conveyed to a user of the wearable device via a feedback component of the apparatus, wherein in the second mode, the user interface is presented on the touch-sensitive display of the apparatus without using the feedback component of the apparatus, wherein the second mode is switched to if the data indicates that the apparatus is within the field of view of the camera of the wearable device, and wherein the first mode is switched to if the data indicates that the apparatus is outside the field of view of the camera of the wearable device.

17. The method of claim 16 , wherein the data comprises camera data.

18. The method of claim 16 , further comprising assigning an area of the touch-sensitive display to sense different finger-gliding actions of the user.

19. The method of claim 18 , further comprising monitoring a peripheral part of the assigned area for provision of a control signal to operate the feedback component, wherein the peripheral part of the assigned area being monitored for provision of the control signal extends continuously across at least a majority of a side of the touch-sensitive display.

20. The method of claim 16 , wherein the camera is configured to view away from the user.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2025
From: PAZMINO, LORENA; MONTOYA, ANDREA ISABEL; NILES, SAVANNAH; BRAGG, MARIO ANTONIO; SOMMERS, JEFFREY SCOTT; LUNDMARK, DAVID CHARLES; GOEL, PARAG
To: MAGIC LEAP, INC.
Reel/Frame 072832/0581 →
PROPRIETARY INFORMATION AND INVENTIONS AGREEMENT Recorded Nov 7, 2025
From: ROCHA, ALEX
To: MAGIC LEAP, INC.
Reel/Frame 073577/0107 →
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0238 →
PROPRIETARY INFORMATION AND INVENTIONS AGREEMENT Recorded Sep 18, 2025
From: ROCHA, ALEX
To: MAGIC LEAP, INC.
Reel/Frame 072941/0258 →
Continuity (3)
Continuation 16448452 · Jun 21, 2019
Provisional Application 62688108 · Jun 21, 2018
Related Publication 20230273431A1 · Aug 31, 2023
References Cited (62)
US 8046719B2 · Skourup et al. · 2011 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 20130127980A1 · Haddick et al. · 2013 [cited by applicant]
US 20130325438A1 · Griffin et al. · 2013 [cited by applicant]
US 20140240245A1 · Kim · 2014 [cited by examiner]
US 20140285352A1 · Cho et al. · 2014 [cited by applicant]
US 20150363980A1 · Dorta et al. · 2015 [cited by applicant]
US 20150379963A1 · Holmanu · 2015 [cited by examiner]
US 20160085403A1 · Koga · 2016 [cited by applicant]
US 20160093108A1 · Mao et al. · 2016 [cited by applicant]
US 20170031502A1 · Rosenberg et al. · 2017 [cited by applicant]
US 20170054791A1 · Hadfield et al. · 2017 [cited by applicant]
US 20170061700A1 · Urbach et al. · 2017 [cited by applicant]
US 20170076502A1 · Chen · 2017 [cited by examiner]
US 20170256144A1 · Khoshkava et al. · 2017 [cited by applicant]
US 20170293351A1 · Li · 2017 [cited by applicant]
US 20170322622A1 · Hong et al. · 2017 [cited by applicant]
US 20170322623A1 · McKenzie et al. · 2017 [cited by applicant]
US 20170329419A1 · Dearman et al. · 2017 [cited by applicant]
US 20170345218A1 · Bedikian et al. · 2017 [cited by applicant]
US 20180314406A1 · Powderly et al. · 2018 [cited by applicant]
US 20180315248A1 · Bastov et al. · 2018 [cited by applicant]
US 20190034076A1 · Vinayak et al. · 2019 [cited by applicant]
US 20190064997A1 · Wang et al. · 2019 [cited by applicant]
US 20190065026A1 · Kiemele et al. · 2019 [cited by applicant]
US 20190197785A1 · Tate-Gans et al. · 2019 [cited by applicant]
JP 2016115039 · 2016 [cited by applicant]
JP 2016224595 · 2016 [cited by applicant]
JP 2017045472 · 2017 [cited by applicant]
JP 2017062559 · 2017 [cited by applicant]
WO WO2017048519 · 2017 [cited by applicant]
WO WO2017222997 · 2017 [cited by applicant]
WO WO2018106299 · 2018 [cited by applicant]
Foreign NOA for JP Patent Appln. No. 2020-570463 dated Oct. 11, 2023. [cited by applicant]
Foreign Response for JP Patent Appln. No. 2020-570463 dated Jul. 13, 2023. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/448,452 dated Oct. 29, 2020. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/448,452 dated Nov. 29, 2021. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/448,452 dated Aug. 9, 2022. [cited by applicant]
Final Office Action for U.S. Appl. No. 16/448,452 dated Apr. 30, 2021. [cited by applicant]
Final Office Action for U.S. Appl. No. 16/448,452 dated Apr. 20, 2022. [cited by applicant]
Final Office Action for U.S. Appl. No. 16/448,452 dated Feb. 17, 2023. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 16/448,452 dated Nov. 9, 2022. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 16/448,452 dated Feb. 28, 2022. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 16/448,452 dated Jan. 28, 2021. [cited by applicant]
Amendment Response to FOA for U.S. Appl. No. 16/448,452 dated Jul. 20, 2022. [cited by applicant]
Amendment Response to FOA for U.S. Appl. No. 16/448,452 dated Jun. 20, 2022. [cited by applicant]
Amendment Response to FOA for U.S. Appl. No. 16/448,452 dated Jul. 29, 2021. [cited by applicant]
PCT International Search Report and Written Opinion for International Appln. No. PCT/US19/38462, Applicant Magic Leap, Inc., dated Sep. 30, 2019 (9 pages). [cited by applicant]
Extended European Search Report for EP Patent Appln. No. 19823120.1 dated Jul. 7, 2021. [cited by applicant]
PCT International Preliminary Report on Patentability for International Appln. No. PCT/US2019/038462 dated Dec. 22, 2020. [cited by applicant]
Foreign Response for EP Patent Appln. No. 19823120.1 dated Feb. 3, 2022. [cited by applicant]
Foreign Response for EP Patent Appln. No. 19823120.1 dated Mar. 20, 2023. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2020-570463 dated Apr. 19, 2023 (with English translation). [cited by applicant]
Foreign OA for CN Patent Appln. 201980054294.7 dated Dec. 25, 2023 (with English translation). [cited by applicant]
Extended European Search Report for EP Patent Appln. No. 23218626.2 dated Mar. 12, 2024. [cited by applicant]
Foreign Response for CN Patent Appln. No. 201980054294.7 dated Apr. 22, 2024. [cited by applicant]
Foreign NOA for CN Patent Appln. No. 201980054294.7 dated Sep. 24, 2024. [cited by applicant]
Foreign Response for EP Patent Appln. No. 23218626.2 dated Oct. 10, 2024. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2023-189938 dated Oct. 22, 2024 (with English translation). [cited by applicant]
Foreign Response for JP Patent Appln. No. 2023-189938 dated Jan. 21, 2025. [cited by applicant]
Foreign NOA for JP Patent Appln. No. 2023-189938 dated Mar. 26, 2025. [cited by applicant]
Foreign NOA for EP Patent Appln. No. 23218626.2 dated Mar. 21, 2025. [cited by applicant]