Granted Patent B2

US 12,645,292 · App. 18/662,964 · Granted Jun 2, 2026

Inventors: Michael J. Rockwell (Palo Alto, CA); Oriel Y. Bergig (Shoham, CA); Geoffrey Stahl (San Jose, CA); Thibaut Weise (Munich, DE); Peter Kaufmann (Zurich, CH); Branko Petljanski (Neubiberg, DE); Jason L. Slupeiks (Sunnyvale, CA); Tom Sengelaub (Oakland, CA); Yanghai Tsin (Palo Alto, CA); Hesam Najafi (Mountain View, CA); Arthur Y. Zhang (San Jose, CA); Cheng Chen (San Jose, CA); Graham B. Myhre (Campbell, CA); Fletcher R. Rothkopf (Los Altos, CA); Marinus Meursing (Poway, CA); Phil M. Hobson (Menlo Park, CA); Jason C. Sauers (Sunnyvale, CA); Jan K. Quijalvo (San Jose, CA); Jia Tao (San Jose, CA); Ivan S. Maric (Campbell, CA); Jeremy C. Franklin (San Francisco, CA); Wey-Jiun Lin (Los Altos Hills, CA)

Assignee: APPLE INC.

G06F3/013G06F3/017

Abstract

Claims (54)

1 . A head-mountable electronic device, comprising:

a housing;

a display secured within the housing;

a first optical sensor disposed adjacent the display, the first optical sensor oriented inward to face a user donning the head-mountable electronic device, the first optical sensor comprising a camera and a light-emitting diode (LED);

a second optical sensor oriented outward and away from the face of the user;

a processor communicatively coupled to the first optical sensor and the second optical sensor; and

a memory device storing instructions that, when executed by the processor, cause the processor to:

identify user gaze data from the first optical sensor;

identify user hand gesture data from the second optical sensor; and

modify a user interface presented by the display based on at least one of the user gaze data or the hand gesture data, the user interface including a virtual object.

2 . The head-mountable electronic device of claim 1 , wherein:

the camera is a first camera; and

the first optical sensor further comprises a second camera disposed adjacent to the display.

3 . The head-mountable electronic device of claim 2 , wherein the first optical sensor further comprises an LED array including the LED, the LED array disposed around a perimeter of the display.

4 . The head-mountable electronic device of claim 1 , wherein the gaze data includes eye-tracking data from the first optical sensor.

5 . The head-mountable electronic device of claim 4 , wherein the first optical sensor is configured to gather the eye-tracking data by receiving, via the camera, light from the LED reflected off an eye of the user.

6 . The head-mountable electronic device of claim 1 , wherein modifying the user interface includes changing a level of virtual reality immersion presented by the display.

7 . The head-mountable electronic device of claim 6 , wherein:

the level of virtual reality immersion includes an amount of virtual content superimposed over real world content; and

changing the level of virtual reality immersion includes changing the amount of virtual content superimposed over the real world content.

8 . The head-mountable electronic device of claim 7 , wherein the real world content includes a visual representation of an environment external to the user and the head-mountable electronic device captured by the second optical sensor.

9 . The head-mountable electronic device of claim 1 , wherein modifying the user interface includes moving the virtual object.

10 . A wearable electronic device, comprising:

a display frame;

a display secured to the display frame and oriented in a rearward direction toward a user donning the wearable electronic device;

an LED disposed adjacent the display and oriented in the rearward direction to project light to an eye of the user;

a first camera positioned adjacent the display and oriented in the rearward direction to detect, via the light projected by the LED and reflected off the eye, a gaze of the user; and

a second camera secured to the display frame and oriented in an outward direction to detect a hand gesture of the user, the outward direction is away from the display frame and different than the rearward direction;

wherein:

the display is configured to present virtual content superimposed over real world content; and

an amount of the virtual content presented is based on at least one of the gaze or the hand gesture.

11 . The wearable electronic device of claim 10 , further comprising a user input mechanism that includes at least one of a lever, a button, a dial, a rocker, a slider, or a toggle.

12 . The wearable electronic device of claim 11 , wherein the amount of virtual content presented is based on a manipulation of the user input mechanism.

13 . The wearable electronic device of claim 10 , further comprising a plurality of LEDs including the LED, the plurality of LEDs disposed around a perimeter of the display.

14 . The wearable electronic device of claim 10 , wherein the outward direction is opposite the rearward direction.

15 . The wearable electronic device of claim 10 , wherein the outward direction includes a downward direction relative to the face of the user.

16 . A head-mountable electronic device, comprising:

a housing;

a camera configured to capture content external to the housing;

an optical module secured to the housing, the optical module comprising:

a display;

a LED disposed adjacent the display; and

an inward facing sensor configured to detect a gaze direction of a user donning the head-mountable electronic device based on light generated by the LED and reflected off an eye of the user; and

a processor electrically coupled to the camera, the inward facing sensor, the LED, and the display;

wherein:

the processor is configured to cause the display to project mixed reality content including a first amount of video passthrough content captured by the camera and a second amount of virtual reality content including a virtual object; and

the camera is configured to capture a hand gesture; and

the virtual object is configured to be manipulated based on the hand gesture.

17 . The head-mountable electronic device of claim 16 , wherein the camera is oriented downward toward the hand when the user is donning the head-mountable electronic device.

18 . The head-mountable electronic device of claim 17 , wherein:

the camera is a first camera; and

the head-mountable electronic device further comprises a second camera configured to capture the video passthrough content external to the housing.

19 . The head-mountable electronic device of claim 16 , wherein the hand gesture includes a movement of the hand.

20 . The head-mountable electronic device of claim 16 , wherein the inward facing sensor comprises a visual camera.

Assignments (1)

Continuity (4)

References Cited (184)