Augmented reality system and method
An augmented reality (AR) system for use by a user in a surrounding environment includes a head-mounted display (HMD), a handheld computing device (HCD) and a control module operatively connected with one or both of the HMD and the HCD. The system is configured to enable digital content to be displayed simultaneously on the HMD and on the HCD, and/or to use one of the HMD and the HCD to modify digital content or to execute an action on the other of the HMD and the HCD, with the HCD being positionable by the user outside of the HMD's lens field of view, thereby providing an enhanced or extended user field of view.
1 . An augmented reality (AR) system for use by a user in a surrounding environment, comprising:
a head-mounted display (HMD) having a lens that is generally transparent or permits video pass-through functionality and through which the surrounding environment is visible to the user, a predetermined portion of the lens for displaying digital content thereon or therethrough for viewing by the user, and a first group of sensors configured for sensing a met condition;
a handheld computing device (HCD) having an HCD display and a second group of sensors configured for sensing the met condition; and
one or more processors operatively connected with one or more of the HMD or the HCD, wherein the one or more processors are configured to:
sense the met condition;
cause digital content to be displayed simultaneously on the predetermined portion of the lens and on the HCD display based on the sensed met condition;
when the met condition is sensed by the first group of sensors, modify the digital content displayed on the HCD display; and
when the met condition is sensed by the second group of sensors, modify the digital content displayed on the predetermined portion of the lens.
2 . The AR system of claim 1 , wherein when wearing the HMD, the user has a natural field of view, the lens has a lens field of view as viewed by the user that is within and smaller than the natural field of view, and virtual images are projectable onto a virtual image field of view as viewed by the user that is within and smaller than the lens field of view, wherein the HCD may be positioned by the user at least partially outside of the virtual image field of view.
3 . The AR system of claim 1 , wherein, to sense the met condition, the one or more processors are configured to sense a user action, and wherein the user action includes one or more of:
a roll of the HMD,
a pitch of the HMD,
a yaw of the HMD,
a translation of the HMD,
an eye movement,
a hand gesture,
a tool gesture,
a voice command spoken to the HMD,
a push of a button on the HCD,
a gesture on a touchscreen/touchpad of the HCD,
a shaking of the HCD, or
a voice command spoken to the HCD.
4 . The AR system of claim 1 , wherein the one or more processors are further configured to:
permit a first modification of digital content on one of the predetermined portion of the lens or the HCD display, and
simultaneously cause a second modification of digital content on the other of the predetermined portion of the lens or the HCD display, wherein the second modification is substantially the same as the first modification.
5 . The AR system of claim 1 , wherein the one or more processors are further configured to permit a first modification of digital content on one of the predetermined portion of the lens or the HCD display without modifying digital content simultaneously on the other of the predetermined portion of the lens or the HCD display.
6 . The AR system of claim 1 , wherein the AR system is configured to enable the user to add additional digital content to the HCD display with the additional digital content also being displayed simultaneously on the predetermined portion of the lens.
7 . The AR system of claim 1 , wherein the AR system is configured to enable the user to add additional digital content to the HCD display without the additional digital content being displayed simultaneously on the predetermined portion of the lens.
8 . The AR system of claim 1 , wherein the AR system is configured to enable the user to add additional digital content to the HCD display and to utilize one or more applications on the HCD for processing the additional digital content.
9 . The AR system of claim 1 , further comprising:
one or more of;
a local ad hoc connection between the HMD and the HCD for facilitating communication therebetween; or
a server operatively connected with the HMD and the HCD for facilitating communication with or between the HMD and the HCD.
10 . The AR system of claim 1 , wherein the one or more processors comprise:
a first processor embedded as a portion of the HMD,
a second processor embedded as a portion of the HCD, or
both the first processor and the second processor.
11 . A method of modifying digital content within an AR system that includes a head-mounted display (HMD) having a first group of sensors configured for sensing a met condition, a handheld computing device (HCD) having a second group of sensors configured for sensing the met condition, and one or more processors operatively connected with one or more of the HMD or the HCD, the method comprising:
sensing the met condition;
causing digital content to be displayed simultaneously on the HMD and on the HCD based on the sensed met condition;
when the met condition is sensed by the first group of sensors, modifying the digital content displayed on the HCD; and
when the met condition is sensed by the second group of sensors, modifying the digital content displayed on the HMD.
12 . The method of claim 11 , further comprising:
receiving a first command from the HMD based on the sensed met condition; and
modifying the digital content displayed on the HCD based on the received first command.
13 . The method of claim 11 , further comprising:
receiving a second command from the HCD based on the sensed met condition; and
modifying the digital content displayed on the HMD based on the received second command.
14 . The method of claim 11 , further comprising:
displaying additional digital content on the HCD with the additional digital content also being displayed simultaneously on the HMD; or
displaying additional digital content on the HCD without the additional digital content also being displayed simultaneously on the HMD.
15 . The method of claim 11 , further comprising:
making a first modification of the digital content on one of the HMD and the HCD; and
simultaneously making a second modification of the digital content on the other of the HMD and the HCD, wherein the second modification is substantially the same as the first modification.
16 . The method of claim 11 , further comprising:
making a first modification of the digital content on one of the HMD and the HCD without modifying the digital content simultaneously on the other of the HMD and the HCD.
17 . The method of claim 11 , further comprising:
displaying additional digital content on the HCD; and
utilizing one or more applications on the HCD for processing the additional digital content.
18 . The method of claim 11 , wherein the one or more processors comprise:
a first processor embedded as a portion of the HMD,
a second processor embedded as a portion of the HCD, or
both the first processor and the second processor.
19 . A method of using an augmented reality (AR) system by a user in a surrounding environment, wherein the AR system includes a head-mounted display (HMD) having a first group of sensors configured for sensing a user action, a handheld computing device (HCD) having a second group of sensors configured for sensing the user action, and one or more processors operatively connected with one or more of the HMD or the HCD, the method comprising:
executing the user action by the user;
sensing the user action by:
the HMD, via the first group of sensors, or
the HCD, via the second group of sensors;
causing digital content to be displayed simultaneously on the HMD and on the HCD based on the sensed user action;
when the user action is sensed by the first group of sensors, modifying, by the one or more processors, the digital content displayed on the HCD; and
when the user action is sensed by the second group of sensors, modifying, by the one or more processors, the digital content displayed on the HMD.
20 . The method of claim 19 , wherein the one or more processors comprise:
a first processor embedded as a portion of the HMD,
a second processor embedded as a portion of the HCD, or
both the first processor and the second processor.