Augmented reality display with frame modulation functionality

1. A head mounted display system comprising:

a frame configured to be supported on a head of the user;

a display disposed on the frame, said display configured to project light associated with a virtual image into said user's eye to display virtual image content to the user and further configured to transmit light from an environment to the user's eye to provide a view of a portion of the environment to the user;

a camera configured to obtain images of the environment at a frame rate, said camera having a field of view; and

processing electronics configured to receive and process images obtained by the camera, wherein said image processing comprises:

assessing relative motion between said head mounted display and one or more features in said environment, said assessment of relative motion comprising determining whether the head mounted display has moved, is moving or is expected to move with respect to one or more features in the environment and/or determining whether one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display;

determining one or more virtual image content locations in the environment corresponding to a location where renderable virtual image content appears to a user when the location appears in the display;

comparing said one or more virtual image content locations in the environment with a viewing zone that includes at least a portion of said field of view of said camera; and

based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content locations in the environment with the viewing zone, altering the frame rate of the camera from a first frame rate to a second frame rate and/or adjusting the amount of processing on the frames obtained by the camera that are processed,

wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based on a determination (i) that the head mounted display has moved, is moving, or is expected to move with respect to one or more features in the environment or that said one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display and (ii) that said one or more virtual image content locations in the environment includes a location within the viewing zone.

2. The system of claim 1 , further comprising a motion sensor in communication with said processing electronics, wherein the motion sensor is configured to perform measurements for the assessment of relative motion.

3. The system of claim 2 , wherein said motion sensor comprises an inertial measurement unit (IMU).

4. The system claim 1 , wherein said assessment of relative motion is based at least in part on measurement of an angular or linear acceleration of said head mounted display.

5. The system of claim 1 , wherein said assessment of relative motion is based at least in part on measurement of an angular or linear velocity of said head mounted display.

6. The system of claim 1 , wherein said assessment of relative motion is based at least in part on measurement of an angular or linear acceleration of said one or more features in said environment.

7. The system of claim 1 , wherein said assessment of relative motion is based at least in part on measurement of an angular or linear velocity of said one or more features in said environment.

8. The system of claim 1 , wherein said assessment of relative motion comprises determining that motion or expected motion of the head mounted display exceeds a threshold.

9. The system of claim 1 , wherein said assessment of relative motion comprises determining that motion or expected motion of said one or more features in said environment exceeds a threshold.

10. The system of claim 1 , wherein the second frame rate is lower than the first frame rate such that the camera is configured to operate with a reduced frame rate based on said assessment of relative motion.

11. The system of claim 1 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based further on a determination (iii) that said one or more virtual image content locations in the environment does not include a location within the near field of regard.

12. A head mounted display system comprising:

a frame configured to be supported on a head of the user;

a display disposed on the frame, said display configured to project light associated with a virtual image into said user's eye to display virtual image content to the user and further configured to transmit light from an environment to the user's eye to provide a view of a portion of the environment to the user;

a camera configured to obtain images of the environment at a frame rate, said camera having a field of view; and

processing electronics configured to receive and process images obtained by the camera, wherein said image processing comprises:

assessing relative motion between said head mounted display and one or more features in said environment, said assessment of relative motion comprising determining whether the head mounted display has moved, is moving or is expected to move with respect to one or more features in the environment and/or determining whether one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display;

determining one or more virtual image content locations in the environment corresponding to a location where renderable virtual image content appears to a user when the location appears in the display;

comparing said one or more virtual image content locations in the environment with a viewing zone that includes at least a portion of said field of view of said camera; and

based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content locations in the environment with the viewing zone, altering the frame rate of the camera from a first frame rate to a second frame rate and/or adjusting the amount of processing on the frames obtained by the camera that are processed,

wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is not reduced if said one or more virtual image content locations in the environment includes a location in a near field of regard region.

13. A head mounted display system comprising:

a frame configured to be supported on a head of the user;

a display disposed on the frame, said display configured to project light associated with a virtual image into said user's eye to display virtual image content to the user and further configured to transmit light from an environment to the user's eye to provide a view of a portion of the environment to the user;

a camera configured to obtain images of the environment at a frame rate, said camera having a field of view; and

processing electronics configured to receive and process images obtained by the camera, wherein said image processing comprises:

assessing relative motion between said head mounted display and one or more features in said environment, said assessment of relative motion comprising determining whether the head mounted display has moved, is moving or is expected to move with respect to one or more features in the environment and/or determining whether one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display;

determining one or more virtual image content locations in the environment corresponding to a location where renderable virtual image content appears to a user when the location appears in the display;

comparing said one or more virtual image content locations in the environment with a viewing zone that includes at least a portion of said field of view of said camera; and

based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content locations in the environment with the viewing zone, altering the frame rate of the camera from a first frame rate to a second frame rate and/or adjusting the amount of processing on the frames obtained by the camera that are processed,

wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based on a determination (i) that the head mounted display has moved, is moving, or is expected to move with respect to one or more features in the environment or that said one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display and (ii) that said one or more virtual image content locations in the environment includes a location in a far field of regard that is (i) beyond the viewing zone and (ii) beyond a near field of regard region.

14. The system of claim 1 , wherein the second frame rate is lower than the first frame rate.

15. The system of claim 1 , wherein the system is configured, based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content location with said viewing zone, to process data from fewer frames obtained by the camera.

16. A head mounted display system comprising:

a frame configured to be supported on a head of the user;

a display disposed on the frame, said display configured to project light associated with a virtual image into said user's eye to display virtual image content to the user and further configured to transmit light from an environment to the user's eye to provide a view of a portion of the environment to the user;

a camera configured to obtain images of the environment at a frame rate, said camera having a field of view; and

processing electronics configured to receive and process images obtained by the camera, wherein said image processing comprises:

assessing relative motion between said head mounted display and one or more features in said environment, said assessment of relative motion comprising determining whether the head mounted display has moved, is moving or is expected to move with respect to one or more features in the environment and/or determining whether one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display;

determining one or more virtual image content locations in the environment corresponding to a location where renderable virtual image content appears to a user when the location appears in the display;

comparing said one or more virtual image content locations in the environment with a viewing zone that includes at least a portion of said field of view of said camera; and

based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content locations in the environment with the viewing zone, altering the frame rate of the camera from a first frame rate to a second frame rate and/or adjusting the amount of processing on the frames obtained by the camera that are processed,

wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based on a determination (i) that the head mounted display has moved, is moving, or is expected to move with respect to one or more features in the environment or that said one or more features in the environment have moved, are moving or are expected to move relative to the head mounted display and (ii) that said one or more virtual image content locations in the environment does not include a location outside the viewing zone.

17. The system of claim 1 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is adjusted based on a determination of limited movement or expected movement of the head mounted display with respect to one or more features in the environment or of limited movement or expected movement of one or more features in the environment relative to the head mounted display.

18. The system of claim 1 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based on a determination that motion or expected motion of the head mounted display does not exceed a threshold.

19. The system of claim 13 , wherein reducing the amount of processing of the frames from the camera comprises reducing the number of frames captured by the camera that are processed.

20. The system of claim 12 , wherein the second frame rate is lower than the first frame rate.

21. The system of claim 12 , wherein the system is configured, based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content location with said viewing zone, to process data from fewer frames obtained by the camera.

22. The system of claim 12 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is adjusted based on a determination of limited movement or expected movement of the head mounted display with respect to one or more features in the environment or of limited movement or expected movement of one or more features in the environment relative to the head mounted display.

23. The system of claim 13 , wherein the system is configured, based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content location with said viewing zone, to process data from fewer frames obtained by the camera.

24. The system of claim 13 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is adjusted based on a determination of limited movement or expected movement of the head mounted display with respect to one or more features in the environment or of limited movement or expected movement of one or more features in the environment relative to the head mounted display.

25. The system of claim 16 , wherein the second frame rate is lower than the first frame rate such that the camera is configured to operate with a reduced frame rate based on said assessment of relative motion.

26. The system of claim 16 , wherein the system is configured, based on (i) said assessment of relative motion between said head mounted display and one or more features in said environment and (ii) on said comparison of said one or more virtual image content location with said viewing zone, to process data from fewer frames obtained by the camera.

27. The system of claim 16 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is adjusted based on a determination of limited movement or expected movement of the head mounted display with respect to one or more features in the environment or of limited movement or expected movement of one or more features in the environment relative to the head mounted display.

28. The system of claim 16 , wherein the frame rate of the camera and/or the amount of processing on the frames obtained by the camera is reduced based on a determination that motion or expected motion of the head mounted display does not exceed a threshold.