IP Library Granted Patent US 10,482,662
Granted Patent B2
US 10,482,662 · App. 15/199,519 · Granted Nov 19, 2019

Systems and methods for mixed reality transitions

Inventor: Glen J. Anderson (Beaverton, OR)
Assignee: Intel Corporation
G06T19/006A63F13/216A63F13/655G06F3/011A63F2300/8082G02B27/017G02B2027/014G02B2027/0138
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,482,662
App. No.
15/199,519
Granted
Nov 19, 2019
Kind
B2
Abstract

An immersive video display system includes a video display to display an image and a processor coupled with the video display. The displayed image includes an augmented reality image and a virtual reality image at different moments in time. The processor is to modify one of the augmented reality image and the virtual reality image to more closely correspond to the other of the augmented reality image and the virtual reality image, and control the video display to gradually transition from displaying the one of the augmented reality image and the virtual reality image to the other of the augmented reality image and the virtual reality image.

Claims (53)

1. An immersive video display system, the system comprising:

a video display to display, to a viewer, an augmented reality image in an augmented reality environment, or a virtual reality image in a virtual reality environment; and

a processor coupled with the video display, the processor to:

modify the augmented reality image or the virtual reality image to more closely correspond to the other, wherein portions of the virtual reality image are stretched by a percentage when the viewer's attention is focused on non-stretched portions of the virtual reality image, and wherein images in a foreground and background of the virtual reality environment are stretched less than images in an intermediate image based on a correlation coefficient with a higher correlation coefficient results in less stretching and a lower correlation coefficient results in more stretching; and

control the video display to gradually transition between the augmented reality image and the virtual reality image,

wherein the processor is to add a set of virtual objects in the virtual reality image that correspond with real-world objects in an environment within a line of sight of the video display that are visible in the augmented reality image while controlling the video display to transition from displaying the virtual reality image to displaying the augmented reality image.

2. The system of claim 1 , wherein the video display is incorporated into a head-mounted video display.

3. The system of claim 1 , further comprising a motion sensor to sense motion of a viewer of the video display, the processor further to modify the displayed augmented reality or virtual reality image in response to the sensed motion.

4. The system of claim 3 , the modifying of the displayed augmented reality or virtual reality image including transitioning from displaying the one of the augmented reality image and the virtual reality image to the other of the augmented reality image and the virtual reality image.

5. The system of claim 1 , further comprising a camera to receive an image of the environment within the line of sight of the video display, the processor further to modify the displayed augmented reality or virtual reality image to incorporate a representation of the image of the environment within the line of sight of the video display.

6. The system of claim 5 , the processor further to detect whether a person is adjacent a viewer of the video display using the camera and to determine whether to control the video display to gradually transition from displaying the one of the augmented reality image and the virtual reality image to the other of the augmented reality image and the virtual reality image based upon whether a person is detected to be adjacent the viewer.

7. The system of claim 6 , the processor further to, based upon a person being detected to be adjacent the viewer, control the video display to gradually transition from displaying the virtual reality image to the augmented reality image or to prevent transitioning from displaying the augmented reality image to the virtual reality image.

8. The system of claim 1 , further including a database of offset directions and amounts for transitioning between displaying the augmented reality image and the virtual reality image, the processor to look up an offset direction and amount from the database of offset directions and amounts and apply the looked-up offset direction and amount to one of the augmented reality image and the virtual reality image.

9. The system of claim 1 , the processor further to determine whether context indicates a near-term transition from displaying the one of the augmented reality image and the virtual reality image to the other of the augmented reality image and the virtual reality image, the context including at least one of a viewer's location, the viewer's activity, the viewer's posture, the viewer's calendar schedule, and the viewer's state of social interaction.

10. A method of transitioning between an augmented reality image and a virtual reality image, the method comprising:

displaying, to a viewer, an augmented reality image in an augmented reality environment, or a virtual reality image in a virtual reality environment on a video display;

modifying, by a processor, either the augmented reality image or the virtual reality image to more closely correspond to the other, wherein portions of the virtual reality image are stretched by a percentage when the viewer's attention is focused on non-stretched portions of the virtual reality image to provide perceptual compensation, and wherein images in a foreground and background of the virtual reality environment are stretched less than images in an intermediate image based on a correlation coefficient with a higher correlation coefficient results in less stretching and a lower correlation coefficient results in more stretching;

gradually transitioning between the augmented reality environment and the virtual reality environment by modifying the display of the augmented reality image or the virtual reality image to the other on the video display; and

adding a set of virtual objects in the virtual reality image that correspond with real-world objects in an environment within a line of sight of the video display that are visible in the augmented reality image while transitioning from displaying the virtual reality image to displaying the augmented reality image.

11. The method of claim 10 , further comprising:

sensing, by a motion sensor, a motion of a viewer of the video display, wherein the sensed motion includes sitting down; and

modifying the one of the augmented reality image and the virtual reality image in response to the sensed motion.

12. The method of claim 11 , wherein modifying the one of the augmented reality image and the virtual reality image includes transitioning from displaying the augmented reality image to displaying the virtual reality image.

13. The method of claim 10 , further comprising:

sensing, by a motion sensor, a motion of a viewer of the video display, wherein the sensed motion includes standing up; and

modifying the one of the augmented reality image and the virtual reality image in response to the sensed motion.

14. The method of claim 13 , wherein modifying the one of the augmented reality image and the virtual reality image includes transitioning from displaying the virtual reality image to displaying the augmented reality image.

15. The method of claim 10 , further comprising:

sensing, by a motion sensor, a motion of a viewer of the video display, wherein the sensed motion includes turning a head of the viewer; and

modifying the one of the augmented reality image and the virtual reality image in response to the sensed motion, wherein modifying the one of the augmented reality image and the virtual reality image includes displaying a different portion of a virtual or augmented reality environment corresponding to the displayed augmented reality or virtual reality image consistent with virtually turning a head of a virtual viewer within the virtual or augmented reality environment.

16. The method of claim 10 , further comprising:

receiving, via a camera, an image of the environment within the line of sight of the video display;

modifying the displayed augmented reality or virtual reality image to incorporate a representation of the image of the environment within a line of sight of the video display; tracking, by a processor, at least one of real-world spaces, objects, and people in the environment within the line of sight of the video display, using the received image of the environment; and

revising, by a processor, at least one of a displayed appearance and location of spaces, objects, and people in the displayed augmented reality or virtual reality image corresponding to the tracked at least one of real-world spaces, objects, and people, using the received image of the environment.

17. The method of claim 10 , further comprising:

receiving, via a camera, an image of the environment within the line of sight of the video display;

modifying the displayed augmented reality or virtual reality image to incorporate a representation of the image of the environment within the line of sight of the video display;

detecting, by a processor, a real-world object in the environment within the line of sight of the video display; and

aligning, by a processor, a corresponding object in the displayed virtual reality image with the real-world object while the displayed virtual reality image gradually transitions to the displayed augmented reality image.

18. The method of claim 10 , further comprising:

looking up, by a processor, an offset direction and amount from a database of offset directions and amounts for transitioning between displaying the augmented reality image and the virtual reality image; and

applying, by a processor, the looked-up offset direction and amount to one of the augmented reality image and the virtual reality image.

19. The method of claim 10 , further comprising determining, by a processor, whether context indicates a near-term transition from displaying the one of the augmented reality image and the virtual reality image to the other of the augmented reality image and the virtual reality image, the context including at least one of a viewer's location, the viewer's activity, the viewer's posture, the viewer's calendar schedule, and the viewer's state of social interaction.

20. At least one non-transitory machine-readable medium including instructions, which when executed by a machine, cause the machine to perform operations comprising:

displaying, to a viewer, an augmented reality image in an augmented reality environment, or a virtual reality image in a virtual reality environment on a video display;

modifying either the augmented reality image or the virtual reality image to more closely correspond to the other, wherein portions of the virtual reality image are stretched by a percentage when the viewer's attention is focused on non-stretched portions of the virtual reality image to provide perceptual compensation, and wherein images in a foreground and background of the virtual reality environment are stretched less than images in an intermediate image based on a correlation coefficient with a higher correlation coefficient results in less stretching and a lower correlation coefficient results in more stretching;

gradually transitioning between the augmented reality environment and the virtual reality environment by modifying the display of the augmented reality image or the virtual reality image to the other on the video display; and

adding a set of virtual objects in the virtual reality image that correspond with real-world objects in an environment within a line of sight of the video display that are visible in the augmented reality image while transitioning from displaying the virtual reality image to displaying the augmented reality image.

21. The non-transitory machine-readable medium of claim 20 , further comprising aligning, by a processor, a size and location of a background space in the virtual reality image with a corresponding background space in the augmented reality image.

22. The non-transitory machine-readable medium of claim 20 , further comprising adding, by a processor, a set of virtual objects in the virtual reality image that correspond with real-world objects in the environment within the line of sight of the video display that are visible in the augmented reality image while transitioning from displaying the virtual reality image to displaying the augmented reality image.

23. The non-transitory machine-readable medium of claim 20 , further comprising altering, by a processor, an appearance of an aspect of the augmented reality image to correspond with a corresponding aspect of the virtual reality image while transitioning from displaying the augmented reality image to displaying the virtual reality image.

24. The non-transitory machine-readable medium of claim 20 , further comprising elongating, by a processor, a foreground depth of the augmented reality image in relation to a point of view in the augmented reality image while transitioning from displaying the augmented reality image to displaying the virtual reality image.

25. The non-transitory machine-readable medium of claim 20 , further comprising elongating, by a processor, a foreground depth of the virtual reality image in relation to a point of view in the virtual reality image after transitioning from displaying the augmented reality image to displaying the virtual reality image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2022
From: INTEL CORPORATION
To: TAHOE RESEARCH, LTD.
Reel/Frame 061175/0176 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2016
From: ANDERSON, GLEN J
To: INTEL CORPORATION
Reel/Frame 039742/0549 →
Cited By (1)
US 12,684,099