IP Library Granted Patent US 9,615,177
Granted Patent B2
US 9,615,177 · App. 14/640,197 · Granted Apr 4, 2017

Wireless immersive experience capture and viewing

Inventors: Robert S. Englert (Jamesville, NY); Meyer J. Giordano (Syracuse, NY)
Assignee: SPHERE OPTICS COMPANY, LLC
H04R5/04G02B27/0093G02B27/017H04S7/302G02B2027/0178H04R2499/15
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,615,177
App. No.
14/640,197
Granted
Apr 4, 2017
Kind
B2
Abstract

Aspects capture, view, present and display photo-realistic, fully-immersive panoramic virtual environments, displaying to a user via a display screen, in a first layer at a first level of opacity, an image of current surroundings of the user from image data captured from the current surroundings of the user by an image capture device; and in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the user as a function of time or of place, wherein the second layer is located coextensive with the first layer. The first level of opacity is selected to present image data of the panoramic image within the first layer as transparent relative to image data of the panoramic image within the second layer; or as opaque relative to image data of the panoramic image within the second layer.

Claims (83)

1. A system, comprising:

a processor;

a computer readable memory in circuit communication with the processor;

a computer readable storage medium in circuit communication with the processor;

at least one image capture device in circuit communication with the processor; and

at least one display screen in circuit communication with the processor; and

wherein the processor executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:

displays via the at least one display screen, in a first layer at first level of opacity, an image of current surroundings of the system from image data captured by the at least one image capture device from the current surroundings of the system, wherein the image within the first layer is image data of a surrounding real time physical environment of the user while wearing the system;

displays via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the system as a function of time or of place, wherein the second layer is located coextensive with the first layer;

selects the first level of opacity to present image data of the image within the first layer from the group consisting of transparent relative to image data of the image within the second layer, thereby blending a streaming of real-time visual images of the surrounding real time physical environment of the user wearing the system with the second layer image data of the another environment, and opaque relative to image data of the image within the second layer;

in response to determining that the user is walking, presents the image data within the second layer at a first level of transparency; and

in response to determining that the user is stationary, presents the image data within the second layer at a second level of transparency that is more opaque than the first level of transparency.

2. The system of claim 1 , wherein the at least one image capture device is a front facing camera.

3. The system of claim 2 , wherein the image of the another environment is a 360 degree panoramic image construct; and

wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:

syncs turning movements of a head of the user to the left or to the right with corresponding movement through geographic features of the 360 degree panoramic image construct of the another environment; and

displays via the at least one display screen, in the second layer at the second level of opacity, portions of the image of another environment that correspond with a viewpoint of the user during the turning movements of the head of the user to the left or to the right.

4. The system of claim 3 , wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:

selects the first level of transparency to present real-time visual images of the surrounding real time physical environment of the user wearing the system to enable the user to see and walk through the physical environment of the user while simultaneously viewing the second layer image data of the 360 degree panoramic image construct of the another environment.

5. The system of claim 4 , wherein the processor executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:

in response to a determination that the user is walking and moving toward a collision with another object, presents the image data within the second layer at a third level of transparency that is more transparent than the first level of transparency.

6. A system, comprising:

a processor;

a computer readable memory in circuit communication with the processor;

a computer readable storage medium in circuit communication with the processor;

at least one image capture device in circuit communication with the processor;

at least one display screen in circuit communication with the processor;

at least one directional microphone in circuit communication with the processor; and

at least one speaker in circuit communication with the processor;

wherein the processor executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby:

displays via the at least one display screen, in a first layer at first level of opacity, an image of current surroundings of the system from image data captured by the at least one image capture device from the current surroundings of the system;

displays via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the system as a function of time or of place, wherein the second layer is located coextensive with the first layer; and

selects the first level of opacity to present image data of the image within the first layer from the group consisting of transparent relative to image data of the image within the second layer, and opaque relative to image data of the image within the second layer;

drives the at least one speaker to reproduce first sound signals that are detected from the current surroundings of the system by the at least one directional microphone, as a first surround sound representation of the first sound signals in a first volume level;

drives the at least one speaker to reproduce second sound signals that are detected from the another environment, as a second surround sound representation of the second sound signals in a second volume level;

sets the first volume level to be greater than the second volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as opaque relative to the image data of the image within the second layer; and

sets the second volume level to be greater than the first volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as transparent relative to the image data of the image within the second layer.

7. A computer program product for capturing, viewing, presenting and displaying photo-realistic, fully-immersive panoramic virtual environments, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising instructions for execution by a processor that cause the processor to:

display to a user via at least one display screen, in a first layer at a first level of opacity, an image of current surroundings of the user from image data captured by at least one image capture device from the current surroundings of the system, wherein the image within the first layer is image data of a surrounding real time physical environment of the user while wearing the system;

display via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the user as a function of time or of place, wherein the second layer is located coextensive with the first layer;

select the first level of opacity to present image data of the panoramic image within the first layer from the group consisting of transparent relative to image data of the panoramic image within the second layer, thereby blending a streaming of real-time visual images of the surrounding real time physical environment of the user wearing the system with the second layer image data of the another environment, and opaque relative to image data of the panoramic image within the second layer;

in response to determining that the user is walking, present the image data within the second layer at a first level of transparency; and

in response to determining that the user is stationary, present the image data within the second layer at a second level of transparency that is more opaque than the first level of transparency.

8. The computer program product of claim 7 , wherein the at least one image capture device is a front facing camera.

9. The computer program product of claim 8 , wherein the image of the another environment is a 360 degree panoramic image construct; and

wherein the computer readable program code instructions for execution by the processor further cause the processor to:

sync turning movements of a head of the user to the left or to the right with corresponding movement through geographic features of the 360 degree panoramic image construct of the another environment; and

display via the at least one display screen, in the second layer at the second level of opacity, portions of the image of another environment that correspond with a viewpoint of the user during the turning movements of the head of the user to the left or to the right.

10. The computer program product of claim 9 , wherein the computer readable program code instructions for execution by the processor further cause the processor to:

select the first level of transparency to present real-time visual images of the surrounding real time physical environment of the user wearing the system to enable the user to see and walk through the physical environment of the user while simultaneously viewing the second layer image data of the 360 degree panoramic image construct of the another environment.

11. The computer program product of claim 10 , wherein the computer readable program code instructions for execution by the processor further cause the processor to:

in response to a determination that the user is walking and moving toward a collision with another object, present the image data within the second layer at a third level of transparency that is more transparent than the first level of transparency.

12. A computer program product for capturing, viewing, presenting and displaying photo-realistic, fully-immersive panoramic virtual environments, the computer program product, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising instructions for execution by a processor that cause the processor to:

display to a user via at least one display screen, in a first layer at a first level of opacity, an image of current surroundings of the user from image data captured by at least one image capture device from the current surroundings of the system;

display via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the user as a function of time or of place, wherein the second layer is located coextensive with the first layer;

select the first level of opacity to present image data of the image within the first layer from the group consisting of transparent relative to image data of the image within the second layer, and opaque relative to image data of the image within the second layer;

drive at least one speaker to reproduce first sound signals that are detected from the current surroundings of the system by at least one directional microphone, as a first surround sound representation of the first sound signals in a first volume level;

drive the at least one speaker to reproduce second sound signals that are detected from the another environment, as a second surround sound representation of the second sound signals in a second volume level;

set the first volume level to be greater than the second volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as opaque relative to the image data of the image within the second layer; and

set the second volume level to be greater than the first volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as transparent relative to the image data of the image within the second layer.

13. A method for capturing, viewing, presenting and displaying photo-realistic, fully-immersive panoramic virtual environments, the method comprising:

displaying to a user via at least one display screen, in a first layer at a first level of opacity, an image of current surroundings of the user from image data captured from the current surroundings of the user by at least one image capture device, wherein the image within the first layer is image data of a surrounding real time physical environment of the user while wearing the system;

displaying via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the user as a function of time or of place, wherein the second layer is located coextensive with the first layer;

selecting the first level of opacity to present image data of the panoramic image within the first layer from the group consisting of transparent relative to image data of the panoramic image within the second layer, thereby blending a streaming of real-time visual images of the surrounding real time physical environment of the user wearing the system with the second layer image data of the another environment, and opaque relative to image data of the panoramic image within the second layer;

in response to determining that the user is walking, presenting the image data within the second layer at a first level of transparency; and

in response to determining that the user is stationary, presenting the image data within the second layer at a second level of transparency that is more opaque than the first level of transparency.

14. The method of claim 13 , wherein the at least one image capture device is a front facing camera.

15. The method of claim 14 , wherein the image of the another environment is a 360 degree panoramic image construct, the method further comprising:

syncing turning movements of a head of the user to the left or to the right with corresponding movement through geographic features of the 360 degree panoramic image construct of the another environment; and

displaying via the at least one display screen, in the second layer at the second level of opacity, portions of the image of the another environment that correspond with a viewpoint of the user during the turning movements of the head of the user to the left or to the right.

16. The method of claim 15 , further comprising:

selecting the first level of transparency to present real-time visual images of the surrounding real time physical environment of the user wearing the system to enable the user to see and walk through the physical environment of the user while simultaneously viewing the second layer image data of the 360 degree panoramic image construct of the another environment; and

in response to a determination that the user is walking and moving toward a collision with another object, presenting the image data within the second layer at a third level of transparency that is more transparent than the first level of transparency.

17. A method for capturing, viewing, presenting and displaying photo-realistic, fully-immersive panoramic virtual environments, the method comprising:

displaying via at least one display screen, in a first layer at first level of opacity, an image of current surroundings of the system from image data captured by the at least one image capture device from the current surroundings of the system;

displaying via the at least one display screen, in a second layer at a second level of opacity, an image of another environment that differs from the current surroundings of the system as a function of time or of place, wherein the second layer is located coextensive with the first layer;

selecting the first level of opacity to present image data of the image within the first layer from the group consisting of transparent relative to image data of the image within the second layer, and opaque relative to image data of the image within the second layer;

driving at least one speaker to reproduce first sound signals that are detected from the current surroundings of the user by at least one directional microphone, as a first surround sound representation of the first sound signals in a first volume level;

driving the at least one speaker to reproduce second sound signals that are detected from the another environment, as a second surround sound representation of the second sound signals in a second volume level;

setting the first volume level to be greater than the second volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as opaque relative to the image data of the image within the second layer; and

setting the second volume level to be greater than the first volume level in response to selecting the first level of opacity to present the image data of the image within the first layer as transparent relative to the image data of the image within the second layer.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2016
From: RAM INDUSTRIAL DESIGN, INC.
To: SPHERE OPTICS COMPANY, LLC
Reel/Frame 037705/0597 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 6, 2015
From: ENGLERT, ROBERT S.; GIORDANO, MEYER J.
To: RAM INDUSTRIAL DESIGN, INC.
Reel/Frame 035100/0508 →
Continuity (2)
Provisional Application 61948741 · Mar 6, 2014
Related Publication 20150254882A1 · Sep 10, 2015