IP Library › Granted Patent US 11,218,685
Granted Patent B2
US 11,218,685 · App. 16/981,785 · Granted Jan 4, 2022

Method, an apparatus and a computer program product for virtual reality

Inventor: Miska Hannuksela (Tampere, FI)
Assignee: Nokia Technologies Oy
H04N13/178G06T3/0087H04N13/111H04N13/128H04N13/161H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,218,685
App. No.
16/981,785
Granted
Jan 4, 2022
Kind
B2
Abstract

The invention relates to a solution wherein a bitstream defining a presentation is generated, the presentation comprising an omnidirectional visual media content and a visual overlay. A first relative distance of the omnidirectional visual media content and a second relative distance of the visual overlay are indicated in the bitstream. Metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units is also associated with the generated bitstream, wherein the scale is for deriving a binocular disparity for the visual overlay. The invention also concerns a solution for decoding the bitstream to obtain data for deriving binocular disparity for the visual overlay.

Claims (58)

1. A method comprising:

generating a bitstream defining a presentation, the presentation comprising an omnidirectional visual media content and a visual overlay comprising monoscopic content;

indicating in the bitstream a first relative distance of the omnidirectional visual media content;

indicating in the bitstream a second relative distance of the visual overlay;

associating, with the generated bitstream, metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units, wherein the scale is for deriving a binocular disparity for the visual overlay to enable the visual overlay to be displayed stereoscopically in the presentation; and

outputting the bitstream comprising the first and second relative distances and the metadata.

2. The method according to claim 1 , wherein the bitstream comprises a container file comprising an encapsulated presentation.

3. The method according to claim 1 , wherein the bitstream comprises a media description describing the presentation.

4. The method according to claim 1 , wherein the omnidirectional visual media content comprises omnidirectional video or omnidirectional image.

5. The method according to claim 1 , wherein the monoscopic content comprises one of the following: a monoscopic video, a monoscopic image, graphics, or text.

6. The method according to claim 1 , wherein the first relative distance comprises ‘1’ or multiple values indicated through a depth map.

7. The method according to claim 1 , wherein the second relative distance comprises one of the following: a norm of a normal vector of a plane on which the visual overlay is rendered; a vector intersecting an origin of a coordinate system and a reference point of a plane on which the visual overlay is rendered; or multiple values indicated through a depth map.

8. The method according to claim 1 , further comprising:

indicating a third relative distance of cube faces for monoscopic omnidirectional visual media content of a cube map projection format, the third relative distance being applicable for directly using the cube map as a rendering mesh, or

indicating that the first relative distance is applicable for directly using the cube map as a rendering mesh.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code; the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:

generate a bitstream defining a presentation, the presentation comprising an omnidirectional visual media content and a visual overlay comprising monoscopic content;

indicate in the bitstream a first relative distance of the omnidirectional visual media content;

indicate in the bitstream a second relative distance of the visual overlay;

associate, with the generated bitstream, metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units, wherein the scale is for deriving a binocular disparity for the visual overlay to enable the visual overlay to be displayed stereoscopically in the presentation; and

outputting the bitstream comprising the first and second relative distances and the metadata.

10. The apparatus according to claim 9 , wherein the bitstream comprises a container file comprising an encapsulated presentation.

11. The apparatus according to claim 9 , wherein the bitstream comprises a media description describing the presentation.

12. The apparatus according to claim 9 , wherein the omnidirectional visual media content comprises omnidirectional video or omnidirectional image.

13. The apparatus according to claim 9 , wherein the monoscopic content comprises one of the following: a monoscopic video, a monoscopic image, graphics, or text.

14. The apparatus according to claim 9 , wherein the first relative distance comprises ‘1’ or multiple values indicated through a depth map.

15. The apparatus according to claim 9 , wherein the second relative distance comprises one of the following: a norm of a normal vector of a plane on which the visual overlay is rendered; a vector intersecting an origin of a coordinate system and a reference point of a plane on which the visual overlay is rendered; or multiple values indicated through a depth map.

16. The apparatus according to claim 9 , wherein the apparatus is further configured to indicate a third relative distance of cube faces for monoscopic omnidirectional visual media content of a cube map projection format, the third relative distance being applicable for directly using the cube map as a rendering mesh, or indicating that the first relative distance is applicable for directly using the cube map as a rendering mesh.

17. A method comprising:

receiving a bitstream;

determining from the bitstream a presentation, the presentation comprising an omnidirectional visual media content and a visual overlay comprising monoscopic content;

determining from the bitstream a first relative distance of the omnidirectional visual media content;

decoding from the bitstream a second relative distance of the visual overlay;

decoding from the bitstream metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units;

using the scale to derive binocular disparity for the visual overlay to enable the visual overlay to be displayed stereoscopically in the presentation; and

outputting information to enable the presentation to be displayed.

18. An apparatus comprising:

at least one processor, and

memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

receiving a bitstream;

determine from the bitstream a presentation, the presentation comprising an omnidirectional visual media content and a visual overlay comprising monoscopic content;

determine from the bitstream a first relative distance of the omnidirectional visual media content;

decode from the bitstream a second relative distance of the visual overlay;

decode from the bitstream metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units;

use the scale to derive binocular disparity for the visual overlay to enable the visual overlay to be displayed stereoscopically in the presentation; and

outputting information to enable the presentation to be displayed.

19. An apparatus according to claim 18 , further comprising computer program code configured to cause the apparatus to:

decode a third relative distance of cube faces for monoscopic omnidirectional visual media content of a cube map projection format, the third relative distance being applicable for directly using the cube map as a rendering mesh, or

decode the first relative distance is applicable for directly using the cube map as a rendering mesh; and

use the cube map as a rendering mesh in rendering.

20. A computer program product embodied on a non-transitory computer readable medium, comprising computer program code configured to, when executed on at least one processor, cause an apparatus or a system to:

generate a bitstream defining a presentation, the presentation comprising an omnidirectional visual media content and a visual overlay comprising monoscopic content;

indicate in the bitstream a first relative distance of the omnidirectional visual media content;

indicate in the bitstream a second relative distance of the visual overlay;

associate, with the generated bitstream, metadata indicative of a scale applicable to convert the first relative distance and the second relative distance to real-world distance units, wherein the scale is for deriving a binocular disparity for the visual overlay to enable the visual overlay to be displayed stereoscopically in the presentation; and

outputting the bitstream comprising the first and second relative distances and the metadata.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 2, 2020
From: HANNUKSELA, MISKA
To: NOKIA TECHNOLOGIES OY
Reel/Frame 053957/0994 →
Priority Claims (1)
FI 20185288 · Mar 28, 2018 · national
Continuity (1)
Related Publication 20210112233A1 · Apr 15, 2021