IP Library Granted Patent US 11,611,842
Granted Patent B2
US 11,611,842 · App. 17/258,829 · Granted Mar 21, 2023

Spatial augmentation

Inventors: Kari Juhani Jarvinen (Tampere, FI); Jussi Leppanen (Tampere, FI); Tapani Pihlajakuja (Vantaa, FI); Adriana Vasilache (Tampere, FI)
Assignee: Nokia Technologies Oy
H04S7/304G06F3/165G06F3/012G06T19/003G10L19/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,611,842
App. No.
17/258,829
Granted
Mar 21, 2023
Kind
B2
Abstract

An apparatus including circuitry configured for: obtaining media content, wherein the media content includes at least one object data; obtaining priority content information, the priority content information including a priority identification identifying and classifying the at least one object; rendering the at least one object based on the priority content information.

Claims (64)

1. An apparatus comprising

at least one processor and

at least one non-transitory memory including a computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:

obtain media content, wherein the media content comprises at least one object;

obtain priority content information, the priority content information comprising a priority identification identifying and classifying the at least one object; and

render the at least one object based on the priority content information, wherein the at least one object is controlled for rendering within one of at least two priority zones of an immersive environment represented with the media content based, at least partially, on the priority content information, wherein at least one of the at least two priority zones is associated with a priority class.

2. The apparatus as claimed in claim 1 , wherein the media content comprises at least one of:

audio content;

video content; or

image content.

3. The apparatus as claimed in claim 1 , wherein the media content comprises at least one spatial audio signal which defines an audio scene within which is at least one audio signal object, and wherein the at least one spatial audio signal is configured to be rendered according to a content consumer movement, wherein the priority content information is configured for identifying and classifying the at least one audio signal object, and rendering the at least one object comprises the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to:

render the at least one audio signal object based on the priority content information.

4. The apparatus as claimed in claim 3 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to:

decode from a first bit stream the at least one spatial audio signal comprising the at least one audio signal object.

5. The apparatus as claimed in claim 4 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to:

decode from the first bit stream the priority content information.

6. The apparatus as claimed in claim 1 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to:

obtain the priority content information from a content consumer input.

7. The apparatus as claimed in claim 1 , wherein the priority content information comprises at least one of:

an object identifier parameter for identifying which of the at least one object the priority information is for;

a priority class parameter for classifying the at least one object with respect to identifying a rendering effect to be applied to the at least one object;

a priority zone size parameter for defining a scene priority zone size associated with the rendering effect to be applied to the at least one object;

a priority zone radius parameter for defining a scene priority zone radius associated with the rendering effect to be applied to the at least one object;

a priority zone shape parameter for defining a scene priority zone shape associated with the rendering effect to be applied to the at least one object;

a trigger condition parameter for defining at least one criteria to meet before applying the rendering effect to the at least one object;

a duration parameter for defining a duration over which the rendering effect can be applied to the at least one object;

a priority object effect parameter defining the rendering effect to be applied to the at least one object; or

an other object effect parameter defining a rendering effect to be applied to another of the at least one object when the rendering effect is applied to the at least one object.

8. The apparatus as claimed in claim 1 , wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to:

define the at least two priority zones.

9. The apparatus as claimed in claim 1 , wherein rendering the at least one object based on the priority content information comprises the at least one memory and the computer program are configured to, with the at least one processor, cause the apparatus to:

move the at least one object from a first of the at least two priority zones to another of the at least two priority zones based on the priority content information, wherein the at least two priority zones comprise respective regions of the immersive environment surrounding a content consumer in the immersive environment.

10. The apparatus as claimed in claim 1 , wherein the priority content information comprises at least one of:

a first level, wherein an object, of the at least one object, associated with the first level is content locked to a content consumer's field of view;

a second level, wherein an object, of the at least one object, associated with the second level is moved to the content consumer's field of view;

a third level, wherein an object, of the at least one object, associated with the third level is highlighted when in the content consumer's field of view; or

a fourth default level, wherein an object, of the at least one object, associated with the fourth level is not modified based on its priority level.

11. An apparatus comprising

at least one processor and

at least one non-transitory memory including a computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:

obtain media content, wherein the media content comprises at least one object;

obtain priority content information, the priority content information comprising a priority identification identifying and classifying the at least one object; and

generate a data stream, the data stream comprising the media content and the priority content information, wherein the data stream is configured to be transmitted to and/or retrieved with a renderer configured to render the at least one object based on the priority content information, wherein the at least one object is configured to be controlled for rendering within one of at least two priority zones of an immersive environment represented with the media content based, at least partially, on the priority content information, wherein at least one of the at least two priority zones is associated with a priority class.

12. The apparatus as claimed in claim 11 , wherein the media content comprises at least one of:

audio content;

video content; or

image content.

13. The apparatus as claimed in claim 11 , wherein the media content comprises at least one spatial audio signal which is configured to be rendered according to a content consumer movement and defines an audio scene within which is at least one audio signal object, and wherein the priority content information is configured for identifying and classifying the at least one audio signal object, wherein the renderer is configured to render the at least one audio signal object based on the priority content information.

14. A method comprising:

obtaining media content, wherein the media content comprises at least one object;

obtaining priority content information, the priority content information comprising a priority identification identifying and classifying the at least one object; and

generating a data stream, the data stream comprising the media content and the priority content information, wherein the data stream is configured to be transmitted to and/or retrieved with a renderer configured to render the at least one object based on the priority content information, wherein the at least one object is configured to be controlled for rendering within one of at least two priority zones of an immersive environment represented with the media content based, at least partially, on the priority content information, wherein at least one of the at least two priority zones is associated with a priority class.

15. The method as claimed in claim 14 , wherein the media content comprises at least one spatial audio signal which defines an audio scene within which there is at least one audio signal object, and wherein the at least one spatial audio signal is configured to be rendered according to a content consumer movement, wherein the priority content information is configured for identifying and classifying the at least one audio signal object, and wherein the rendering of the at least one object comprises rendering the at least one audio signal object based on the priority content information.

16. The method as claimed in claim 14 , further comprising:

defining the at least two priority zones.

17. The method as claimed in claim 14 , wherein the rendering of the at least one object based on the priority content information further comprises:

moving the at least one object from a first of the at least two priority zones to another of the at least two priority zones based on the priority content information.

18. The method as claimed in claim 14 , wherein the obtaining of the priority content information comprises obtaining the priority content information from a content consumer input.

19. A method comprising:

generating media content, wherein the media content comprises at least one object;

obtaining priority content information, the priority content information comprising a priority identification identifying and classifying the at least one object; and

signalling the media content and/or the priority content information, wherein the media content and the priority content information are configured to be transmitted to and/or retrieved with a renderer configured to render the at least one object based on the priority content information, wherein the at least one object is configured to be controlled for rendering within one of at least two priority zones of an immersive environment represented with the media content based, at least partially, on the priority content information, wherein at least one of the at least two priority zones is associated with a priority class.

20. The method as claimed in claim 19 , wherein the media content comprises at least one spatial audio signal which is configured to be rendered according to a content consumer movement and defines an audio scene within which there is at least one audio signal object, and wherein the priority content information is configured for identifying and classifying the at least one audio signal object, wherein the renderer is configured to render the at least one audio signal object based on the priority content information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2021
From: JARVINEN, KARI JUHANI; LEPPANEN, JUSSI; PIHLAJAKUJA, TAPANI; VASILACHE, ADRIANA
To: NOKIA TECHNOLOGIES OY
Reel/Frame 056410/0717 →
Priority Claims (1)
GB 1811542 · Jul 13, 2018 · national
Continuity (1)
Related Publication 20210271315A1 · Sep 2, 2021