IP Library › Granted Patent US 11,006,091
Granted Patent B2
US 11,006,091 · App. 16/201,562 · Granted May 11, 2021

Opportunistic volumetric video editing

Inventors: Zhu Liu (Marlboro, NJ); Eric Zavesky (Austin, TX); David Crawford Gibbon (Lincroft, NJ); Behzad Shahraray (Holmdel, NJ); Tan Xu (Bridgewater, NJ)
Assignee: AT&T INTELLECTUAL PROPERTY I, L.P.
H04N13/122H04N13/117H04N13/383H04N13/388
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,006,091
App. No.
16/201,562
Granted
May 11, 2021
Kind
B2
Abstract

A processing system having at least one processor may detect a first object in a volumetric video that is a focus of a user viewing a presentation of the volumetric video, detect an obstruction of a view of the user of at least a portion of the first object in the volumetric video, and perform at least one manipulation of the presentation of the volumetric video to the user to mitigate the obstruction of the view.

Claims (59)

1. A method comprising:

detecting, by a processing system including at least one processor, that a first object in a volumetric video is a focus of a user viewing a presentation of the volumetric video;

detecting, by the processing system, a predicted obstruction of a view of the user of at least a portion of the first object that is the focus of the user in the volumetric video, wherein the predicted obstruction is detected based upon at least one of: a trajectory of the first object or a second object in the volumetric video, or an inspection of a subsequent portion of the volumetric video that follows a portion of the volumetric video that the user is viewing; and

performing, by the processing system in response to the detecting the predicted obstruction of the view, at least one manipulation of the presentation of the volumetric video to the user to mitigate the predicted obstruction of the view, wherein a viewing perspective of the user to the first object is unchanged via the at least one manipulation.

2. The method of claim 1 , wherein the predicted obstruction of the view comprises the second object in the volumetric video, wherein the at least one manipulation comprises:

increasing a transparency of the second object in the presentation of the volumetric video; or

removing the second object from the presentation of the volumetric video.

3. The method of claim 1 , wherein the predicted obstruction of the view comprises the second object in the volumetric video, wherein the at least one manipulation comprises:

shifting a position of the second object in the presentation of the volumetric video.

4. The method of claim 1 , wherein the at least one manipulation comprises:

shifting a position of the first object in the presentation of the volumetric video.

5. The method of claim 1 , wherein the at least one manipulation comprises:

rotating the first object from a recorded position within the volumetric video.

6. The method of claim 5 , wherein the predicted obstruction of the view is a predicted obstruction of a feature on an occluded side of the first object.

7. The method of claim 1 , wherein the at least one manipulation is in accordance with a preference of the user for at least one type of manipulation.

8. The method of claim 1 , further comprising:

presenting a notification via an endpoint device of the user of the detecting of the predicted obstruction of the view; and

receiving a selection of the at least one manipulation from the user via the endpoint device, wherein the performing the at least one manipulation is in accordance with the selection.

9. The method of claim 1 , wherein the first object being the focus of the user is detected from at least one of:

an eye gaze of the user;

a head movement of the user;

an verbal input of the user;

a gesture input of the user; or

an input of the user via a tactile interface.

10. The method of claim 1 , wherein the user is viewing the presentation of the volumetric video via an endpoint device comprising:

an augmented reality device; or

a virtual reality device.

11. The method of claim 1 , further comprising:

identifying a third object that is not the focus of the user.

12. The method of claim 11 , further comprising:

removing the third object from the presentation of the volumetric video.

13. The method of claim 11 , further comprising:

identifying an object type of the third object; and

replacing the third object in the presentation of the volumetric video with a model of the object type from a catalog of models of a plurality of different object types.

14. The method of claim 13 , wherein the processing system comprises a network-based processing system, and wherein the presentation of the volumetric video comprises a streaming to an endpoint device of the user of:

the volumetric video;

a three-dimensional rendering of the volumetric video in accordance with at least one perspective of the user within a space of the volumetric video; or

a two-dimensional rendering of the volumetric video in accordance with the at least one perspective of the user within the space of the volumetric video.

15. The method of claim 13 , further comprising:

storing the volumetric video with the model of the object type replacing the third object; and

providing, to a second user, a second presentation of the volumetric video with the model of the object type replacing the third object.

16. The method of claim 1 , further comprising:

modifying the volumetric video to include the at least one manipulation; and

storing the volumetric video that includes the at least one manipulation.

17. The method of claim 16 , further comprising:

providing, to a second user, a second presentation of the volumetric video that includes the at least one manipulation.

18. A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:

detecting that a first object in a volumetric video is a focus of a user viewing a presentation of the volumetric video;

detecting a predicted obstruction of a view of the user of at least a portion of the first object that is the focus of the user in the volumetric video, wherein the predicted obstruction is detected based upon at least one of: a trajectory of the first object or a second object in the volumetric video, or an inspection of a subsequent portion of the volumetric video that follows a portion of the volumetric video that the user is viewing; and

performing, in response to the detecting the predicted obstruction of the view, at least one manipulation of the presentation of the volumetric video to the user to mitigate the predicted obstruction of the view, wherein a viewing perspective of the user to the first object is unchanged via the at least one manipulation.

19. A device comprising:

a processing system including at least one processor; and

a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:

detecting that a first object in a volumetric video is a focus of a user viewing a presentation of the volumetric video;

detecting a predicted obstruction of a view of the user of at least a portion of the first object that is the focus of the user in the volumetric video, wherein the predicted obstruction is detected based upon at least one of: a trajectory of the first object or a second object in the volumetric video, or an inspection of a subsequent portion of the volumetric video that follows a portion of the volumetric video that the user is viewing; and

performing, in response to the detecting the predicted obstruction of the view, at least one manipulation of the presentation of the volumetric video to the user to mitigate the predicted obstruction of the view, wherein a viewing perspective of the user to the first object is unchanged via the at least one manipulation.

20. The device of claim 19 , wherein the user is viewing the presentation of the volumetric video via an endpoint device comprising:

an augmented reality device; or

a virtual reality device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 28, 2018
From: LIU, ZHU; ZAVESKY, ERIC; GIBBON, DAVID CRAWFORD; SHAHRARAY, BEHZAD; XU, TAN
To: AT&T INTELLECTUAL PROPERTY I, L.P.
Reel/Frame 047613/0468 →
Continuity (1)
Related Publication 20200169715A1 · May 28, 2020
Cited By (1)
US 12,417,601